PJ Belcher Pro Audio.

Since the release of Floe onto the App store, and seeing it’s growing success, I felt it would be interesting to detail the design process I went through to create the audio of Floe. Although it may appear a simple puzzle game there is more to it than meets the ear, or so to speak…

Although Floe was released early December this year (2010), I actually had a complete design in place sometime towards the end of May. To be honest we could have quite possibly shipped it then, but its all the additional development and refinement that makes a game so special and appear so polished.

When I get asked to work on a game the very first thing I will do is play it and comprise a list of every single event and situation that needs audio. Designing audio is no linear process, since the sound needs to be just as dynamic as the gameplay, so this adds an additional challenge and requires a lot of forward thinking. With this in mind I try to not work to any model when designing audio, and instead try to approach each project like a blank canvas with myself holding no pre-conceptions. This helps the audio become as fresh and innovative as the game itself, which in the iPhone market is key given the large amount of originality being developed for the app store.

It was very early on in the design process that I decided the game shouldn’t have any in game music. This may seem an odd choice for the composer, but even the world’s best pieces of music will eventually become repetitive if listened to enough times. And since Floe was a puzzle game, players could spend any amount of time on a single area. The second influencing factor was asset size and memory issues on the iPhone. This basically meant I was strictly limited in how many pieces of music I could include and how long they could be, which over the course of 48 levels would not see them stay “fresh” very long.

Clips of some of the music from Floe.

Obviously the playable elements of the game couldn’t just be left with event driven sound effects, so I set about producing an ambient sound track. The problem here was that in order for the ambience to be effective it has to be at the very least 30′s long to stand up to repetition, and even then that was questionable. Of course an additonal 30 second sound file (or bigger) was not an option. This led myself and our programmer (Gareth Rees) to develop a system that generated the ambience in real time based on a series of assets. The assets comprised of 6 short stereo PCM samples made up of various sounds, and 3 longer samples made up of just wave sounds. The total accumulated asset size was nowhere near 30 seconds of even compressed audio. The 6 samples where then played back at random intervals, while the 3 longer samples played continually one after the other, in a random order. Not only did we eliminate all of the memory issues, but due to the generative nature of the system, the sound was always random, sounded more realistic and never became repetitive.

An example recording of the ambience system.

At the time of starting this project I was heavily influenced by the works of Carl Stalling, the composer for Warner Bro’s Looney Tunes cartoons. Stalling realised when starting to compose for the series, that the Warner Bro’s orchestra was often sitting around going unused, so he deiced to compose original pieces of music (often based on parodies of existing themes) for use in the show. In addition to this was the lack of convincing sound design at the time, meaning soundscapes required a lot of work to become authentic sounding, or where simply left sounding bare. To combat this Stalling decided to accompany all events in the animation that would normally require a sound, with a short original piece of music, a musical gesture. Think of Wylie Coyote walking along on his tip-toes, or any character blinking their eyes, these where all backed by musical gestures. Of course this style of composition soon became synonymous not only with the Looney Tunes series but with cartoons in general. Given the slight cartoony nature of Floe I decided to adopt this form of composition and apply it to the game. Unfortunately several aspects got dropped over development, but some remain in the game released today. These include Flo’s movement being accompanied by a Marimba, the gesture when she gets squashed, a slide whistle as she is lifted and dropped by the balloon and a fan fare at level completion. These are all things that I would like to consider a “nod of the head” to Stalling’s composition style.

Actually implementing the Marimba sound to Flo’s movement was quite a tricky process. For it to appear convincing, the sound had to be carefully matched to Flo’s movement, which could be of a wide variety of lengths. For this to be one completely generatively was not an option, since I wanted each musical gesture to be carefully composed, much like what Stalling produced, and not just random. I’m aware some generative systems exist that can accurately re-produce works in the style of great composers, but the development cost of this far out weighed the system I produced. Instead the potential length of Flo’s movement was split into 4 categories, short, medium, long and extra long. I then set about composing a variety of gestures set to the length of each category. What proceeded in game was quite clever, as you asked Floe to move it would measure the distance of travel, pick the appropriate category, and randomly select one of the pre-composed gestures from that bank. What we ended up with was a series of musical gestures cherry picked for each appropriate use. This careful compromise between thoughtful composition and dynamic generation emulated the sound perfectly, and with little cost.

Some examples of the Musical Gestures associated with Floe’s movement.

You may be thinking at this point that none of the music sounds much like Stalling? Well this was due to a change in design part way through the development process. Originally I composed in a style that emulated Stalling, arrangement, orchestration and production values. The design change was quite simply to make the game sound cuter! This is where the tuned percussion sound you hear now came in, cellos and brass were swapped for marimbas and glockenspiels, in a far more focussed arrangement with modern day production values. As much as I agreed with the change in direction I still wanted to keep in some of the Stalling spirit, which is why some elements still remain, carefully snuck in……

When it came down to giving Floe a voice I knew I had to use a real person. This may seem obvious, but a large amount of sound design comes from completely unrelated sources, as long as it can provide a decent base to work from all sorts of interesting sounds can be achieved. Most of my sound design comes from house hold objects, or from random library sounds. For instance I once created a gun shot from the sound of a football match, the key featured I required was a powerful transient in the sound file, and this came in the form of someone falling over. When it comes down to voices though I know there are certain nuances, dynamics and harmonics unique to the human voice that I wouldn’t find characterised anywhere else. I know Flo doesn’t actually say anything, and she is a polar bear, but she is the only protagonist of the game so she needs a voice that is convincing. Plus she walks around on her hind legs, trying to navigate a series of colourful blocks, so lacking realism aside, she does has human like features. With this in mind I set about writing up directions that would describe each of the sounds Flo had to make. These included “Your upset you can’t move to the place the user is asking you to move to” and “You just got moved by a block against your own will! Your surprised and annoyed by this”. I then relayed these requirements to a vocal actor whom proceeded to make a series of sounds. I edited and processed these to make the sounds of Flo.

Some examples of Flo’s voice.

The last thing I would like to mention about the development of Floe’s sound is the animal encounters. It was well towards the end of development that these were added and I was handed the task of giving them some sound. I decided the animals should sound like caricatures of the animals they are, while at the same time sounding like they are speaking, without actually using any words. This was tricky as I had to apply multiple design principles for several points, over the course of 4 types of animals. Each animal then had a baby counterpart, and spoke several lines of dialogue. To make matters worse I had just 2 days to do this.I had previously developed a technique while working with Introversion on Subversion. So I had my basic principle in place, but how would I apply to this the brief I had set myself in Floe. I started by listening to how the animals sounded in reality, and then making a decision on how this could be if it were a cartoon and then creating that sound from what ever sources I could. This proved very tricky in some places. I then edited and arranged my creations to sound similar to spoken words based on tonality, before applying my gibberish technique. I created several variations for each animal and of course the baby versions, before I ended up with the finished product. The walrus’ are actually myself saying single unrelated words, and Flo was created from the samples of her original recording.

Excerpts of the animal vocalisations.

So was there more than meets the ear? How about the eyes? Our artist Ed Sludden has also created a similar article focussing on Floe’s visuals if you would like to go on exploring the development of Floe then follow the link to the Otterly website.. I hope you enjoyed reading, and I hope you enjoy playing Floe even more.

PJ Belcher