Quick tip: using templates in Wwise

February 6th, 2016

(this originally appeared as a post for designingsound.org)

Like any tool in a game developers toolbox, Wwise is a deep, complex program with an owners manual longer than most novels. Who has time to read through an entire manual these days? I wanted to show off a simple, often overlooked feature in Wwise, which may not be readily apparent to someone who hasn’t read the manual. The ability to import a folder structure and apply a Wwise structure as a template to it can save a ridiculous amount of time when setting up structures which may have a similar layout to other ones already in your project. With a little forethought and a few mouse clicks, the process of setting up complex structures in Wwise becomes an automated dream.

To start, let’s say we have a series of impact sounds which blend between soft and hard impacts based on the force of the impact and an additional layer of bounce sweeteners which only play above a certain force. We also do some filtering and pitch randomization based on the force and hardness of the objects colliding (via an RTPC). This is organized in Wwise as a blend container with some child random containers which each contain audio files:A blend container layout in Wwise we'll use as a template for importing a new structure. Click for a larger, readable version.

Now let’s start thinking about each of these structures as a folder. If we want to use this structure layout elsewhere in Wwise, we can “re-build” or emulate this structure layout in Windows using folders. Where we have structures in Wwise (their nomenclature for containers, actor mixers, virtual folders, etc.) we create folders in Windows which will serve as a guide for Wwise when we import new sounds. A Windows folder-based layout mirroring the impact structure above would look something like this:

Windows folder layout which corresponds to the structure layout you wish to emulate in Wwise

Similar to the blend container in Wwise example above, we have a “master folder,” in this case obj_sheet_metal_impact, which contains three folders: bounce, impact hard and impact soft and within each of those folders are corresponding wav files. With a folder structure in Windows that mirrors the structure we want in Wwise, we can import into Wwise complete with all of our bussing, RTPCs, crossfades, etc. created for us! (As an aside, I always build my folder structures directly in the Originals folder so that the organization is already in place without having to move wav files and folders around as an extra step in the File Manager).

Once your folders are laid out in a similar manner to your Wwise hierarchy, open the Audio File Importer in Wwise and click the “Add Folders” button. Navigate to the top level folder of your new structure, in my case “obj_sheet_metal_impact” and click “Select Folder.” This will open that folder in the Audio File Importer. You can now assign each folder level as a different structure in Wwise such as a random, sequence or blend container. The magic, however, happens when we click the arrow in the Template column and select “Browse” then navigate to an existing structure whose layout and parameters we want to emulate:

The Wwise template layout

As you can see Wwise automatically fills in which structure each folder should represent and even handles having more (or less) audio assets in a folder. Shuffle things around as/if needed, then click Import, and you’ll have a new structure mirroring the template structure complete with all rtpcs, crossfades, etc.

Our new structure with all template properties applied

Once we import the folder structure using an existing structure in Wwise as a template, we’re then free to tweak it to our hearts’ (or games’) content, but most of the grunt work has been taken care of through some simple folder organization. Happy templating!


Trip report: Satsop nuclear power plant (Glass recording)

October 18th, 2015

On October 6th, I had the privilege to join a few other sound designers (James Nixon, Kristoffer Larson, Pete Comley and Andy Martin) on a trip to the Satsop Nuclear Power Plant out in Elma, Washington about a half hour west of Olympia. While we bore a ridiculous number of radioactive jokes afterwards, the plant was never finished and thus never had any actual nuclear material near it. Construction began in the 70s during the energy crisis and in 1983, after falling $60 million over budget, they canned it. It was apparently about 75% complete. The county of Grays Harbor has now turned the complex into a business park, so there’s a handful of businesses out there and various films, music videos and performance groups also rent it out from time to time. James Nixon had wanted to record some large glass breaks, and when he opened up to the group to see if anyone would be interested in breaking shit at a nuclear power plant, the sentiment was “I would be completely insane to say no!”

We had the run of the place from 8am until 4pm on a gray, overcast Tuesday. I’ve seen the cooling towers from the freeway on the way to the Olympic Peninsula, but it was a completely different thing to be standing right under one. They were huge! We signed releases to climb up the stairs along the outside all the way to the top, but alas, we didn’t have time. We did walk around though, scouted out a space to do our glass breaking and also scouted some areas to capture impulse responses and the like. The picture below shows one of these spaces: what was meant to be a cap to an unfinished containment unit is now a parking garage with some insane reverbs inside.

Andy walking into the bat cave, aka the cap of a containment vat turned into a parking garage.

Andy walking into the bat cave, aka the cap of a containment vat turned into a parking garage.

We burned the morning trying to figure out how to rig the glass. James wanted the break sound isolated from the debris that would come with just throwing the panes on the ground. While he was working on figuring that out, Andy and I did a little exploration into a side alley which had some amazing reflections that changed radically from the entrance to the back of the alley. In the back of the alley were some 15 foot long pieces of rebar that we started playing with. Andy grabbed one and started dragging it along the floor and the sound was insane! So we recorded a bunch of that and some other rebar in the alley fun.

 

By then, we had gotten the maintenance crew to bring a man basket lift (a silly, sexist name if there ever were one) and we began figuring out how to get the glass set up so it could be broken safely with a few milliseconds between the impact and the shard fall. The plan was to have 2 people on the roof of the moving van we rented, but the roof was fiberglass with a few metal support ribs. Good thing Andy is a small human! He was volunteered to be the man on the truck breaking the glass. The rig called for a rope running from the truck with 2 suction cups attached to it, going up through the man basket and back down to the back of my car where we tied off so we could easily hoist new panes of glass up for each break. Andy would break each piece with a hammer or crowbar and await the next one to be hauled up.

We set up a ton of mics through Kristoffer’s Nomad recorder, 4 Sound Devices 702’s and a couple other field recorders. I think in the end we had a Sennheiser MKH8040 stereo pair on the truck pointing toward the impact. James and Pete and I each had 8060s pointed at the impact areas. We had Pete’s Neumann binaural head (aka Fritz) near the impact area as well as a few other close mics. I had a Shure KSM141 stereo pair pointed at a concrete wall just off from the impact zone to capture the reflections and Andy had his Omni M/S rig in the alley we were playing in. We also had a couple contact mics set up in interesting places (attached to a small satellite dish and one to the rental truck) which actually got some pretty decent recordings. Kristoffer ran the main recording area, capturing 8 tracks from around the impact site into his Nomad. Pete and James were both operating their booms, Andy was destroying, and I was tucked into the cab of the truck recording with my 8060 and monitoring Pete’s rig. In between breaks we all ran whatever tasks were needed: Pete would sweep out the existing glass, James would prep the next pane, Kristoffer logged the recordings and helped me with stopping some of the remote recorders, and I would stop the remaining remote recorders, get in my car to lower the rope and raise a new pane, then run around in a tizzy turning the recorders back on and hopping in the cab to do it again. Chaotic sonic fun!

Satsop is a business park and there’s some other industrial areas nearby including a power plant, so it is not a completely quiet place. We were plagued throughout the day by truck backup beeps, riding lawn mowers, strange radio squawks, wind, etc. But we got some surprisingly clean and diverse recordings of a broad range of glass from mirrors to plate glass to bottles to wine goblets (which made an awesome whooshing sound as they traveled through the air) and lightbulbs which sounded way more powerful and percussive than their size betrays).

It was a super successful day, but unfortunately we had no time left after breaking all the glass and cleaning up to record anything else. We’re planning another trip soon hopefully to capture impulse responses of various rooms, buildings and spaces (1000 foot tunnels!) and whatever else suits our fancy. Here’s a compilation of some of the breaks I captured with my Sony RX100 camera. The recordings via our Rycotes sound much cleaner (no wind) and as far as those truck beeps, well that’s what RX is for!


Best practices in using the PS4 Dualshock controller speaker

April 6th, 2015

The controller speaker in the Dualshock 4 is pretty damn cool, though it’s by no means the first of its kind. The Wii Remote famously had a speaker in it which was used occasionally. The archery and fishing sounds from Twilight Princess were the first time I really noticed its use. In the Wii version of Spider-man Web of Shadows we played Spider-man’s web flings out of it. It was a fun novelty, but not the best quality speaker. The Wii U has not one, but two speakers!  Stereo sound on a controller. Now that’s awesome! (or possibly overkill). The PS4 DualShock controller has a single speaker, but it’s a nice quality one (the same as the Vita speakers). In my time playing with it, I’ve come up with a set of best practices I would like to share. I think these concepts apply to the Nintendo controller speakers as well, and probably any environment where you have a “special” speaker close to the player, yet separate from the normal sound field:

 

Mind the (Latent) Gap

The Dualshock 4 controller connects via bluetooth and with bluetooth comes inherent latency. For this reason, you really shouldn’t try synching the controller speaker with the game speakers. It just won’t work consistently well. Maybe it will one time out of a hundred, but every other time, it’s going to be off by some amount, which can be a little disorienting. The discrepancy between the timing of the controller speaker and game speakers is fine for more amorphous sounds, but for anything the player needs sample accurate cognition of (like critical dialogue) choose one or the other. There are some really cool techniques you can do with dialogue, which I’ll touch on further down.

 

Treat it like an LFE

The LFE channel of a surround system is commonly known as the subwoofer (the speaker it plays out of), but LFE itself stands for Low Frequency Effect.  The key word here being “Effect.” If you’re constantly hitting the sub with sounds, not only does the mix start to feel muddy and fatiguing, but you also dilute the power of the LFE’s intended purpose: to emphasize key, special moments or events. I strongly believe the controller speaker should be used in the same manner. Make sure what you’re sending through it has purpose and reason. Generally speaking the best sounds to send through it are UI/notification sounds and “first person” sounds, or those that make sense to the player when they emanate directly in front of him/her instead of in the landscape of the room speakers. By no means are these the only categories of sounds you can use the speaker controller for, but it’s good practice to ensure you’re not breaking immersion through its use (unless of course that’s your intention).

 

Avoid using it for critical sounds

As designers, there are a lot of unknown factors we need to consider when deciding what to pump through the controller speaker. Listening environments vary greatly and sometimes the noise of a child crying, a dishwasher running, or a friend yammering endlessly about how awesome they are can completely overshadow the sounds coming through the controller speaker. Furthermore, users can adjust the speaker volume in the system menu, and while there are now ways to query that volume and ideally use that information to determine whether to route a sound through the controller speaker or the main mix, it bears considering that sounds you want to emanate from the controller speaker may not be heard by the player. For this reason, I recommend not using it for any critical sounds that the player absolutely must hear. Whether or not you follow this advice, always design a contingency plan for any controller sounds you want to ensure the player hears. If they’re using headphones, if the controller is turned down, etc. In a perfect world, the PS4 would know via its HDMI connection what the audio setup of the user currently is (headphones, stereo, 5.1, etc.), and with a microphone attached to the system, we could be sampling the ambient noise of the room and adjusting the mix dynamically as Rob Bridgett suggested in his recent GDC talk on adaptive loudness. If these two concepts were achieved, the engine could determine when to send your controller sounds to the controller speaker and when they need to be diverted to the main mix instead. But until we get there, have a plan in place for controller sounds the user must absolutely hear.

 

Be creative!

The speaker controller is a fun tool and can really add an extra level of immersion beyond the normal mix. We received a lot of positive feedback for our use of it in inFAMOUS Second Son: from the ball shakes of the graffiti can to the way we used it for draining powers (the drain sound started at the source of the power in the world and slowly moved into the controller speaker as Delsin absorbed the power), and there’s tons of other developers out there doing neat stuff with it. I loved how Transistor played the narrator’s voice through it (only if you select to use the speaker in the options menu), but still sent the reverb sends to the main mix. It created a fantastic sense of your sword intimately talking to you, but still being in the world (and by only having the dry mix go through either the mains or the controller they avoided the sync issues of sending the VO through both). The Last of Us Remaster did a similar feature with their audio logs. Shadow of Mordor had a great example of a first person notification in playing a bush rustle sound whenever the player would enter high grass. It helped communicate to the player that they were in cover using an in-world sound rather than a possibly-immersion-breaking UI sound. The bush rustle sound also brings up one last point: while it is a decent quality speaker, it’s still a small speaker in a plastic housing, best to keep it relegated primarily to mid and higher frequencies.

Perhaps we need to give the speaker controller a fancy acronym akin to LFE to help explain its best uses, something like Personal Mid-to-High-Frequency Effect (PMtHFE). Although that’s more syllables than “speaker controller,” so let’s just remember to use it wisely.


The Sound Design of inFamous Second Son: Video Powers

March 21st, 2015

Of all the powers in inFamous Second Son, Video powers may have been the most esoteric. I mean smoke at least has an analog in fire (and we used some fire elements in both the visual and sound design), but video? You think video, you may think laser, but we already had a neon power (which was even sometimes referred to as laser). So how the hell did we get something sounding as unique as our video powers without treading on the other power sets?

Part of the answer is interestingly with how the power set itself was initially conveyed to the team. Video power was actually called “TV power” internally for most of production. Heaven’s Hellfire, the video game that Eugene, the video power conduit, is obsessed with was initially a TV show. We realized after many months that it made more sense to make it into a video game instead and that would open up more avenues for us to play around with in the gameplay (such as the mildly retro boss battle).

But we still had “TV powers” stuck in our brain and when Andy and I began brainstorming about how to make sounds that were powerful and unique and “TV like” we started thinking about televisions. We stalked thrift stores around town hoping we’d come across some old 1970s vacuum or cathode tube televisions to take apart and record. We failed there, but Andy eventually came across a couple old CRT TV/VCR combos. Double obsolete points! We brought these into the studio and proceeded to record all kinds of sounds with an array of microphones from shotguns to contact mics to crappy telephone microphones which did an amazing job of capturing bizarre electromagnetic interference around the power supply, and other surfaces. We recorded all possible permutations of power on and power off sounds and even got the VCR mechanisms to give us some very bizarre whines and hums. We also did some recordings of the Sucker Punch MAME arcade cabinet which has a very old CRT monitor in it with tons of wires exposed, as well as a shortwave radio I’ve had for years, but never really needed for a video game sound before.

We recorded all of these sounds at 192kHz and the frequency content of the recordings on the CRT monitors at the higher frequencies was pretty astounding. While some of them we had to remove the >20kHz content to save our ears and speakers, Andy also did some pitch shifting to play around with some of these normally inaudible sounds and they became part of the video power palette.

A few words on the telephone microphones we used: they are cheap and really neat for recording electromagnetic interference. Although Radio Shack may be dead and gone now, you can still get them online. It’s pretty neat the wide array of sounds you can get from one of them by waving near essentially any power source from a monitor to a computer, plugs, etc. Basically any electronic device will give you some interesting content. For a lot of the TV powers, Andy took various EMF sounds and morphed them together using Zynaptiq’s Morph plugin.

So, similar to our other power sets, below is a video showing some of our field recording as well as the final in-game sounds.  What’s different here is that the video powers were finalized later in the project and we were so focused on finishing the game, that we did not make a fancy, fun video for the team. So, it may not be as fun as the previous videos, but still shows what we recorded and how it ended up sounding.

 


The Sound Design of inFamous Second Son: Concrete Powers

March 16th, 2015

It’s hard to believe that inFamous Second Son is a year old already!  I’ve been completely lagging on finishing up these posts about the powers design for the game, so let me use this opportunity to make good and present the first of the final 2 parts of this series. I will hopefully get around to posting my presentation on the Systems Design for the game soon as well so those who haven’t heard/seen it can have the information available to them. Anyway, on to the magic and mystery of concrete!

For those who haven’t played or seen inFamous Second Son you play a guy who gets superpowers battling an authoritarian government agency called the DUP whose soldiers are all imbued with concrete superpowers by their leader Dana Augustine (as normally happens with government agencies).

The biggest challenge for us with concrete was how to make it sound unique. It’s just rocks and stone right? We’ve all heard countless variations on rock sounds in everything from impacts to destruction and rubble/debris sounds. We needed to figure out ways to make our sounds stand out as unique, while also conveying the power of the enemies in the game who used concrete.

The powers ran the gamut from concrete grenades to spawning concrete shields to launching off spires of concrete and forming a concrete balcony on walls. In short there was tons of concrete objects being created and broken in the world. Not only did we need these to sound unique and “powered” but they also had to sound completely distinct from all the “normal” concrete in the world you could destroy or collide other objects with. It was a huge challenge, but one that Andy Martin was definitely up for.

The place to start, naturally, was by buying a bunch of concrete. I looked into the process of concrete, which is usually just a mixture of water, an aggregate like sand or gravel, and Portland cement (named after a type of stone used in the UK, not the sleepy hamlet of the Pacific Northwest of the US). While the thought of mixing up my own concrete sounded appealing to my construction worker wannabe side, we weren’t in a position in the project where we had limitless time to experiment. So we did the next best thing: went to Home Depot. Andy and I both made trips to the hardware store and bought all kinds of concrete and stone, from paver stones (which were often too resonant) to clay bricks, cinder blocks, and more. They were demolishing a building across the street from my house and I noticed some particularly large chunks of both asphalt and concrete sitting on the other side of the fence. I waited until nightfall, donned my ninja costume (really just a bathrobe with a scarf tied around my head) and absconded with the almost-final resources we would need to make our concrete powers come to life.

From here, Andy began to run wild and experiment with all kinds of torture he could enact on our various pieces of concrete. From scraping everything against the slabs from metal disks to binder clips to resonating a jews harp against them to, yes, crushing, beating and destroying, he created an elaborate and unique palette of concrete sounds. As a few of the characters in the game developed, their powers also evolved. Some characters now had “beams” of concrete they would shoot out to shield allies while another burrowed underground like Bugs Bunny on his way to Albuquerque, and another sat atop a giant swirling tornado of concrete chunks. We needed something unique here and I devised a way to record a constantly moving collection of some of the concrete chunks we had broken (and wrote up a blog post about it here).

Andy’s wizardry both in recording these sounds and in shaping them in ProTools and Wwise into the layers of concrete powers was top notch as always and now it was time to show the team what we’d been doing (and that our jobs are more fun than theirs). Below is another Sonic Equation of sorts which we showed at a company meeting demonstrating some of recording techniques used to make the concrete powers of Second Son:

 

Thanks again for reading. I hope to get a write-up of the video powers (which naturally entailed a lot of fun creative recording and manipulation) done next week in time for the proper anniversary of Second Son’s release. Stay tuned!