(This post originally appeared on designingsound.org)
When we discuss what makes the sound in any medium: film, music, video game, theater, etc. effective, it usually boils down to one of four aspects, and often all four: the detail in the design, the emotion conveyed, the way the sound meshes with the visuals, and the mix of the sonic elements. When sound design exists on its own, with visuals and voiceover removed or non-existent, we lose some of these key elements (or should we call them crutches?) we can rely on to help express these concepts, especially emotion, since we process so much emotion via spoken word, visual imagery and music. So how do we bridge that gap and create effective pure sound design that can still evoke emotion? The answer may lie in the concepts behind the very elements we have removed.
Emotion
While we can consider emotion a sense of feeling, it is a fairly abstract concept to describe objectively, due to cultural and personal differences of perception.
Setting and ambience can go a long way in creating emotion, but it may not always have the intended effect due to the listener’s emotional conditioning. For example, if you create a beach scene complete with waves rolling up the sand, gulls crying, children laughing, etc., Some people may begin to relax because it reminds them of summer vacations or surfing every morning. Other people may have a more anxious response because they can’t swim, got stung by a jellyfish once, or saw too many Sharknado movies.
Even individual sounds can evoke an emotional response within different cultures which has been studied in sounds like birdsong and laughter/crying. As a (potentially fun, potentially unnerving) experiment, set a tea kettle to boil with other people around and once it starts screaming, watch how people react. Does their body language change? Do they ask you to turn it off? Or even run over to the stove to silence the scream? Singular sounds evoking certain emotions (if I just say “fingernails scraping on a chalkboard” do you hear the sound and shudder?) is well known, well documented, and well studied, but what about pure sound design? To incorporate emotions triggered by individual signature sounds we must weave a tapestry through our sound design, just like a successful mix in a movie, game or show; allowing design to rise and fall and evolve just as emotions do in real life.
Like reading a book or watching a movie, evoking emotion within pure sound design may involve empathy. When we cry in a movie, it’s often an empathic response to what the characters are going through. Perhaps it relates to a similar trauma we’ve experienced in our life, such as the death of a loved one, but it can also be a purely empathic response that we feel for this imaginary character due to their journey and our reaction to them as a character, or even a person. Since empathy is generally only triggered in response to other living creatures, it’s something that is only effective when dealing with creatures/beings/objects which are alive, at least within the scope of the project.
It’s also easier to create an emotional response when there is a narrative involved, but pure sound design doesn’t need a narrative to be successful and it doesn’t need a narrative to evoke emotion either. We don’t need an empathic response in a pure sound design piece even if our goal is to evoke emotion, but it is possible using vocal samples or recordings, or other sounds representative of a character’s emotional state.
Music
The easiest, most prevalent analog which pure sound design can follow to trigger an emotional response is music because, like pure music, pure sound design, has no accompaniment. All perception by the listener is based solely on the audio. Sometimes (and much more so with lyrical music) there is a narrative within music, but a narrative is not required to evoke emotion.
When we speak about music and the affect it has on us as humans, it nearly always involves some reference to the way music evokes emotion. There have been many studies demonstrating how music does this via brain function, as well as how individuals may perceive music differently or how entire cultures respond similarly. Famous examples in history include the riots after the debut of Stravinsky’s Rite of Spring or the myriad interpretations of Beethoven’s Ode to Joy since it’s release over 230 years ago.
The elements that make these pieces so powerful even so long after their release is in their use of musical structures, often in unique and novel ways. Elements like melody, tempo, harmony and dissonance can all play a part in producing a response from someone whether it is via music or pure sound design and there is a lot more to explore in using the language of music to apply to pure sound design.
With a wealth of reference and history in various forms of media, not only music, but film and game audio to radio dramas, there exists an entire toolkit we can use to evoke emotion and add detail via pure sound design. While most of these concepts often involve music or voiceover as the emotional triggers, similar emotions can be produced within a listener using pure sound design. In David Sonnenschien’s book Sound Design he includes a table created by Friederich Marpurg in the 18th century outlining musical techniques used to create various emotions. While Sonnenschien notes cultural bias and evolution may have altered this list over the last 200+ years, it’s still largely relevant, and furthermore I believe many of the techniques he notes for music can be applied to pure sound design as well.
Voice
A well acted line, be it in film, tv, theater or even video games, can bring a listener to laughter, tears, rage, or melancholy. A poorly acted line, usually just laughter. Even voice works on multiple levels for how it affects perception. On the one hand, you have the content of the line. What is being said with the dialogue and often how that plays into the emotions of the character (as well as any empathy we may or may not feel for that person). The other piece of the puzzle with voice, which can effectively be used in the realm of pure sound design, is the tone of delivery. How we deliver a line often has as much to do with how it is perceived as the actual words we use. A great, albeit perhaps simplistic, example of this is in talking to a dog. If you say “Good dog!” very happy and excitedly to a dog, it will wag it’s tail and behave happily. Guess what? If you say, “Stupid dog!” using the exact same tone you evoke the same behavior from the dog because the tone is what the animal is perceiving. The same goes for saying something positive with a loud, angry voice. Confuse your pets! It’s fun! The takeaway here though is that the tones we use in our speech patterns can be effectively mimicked in pure sound design as an additional tool to help our sound convey emotion.
Action!
Let’s put all this together and script out a potential pure sound design piece using elements and structures from music, voice, and neuroscience as an example. Let’s revisit our aforementioned tea kettle and craft a story via sound about how it lived a happy, serene life until someone built a fire under it. It slowly grew more and more agitated until it couldn’t take it anymore and snapped in a fit of rage. We begin with some gentle, rhythmic sloshing sounds with some metallic resonance. The kettle is at rest, relaxed, and the slow tempo of the slosh can help convey this sense of calm. We hear the gas burner light and slowly we begin to hear unexpected, dissonant sounds of arrhythmic metal pops and clanks and different volumes. Additional layers of a low roiling slowly begin to be introduced, with more layers perhaps with dissonant harmonic overtones. As these layers grow in volume, a low rumbling begins to eek its way into the mix. What’s happening? Suddenly— silence. Just for a beat, and then the telltale scream of the kettle, with some additional discomfiting layers, including perhaps, vocalizations of human or animal screams of rage and anger. Pure sound design, and the emotional journey of a tea kettle no less!
Pure sound design is such a fun, challenging endeavor because we lack many of the common references we use for sound design cues when visual media is involved. But at the same time it is also a remarkably liberating practice because not only does it involve an unbridled use of creativity, it also allows us to apply what we know about sound in novel ways to fill in the gaps we normally rely on visuals, voice and music to convey. Using some of the sonic properties from these elements we can build emotion into our pure sound design, if we so choose, as a way to make it even more effective. I would further posit that with the rise of VR and 3-d audio peripherals, enhanced positionality will make pure sound design even more effective since it creates a more immersive environment around the listener.