Rhetorical Sound Design

Hearing is weird. It’s abstract in a way that sight isn’t. A picture can clearly communicate a sense of size and space. A series of pictures can communicate speed and distance. Sound is only movement. Almost any movement. It’s the vibrations people and things make when they pass through the air and come into contact with each other.

Hearing is also different from sight in part because we have less control over what we hear. We don’t open and close our ears, though we can try to block them. We don’t really focus our ears in the way that we do our eyes. We’re always hearing (as long as we are able to), and so we often become so used to sound that we don’t actively notice it unless we make the effort. We learn to tune a lot of sounds out, but instinctually notice when they are absent.

I think this is why sound design often goes unnoticed unless it is so incongruent that it breaks the audience’s immersion. Effective sound design sells the argument that what they are seeing with their eyes is real. It reinforces all of the concrete information about size, space, and action that they see on a screen. It is simply expected to be there and to sound “right.”

For that reason, it can be useful to have certain heuristics to apply to this problem; the problem of making things sound “right.” I’m loosely adapting Aristotle’s main rhetorical appeals—logos, ethos, and pathos—as a framework for thinking about effective sound design, with a particular focus on game audio. There is overlap between these appeals, because they are all fundamentally related (emotion and logic are never truly separate) and because each sound effect is essentially its own argument that should ideally succeed on multiple levels.

Pathos: The Emotional Appeal

An important function of any synchronized or reactive audio is to reinforce the emotional experience of the scene. This is where the role and function of sound design overlaps most with that of the musical score. Does an impact feel big? Does the gun the player is firing feel powerful? Does the giant monster they’re fighting feel enormous and deadly? Does that abandoned mansion feel haunted? This is the visceral, “game feel” component of game sound effects.

This has important implications for game design. Emotionally satisfying audio cues influence player behavior in a variety of ways.

  • A feeling of constant or impending danger can make players play slower and more cautiously.
  • A powerful-sounding weapon can inspire confidence and encourage players to be more aggressively.
  • A weak-sounding weapon might be used less often, regardless of its practical functionality.

Zander Hulme told a relevant story along these lines at a panel at PAX Aus 2016 about multiplayer weapon sound effects in a Wolfenstein game.

The players with the weaker-sounding weapon believed they were at a disadvantage and performed worse, despite both teams having functionally identical weapons. Replacing the weaker sound effects with something more satisfying fixed the perceived weapon imbalance. Game audio doesn’t simply play a passive support role in game design.

Logos: The Logical Appeal

Another important function in game audio in particular is the ability to communicate factual information to the audience. What exactly is making the sound? What direction is the sound coming from? From how far away? In what kind of space? Is the audience in that space or a different space? Can your audience discern all of these things or are they intended to? Lack of clarity and focus should be an intentional choice, not the result of carelessness or oversight.

Much like the emotional appeal, this too is a practical game design consideration. Audio information provided to the player can directly influence their decision-making and behavior in the game space, in a wide variety of contexts.

  • The recognizable sound of an enemy charging a powerful attack helps the player discern when to evade.
  • The distinct sound of a sniper rifle being fired makes them reconsider peeking around a corner.
  • The suddenly loud crack of their foot-steps on a tile floor tells them that sneaking will be difficult and may require them to slow down.
  • The clarity, volume, and propagation of sounds in competitive multiplayer games can significantly impact what kind of information players have about strategies of their opponents, even without line of sight.

In Counter-Strike, for example, players have to be mindful of moving at full speed, because running foot steps and jump landings can give away valuable information to their opponents with hearing and inform counter strategies. At the same time, being aware of this fact allows players to intentionally make noise to create misinformation.

Below is a clip of a CS:GO streamer, DaZeD, faking a drop by jumping on the ledge above. The opposing players throw a flash grenade and attempt to retake the room, expecting him to be below and blinded, but they don’t predict his superior positioning and lose the fight.

This only works because both teams are aware of the landing sounds and because these sounds are audible from positions outside of the room.

A subsequent update added unique landing sounds per surface, which complicates this scenario. In this clip, he actually jumps on a wood surface at the end of the upper tunnel. Now, an observant player could note that this surface sound effect is not what would they would hear when opposing players drop on the stone floor below. If he instead faked further to the left, the sounds would match as they did on older versions of the game.

Sound effects can provide extremely valuable information to players beyond the limitations of line of sight. It’s important to keep this in mind, even for members of the development team who don’t deal directly with audio. If footstep propagation distance determines when and where players can afford to move at full speed, this can influence how major routes through the map are designed. If this isn’t accounted for, it can have unintended consequences on player behavior and map flow. This applies in many other seemingly non-audio design contexts as well.

Ethos: The Appeal to Character

In the context of sound design, it’s useful to think of ethos as authenticity. Does the audience accept that this sound belongs in the space? Does it fit the art direction of the game? What stylistic considerations must be made to ensure that is the case? If the game is heavily stylized, there is plenty of room for stylized sound effects. If the game strives for pseudo-realism and photo-realistic graphics, it is probably appropriate to keep the sound effects relatively grounded. Often, however, what the audience expects is very different from  reality. Authenticity is what it seems like something should sound like, rather than necessarily what it actually does.

Practically, this has a large degree of overlap with Pathos, the emotional appeal, in that the most emotionally resonant sounds should also be authentic, but they are distinct. An ambience could be suitably unsettling, but not feel authentic in the wrong space. Creaking wood and howling wind might suit a creepy, old house, but be very much out of place in an abandoned space station, even though both evoke a lonely, isolated atmosphere. An impact could be distinct and punchy, but not fit the style of the game or the source object or actor.

Conclusion

A very common example of all of these elements in action is in effective gun shot sound effects, particularly for real world weapons. Fire arm field recordings on their own are rarely very interesting or particularly distinct. This is in part because of the difficulty in capturing the character and impact of sounds at extreme volume levels. Raw field recordings of fire arms tend to sound similar. To account for this, sound designers need create hyper-realistic gun shot sounds with a variety of explosive, mechanical, and environmental layers and processing in order to create the explosive, powerful sounds that audiences expect. This is both more authentic than a simple gun shot field recording, and more emotionally impactful. A core goal of satisfying weapon sounds is to recreate the visceral, explosive impact of firing them.

Given that, in situations where a large variety of weapons are called for, the sound designer will need to differentiate each of these weapons. This is especially true of games with a large selection of realistic weapons. It is important to both establish unique character for each and communicate that distinction to the player, who should ideally be able to tell what weapon is being fired at them from the sound. A sniper rifle might have an exaggerated, long reverb tell to really sell its firepower. Pistols and submachine guns might emphasize the mechanical elements over the explosive punch and the reverb tail to make it feel smaller. An assault rifle might lie somewhere in between.

Establishing these rhetorical choices and applying them consistently provides emotional satisfaction, authenticity, and clarity to the player.

Portfolios Require Planning

After quite some time struggling to put together materials from a bunch of different projects (most of which is either gone or broken, apparently), dating as far back as the spring of 2013, I’ve finally just gone ahead and finished a new project of my own that actually shows that I can do something: a Wwise implementation demonstration in Unity. Of particular note to me is the RTCP implementation for the player jet sounds, which modulates the volume and pitch of two separate layers based on the magnitude of the player ship’s movement vector (basically translates to speed).

I also took the opportunity to whip up a nifty animated logo with some punchy sound design to go with it. It’s pretty simple, but I’m not an experienced animator—by which I mean I have no experience whatsoever— so I had to start from scratch with what I had. I like it a lot.

Combustion Prototype Music

I recently got an opportunity to move a bit out of my comfort zone. It’s always a good exercise to explore different styles and aesthetics for sound and music. In this case, I tried to match the PSX-style, low poly, dieselpunk aesthetic of this project. I ended up playing around with the sounds and effects of the Korg M1 in an attempt to get a period (90s) appropriate, sampled instrument sound.

Misspelled my name in the video, though. 😉

Etrian Odyssey (How Atlus Got Me Into JRPGs)

The Nintendo DS was a system with a lot a great games. More importantly, and more relevant to this post, it had a lot of great RPGs. From remakes to ports to brand new games, it’s an understatement to say that the DS has an impressive library of fantastic niche titles (and no shortage of trash either). Nonetheless, for me, Etrian Odyssey was the first quintessentially “DS RPG.” It was the first RPG that felt like it could have been nowhere else. The main reason for this is the mapping system, which while perhaps outwardly gimmicky solidified the old-school pencil and graph-paper experience so often associated with the PC RPGs of yore, in a manner notably more tactile than the old school SMT/Persona experience. I mean Wizardry and the first person dungeon crawlers that followed it, the game that most obviously influenced the birth of the Japanese RPG along with Ultima.

But it was more than that. The mapping system wasn’t just some fleeting mechanical flourish incorporated to make the touch screen suddenly useful or a clever reference to the shared experience of certain older gamers that remember, perhaps fondly, the “good ol’ days” of PC RPGs. Ok. Sure. It may have been those things, but it was also your most important tool in a harsh and relentless labyrinth. It was how you kept track of the many dangers between you and the beautiful, but deadly depths of the twisting labyrinth and its many strata. It was how you remembered where that pool of healing water was to keep your party in fighting shape. It appealed to that irrational annoyance I have with auto-mapping systems that leave empty grid spaces, because I didn’t need to walk into that particular corner. It was the first ever present reminder that the dungeon space is always there. It was that small bit of control, that small way in which the player could touch the world that truly sparked the imagination, as they sketched the outline of a world on their virtual graph paper.

And imagination is a big part of the Etrian Odyssey experience, because you won’t find any cut scenes here. You won’t find paragraphs of exposition, exaggerated character archetypes vying for your attention and trying their best to make you believe they have depth and some meaningful emotional conflict that you should definitely care about. There is no overarching plot by some ridiculous villain to destroy the world so he can actually save it. There is no melodrama. The sole town and the NPCs in it are static hand-drawn images, with a simple interactive menu overlay. You get requests. You get missions. You complete them. And along the way, many small events add up to a memorable experience. Something hidden in a bush, a fruit plucked from a tree, a spring filled with healing water. A wild animal. It might be friendly; it might just eat your face.

There is so much that the game shows you and very little it feels obligated to outright tell you. For me, that made the story very personal, because I was the one who pieced all of those small moments together into a story. Each member of your guild is someone you gave a name, a face, and a job. Maybe you came up with a back-story for them from the beginning. Maybe one developed as you played the game. Maybe you named them after your real life friends. Maybe they remained a mystery, but they are YOUR characters. They represent YOUR agency in this mysterious place.

It’s the synergy between these characters and their skills that will keep them alive as you venture deeper (something that has been expanded and re-balanced as their series continues). Because of this, the bond between your guild members has a very practical foundation. The integrity of your five person team is essential to surviving the tougher parts of the game to a greater degree than most RPGs, regardless of your combination of character types. The loss of any of them, even with Revive spell or a nectar handy is a serious blow to the party’s fighting strength and something that can mean the swift end of a harder fight (and sometimes seemingly easier ones). For me, that simply served to further reinforce the importance of these characters to my experience as the game continued. Even though they never anything more than a series of portraits within a nest of menus, they feel like a close nit group of comrades, bonded by the dungeoneering experience they all share.

True to its roots, Etrian Odyssey is structured around repeated trips back to the labyrinth and slow, measured progress deeper and deeper into its twisted depths. It’s an unforgiving experience, entirely unafraid to decimate your party with a simple random encounter. Your support classes hide in the back row. Status effects are a VERY serious danger that you have to prepare for. You send your party in, explore until your resources run low (HP being one of the big ones) or your bag becomes stuffed to the bursting point with plant and animal parts, and then you return to the town.

You probably drop by Shilleka’s to offload your loot, the plant and animal parts you’ve harvested, restock on healing items and whatever new equipment can be hacked together from the latest hide you found, and you head to the Inn lick to your wounds (and save, of course). Maybe you’ll drop by the Golden Deer Pub that evening to relax, check with Valerie for any more jobs, and talk to some fellow adventurers, ex-adventures, and whoever else is carving out a life in the bustling town of Etria. These interactions aren’t in-depth, but they add just the right amount of detail to the place. The town theme quickly becomes a sound both familiar and safe, as it accents the peaceful beauty of that small settlement.

The next morning, you head back in, hoping you can make it further down this time and that those Venomflies you ran into on the second floor don’t decimate your party with their surprisingly potent poison. You fight your way through the various animals inhabiting the area, including some oddly dangerous rabbits that don’t seem to like you very much. There is a certain comfort in this routine and the satisfaction of making it just a little bit further into its depths, as punctuated by the new areas that you’ve mapped out yourself, is immense…

Then you meet your first FOE (Formido Oppugnatura Exsequens). Well, to be fair, this happens pretty much immediately on the second floor. I delayed it in this narrative for the sake of the dramatic twist. That’s just quality storytelling. You don’t have to tell me. I know you’re shocked and surprised.

FOE’s are basically bad-ass mini bosses that wander around the same space your party does. If you run into one, they will attack you and in all likelihood brutalize your woefully unprepared group of adventurers. The necessity of awareness and avoidance, which only becomes more complex deeper in, takes the tension of exploration up notch in a manner somewhat less psychologically distressing than a reflex-based game like Demon’s Souls or perhaps more appropriately, its first person predecessor: the King’s Field games. Things get more interesting a bit later, when you meet aggressive FOEs that will give chase if they spot you. It’s important that you always watch your map, because turns that pass in battle pass in the dungeon as well. If you get into a fight and you’re in the path of a FOE, they WILL join the battle and make your life a hell of a lot more difficult.

The major bosses are an extension of this, from Fenrir and its pack of wolf minions with no sense of fairness, to a dragon spewing its fiery breath around an entire dungeon floor. The sense of place, that they are moving through and interacting with the same area as your band of 5 intrepid adventurers, gives these towering beasts a presence they often lack in other similar experiences, despite the decidedly retro presentation. They loom over the battlefield even before the 3DS titles that put them in their fully rendered glory in the dungeon view. I can’t think of a more threatening videogame abstraction than that glowing orange ball, plodding towards you with every step you take, threatening to corner you if you make the wrong move. You can practically feel the ground rumble with every movement.

And, honestly, that’s pretty much the EO experience, without spoiling much. Things get hairier further down, with more environmental hazards and meaner monsters. The locale changes every 5 floors along with the excellent backing track, courtesy of Yuzo Koshiro. There might be a couple of unexpected narrative twists down the line. All told, it’s a pretty pure dungeon-crawling experience, but Etrian Odyssey never lets you forget that you are always in the dungeon. You aren’t in a mystical, abstract random encounter dimension and battles feel unusually immediate because of this, despite their turn-based foundations. The inter-play between the “real world” of the labyrinth and the pseudo-cyberspace of combat is a central theme. You are in Yggdrasil and it’s never truly safe until you’re out of it… But I always want to come back, because it’s such a curious place with so much to see and it doesn’t take hours of melodrama or a plethora of convoluted and ultimately irrelevant lore to make it engaging. That’s the magic of Etrian Odyssey and it’s why I love the series as a whole, but the first one is a special experience for me (the 3DS remake is great too).

Greetings

I’ve created this site in anticipation of the completion of several major projects in the next few months. As such, my portfolio is rather barren and this site is clearly a work in progress. It should progress very quickly.