VR Video Games and Music Composition: an Interview with SkewSound
There are many moving pieces in the worlds and adventurous spaces of video games. Even the name used: video games: to designate a virtual space comes from a layered past of arcades, early computers, clunky headsets, and television consoles. In the last two decades these virtual worlds have taken on increasingly heightened dimensions and realities, as they enter the realms of VR, 360, and other augmented media types.
In today’s interview we’ll hear from two musicians and composers—Chris Wilson and Steve Pardo—working in video games & VR, members of the audio group SkewSound.
SkewSound is an audio studio creating authentic, signature music for video games, experiences in VR , and more traditional linear media. Its members are sound designers Nicholas Kallman and Dan Crislip, (who is also an audio director), and composers Chris Wilson and Steve Pardo. They’ve worked together on an incredible array of projects, such as Risk of Rain 2, Grim Dawn, Chariot, Rock Band VR, and Fantasia: Music Evolved.
SkewSound’s latest work can be heard in Echodog’s deck-building game, Signs of the Sojourner.
“I often want the music to feel like it’s a part of the world or the game, as if the characters might have written it themselves, because I think that it adds a layer of depth to not only the story, but helps the world to breathe a little bit.” — Chris Wilson
Andrew Cheek: Could you share a little about what goes into making a video game soundtrack for those unfamiliar with the process?
Chris Wilson: The first thing we do is talk in depth with the developers. The music is such an integral part of the story that it’s so important to learn as much as possible about the project, the expectations of the team, and the role the music will fill in the game. We’ll play builds of the project, study concept art, collect and listen to references we’ve asked for to hear examples of tone and direction, and brainstorm ideas for interactivity.
Once we feel like we have a good handle on how we want to approach the project, we’ll start doing some quick sketches and send them over to the team for feedback, a process we’ll repeat several times until we’re all happy with the direction; only then do we start writing in earnest.
After that it’s just an iterative process of creation, implementation, and feedback until we arrive at a soundtrack that sounds great, supports the story, compliments the gameplay, carries the right emotional weight, and is, most importantly, satisfying for the player. Once we get to that point we start polish passes, including rerecording sampled tracks with live players, more detailed mixing and mastering passes, preparing stems as needed for interactivity, and implementing everything in the engine.
“Still the job of the soundtrack remains, so the composer makes compositional and orchestrational choices that neither distract nor undersell the moment at hand.” — Steve Pardo
Andrew: What makes sound design and composition distinctive for VR games and other immersive VR experiences?
Steve Pardo: Creating audio content and implementing for traditional games and VR games certainly have a lot in common, but, from many perspectives, the immersive nature of a VR game requires more dynamics, subtlety, and realism.
Mimicking the acoustics and timbre of objects being manipulated in a VR space requires precise capture methods coupled with hyper-realistic reverb and delay systems. And, fully realizing the ambience of a given space requires rendering methods that can account for the six degrees of freedom players are afforded in VR. Thankfully, the primary stakeholders of the technology have delivered many of the tools necessary to support state-of-the-art audio technologies and file-formats, such as Dolby Atmos, ambisonics, real-time binaural rendering, audio propagation, and environmental modeling.
While that sounds quite technical, the truth of it is that it is an incredibly engaging assignment to fill out the world of a VR game: making creative decisions about what sounds should meet players’ expectations and which ones should break them. When you dial in the knobs just right, it feels like you are crafting a brand new world.
Andrew: One of your recent projects includes scoring the narrative deck-building game Signs of the Sojourner which is, interestingly enough, about conversation. Could you talk about that project a little and some of your intentions?
Steve Pardo: First of all, Signs of the Sojourner is such a unique game, with a conversational card game mechanic that alone is awe-inspiring coupled with a beautiful art style. Getting to write music for this world was a blast, and overall felt as if I was making music from a personally uncompromising place, as if I was making a record I had been wanting to make for a very long time.
The team’s vision for a moving narrative and a rich, multicultural world meant that the music should feel personal, human, and evolving. Each town has its own theme and orchestration, alongside character-specific instrumentation and mood shifts. But overall, the job of this soundtrack is to reflect the journey of the main character, starting from their hometown, moving further away, exploring wildly different cultures, conflicts, and technologies.
As a multi-instrumentalist, I pulled from a wide range of different sounds such as bass clarinet, charango, electric guitar effects, found-sound percussion, and a good old-fashioned Moog synthesizer. We also had a fantastic lineup of supporting Tennessee-based musicians such as Mike Baggetta and Jon Estes rounding out the vibe.
Andrew: How does the subject matter of a project you’re working on influence your experiments in sound, or approaches to musical technique?
Chris Wilson: For me, I often take cues from the game’s environment or style to give me a sense of the directions I want to explore. I often want the music to feel like it’s a part of the world or the game, as if the characters might have written it themselves, because I think that it adds a layer of depth to not only the story, but helps the world to breathe a little bit.
For example, we once worked on a game that took place in a run-down town in the swamps of Florida. I scoured Craigslist for guitars and drum sets that were old and in bad shape, because I wanted it to sound like the music was played on instruments that had been left in a dirty, humid attic and forgotten about for years, much like the town in the game had been.
Another aspect I’ll try to draw inspiration from is the characters themselves, their relationship, and how that informs gameplay. We worked on a game called Chariot, a really wonderful platformer, that you could play single or “couch co-op” as the developers put it. One aspect of our approach to it was to assign an instrument to each character: a flute for one and a violin for the other.
We would write a kind of base track for each level and over that we would play the more melodic phrases of the instrument that represents the character you were playing as. If you were playing co-op, both instruments would play their respective parts together, like a duet, complimenting each other just as the characters in the game must in order to be successful.
Andrew: Do you have any experience with spatial audio (like Meyers sound installations) or composing in 360 degrees? What’s that like?
Steve Pardo: We have worked and are working on a number of titles in the VR space, namely Rock Band VR (Harmonix/Oculus), Dance Central (Harmonix/Oculus), SAXOPHONE (The Pardo Brothers), and Fated (Frima), among others. Utilizing spatial audio for VR is almost a prerequisite at this point, as it is the primary way to convert traditional game audio best practices to a 360 degree environment.
There are a number of ways to capture source material for spatial audio. One great example is found in constructing an environmental ambience. A sound designer has the option of initially going out on a field trip with a field recorder and an ambisonic microphone, capturing raw 360 degree images such as wind, water, and interiors from scratch.
But one can also construct such an image in a DAW (digital audio workstation) utilizing spatial audio plugins alongside ambisonic, surround, stereo, and/or mono audio sources. Game engine tools can then render these 360 degree images in real-time, recognizing the perspective and location of the player in virtual space and applying panning, filtering, and attenuation appropriately in real-time for full immersion.
Music composition and implementation for VR requires a more dynamic and introspective approach. In traditional media, the soundtrack can take on a role that dictates how the viewer should feel and react to a given conversation or moment taking place.
But in VR, as the player is inhabiting a 360 degree environment, the player is afforded more access to the body language and facial expressions of the other virtual characters in the space, among other environmental cues that are attempting to truly mimic a real-life space.
Still, the job of the soundtrack remains, so the composer makes compositional and orchestrational choices that neither distract nor undersell the moment at hand. Personally, I find that I write more texturally, with an instrumentation that fits in with the sound design and environmental ambience.
Andrew: Do you (as a group, or individually) have any creative routines or habits you’d like to share?
Chris Wilson: For me, I think that not being afraid to put something away for a little while is really important. Sometimes I just don’t have the right inspiration or mindset, and trying to force myself through that is counter productive. Instead I’ll move to a different cue or project, or just take a step back from the whole process and take a walk with my dog.
Other times I get so far into the creative zone that I lose perspective, and I’ll make myself stop and sleep on it. Almost always the next day, with fresh ears and a fresh perspective, I find that a lot of the decisions I made the night before were not so great. Within SkewSound we also do a lot of peer feedback, where we’ll share what we’re working on with the rest of the team to get additional perspective on what we’re working on.
Andrew: What’s one piece of advice you’d give to an aspiring musician or producer interested in film and video games?
Steve Pardo: Beyond being able to execute at an extremely high-quality level, the primary reason a developer or director will be interested in working with you is due to your artistry and particular character that your talents would bring to the project.
It is certainly tempting to adapt your sound to fit within a particular mold that has found established success in the industry, and there is indeed a lot to garner from studying the output of others. But, at the end of the day, it will be more rewarding on many, many levels if what you brought to the table added to the conversation and culture.
With that being said, never forget that it is our job to help fully realize the vision of the project’s creative director. Maintaining a positive, friendly, patient attitude while being fully committed to delivering on the project’s vision can make all the difference.
Andrew: Is there anything else you’d like to add?
Chris Wilson: Don’t overlook the technical aspects of writing music for games. So many young composers spend all their time worrying about more traditional compositional concerns, such as melodies, progressions, and arrangements. But always bear in mind that how you are hearing your music as you write it is not how the player is going to hear it. They’ll hear it cut up, looped, and stemmed out.
Understanding how those music systems work in the game engine will help you write music that will be better suited to those systems. Better yet, learn to do your own implementation, so you know you have full control over the quality of what the player will eventually hear, rather than just writing cues and tossing them over the wall to the team’s technical sound designer.
*For more from SkewSound, check out their: website (it’s a cool site) // Signs of the Sojourner (available on Steam + Discord)
*Media courtesy of Impact24 PR
