3D Audio: The Next Step in Realism for Gaming

Abstract

When using 3D audio in games, the goal should be to create an immersive, believable auditory experience. To achieve this on the technical side of things, game developers must combine the appropriate hardware with sophisticated software algorithms and sound propagation techniques. Regarding content, creators need to distinguish between 3D sounds inside the scene and sounds with a narrative function that are non-diegetic. This article looks behind the curtains of the spatial audio hype and lets you know what you’re in for and what is coming in the future.

1. Introduction

There can be little doubt that 3D audio can greatly enhance the gaming experience when designed well. Being able to hear sounds from all directions can make for a considerably more immersive and exciting game. However, there are also some pitfalls to consider when using 3D audio in games. In this article we will discuss the state of the art and future potential of this so-called “spatial Audio”.

2. Make Everything 3D – or Not?

Spatial Audio is utilized within digital gaming spaces to help players experience sound like in real-life – sounds happening around them, being immersed inside the experience, maybe even making them forget to wear headphones. This phenomenon, called externalization, is meant to help players discern which sounds are part of the scene. While 3D audio is a fantastic tool in the game developer’s toolkit, if it is not used carefully, 3Daudio can become distracting. For example, mono playback is still favored for narration and stereo is preferred for music. The reason is that when wearing headphones, you’ll localize the sound within your head, not around you. This is called inside head locatedness (IHL). This is where the sounds waves traveling from the left and right loudspeakers meet – inside of our head. As we have become used to this, with the introduction of 3D audio it becomes clearer that this habit can become a problem.The key to making spatial audio most effective is to ensure the player can immediately discern the context of the sounds they are experiencing. For example, if the sound of a voice can be localized, by moving through the scene, the player would know that the sound is part of the scene (3Dsound). Meaning, there must be another character close by talking. If the sound cannot be localized when moving around the scene (mono sound), so it’s independent from any movement, the player would understand that the voice does not represent a character in the scene. Therefore, it must be a form of narration, giving you information about the scene, while not being visually represented.

3. The Future of Audio

3.1. Head-tracking.

So, what’s next? Gamers are used to looking at a screen in front of them. But in our daily lives, our head makes continuous micro-movements which help us to orient ourselves in our environment or when watching a screen. This is where head-tracking introduces the next evolution of 3D audio.Slight movements help to distinguish sounds coming from the front with sounds from the back. With this new infrastructure of hardware, there will be a new market for games that put sound first. They could be audio-only or have auditory cues where your ears help your eyes to unlock the next achievement. This will create a new way to immerse yourself in a game. With most games,you just watch, facing to the front. Now with head-tracking, you are free to move your head on multiple axes.Virtual reality (VR) experiences are already taking advantage of the physical body movement. But what many of these virtual worlds are missing is true realism.Apple is already building spatial audio technology and head-tracking into their AirPods! So,there are millions of devices on the market already. Gaming headsets are also starting to implement gyroscopes or webcam trackers as add-ons

3.2. Increased realism.

In addition to new hardware, there is a growing interest in increased realism and higher fidelity in audio, especially in games. This strong focus on graphic fidelity left few resources for research and development of more realistic in-game audio. With increasing diminishing returns of investments in graphic fidelity, more and more studios and developers are getting back to game audio, seeing its enormous potential to improve the player’s experience. Topics like 3D audio rendering, realistic room simulation, and sound propagation are getting more attention to finally create fully realistic audio in games.One of the major pitfalls in the drive to reach a higher level of realism is that true acoustic reality is impossible and often undesirable to achieve. The true goal should be an authentic, natural-feeling, high-fidelity psychoacoustics based auditory world.Audio designers will always have to creatively adapt a simulated environment to a game’s needs. It is similar to sound attenuation being rarely based upon a physically correct simulation of the decrease in sound pressure level and perceived loudness. Next Generation Audio features the need to be carefully integrated into a game’s content with an extensive tool set that enables designers to fully achieve their creative visions.

3.3. Real-time room simulation.

The future of audio in games will also be improvements in real-time room simulation. While simulating sound obstruction in video games with simple ray-casting methods have been common for years, more sophisticated sound occlusion, absorption, transmission, and propagation methods operating with ray tracing operations are still uncommon.Now that more people have better access to high-performance hardware in the form of current-gen consoles or cloud computing via streaming, these technologies will start to seem more and more viable. But even in their near-future state, even the most advanced room simulation technology would still have scope for improvement, such as the creation of more sophisticated and optimized early reflection systems that are using all the advantages of 3D audio. The exact positioning of the first order early reflections that are not only spatialized but also each have their own occlusion values calculated will provide a more natural sounding expansion of audio waves in a simulated environment.

4. Conclusion

The games industry is always on the lookout for new ways to improve the gaming experience, and 3D audio is one way that seems to hold a lot of promise. However, with the right combination of technologies, 3D audio can not only improve the gaming experience but create new ways to play using head-tracking. The computer graphics industry has been striving for realism for years. Now it is up to game audio to lead the way to even higher fidelity, and 3D audio will only be the first step in achieving this goal.

© 2023 Copyright held by the owner/author(s). Publication rights licensed to ACM.