At least since Apple Music added the category “Spatial” to its program, immersive music has been on everyone’s lips… or rather, on everyone’s AirPods, earbuds, soundbars, etc. In particular, the available catalogue of Dolby Atmos mixes is growing rapidly, and statistics show that listeners often prefer immersive sound formats to traditional stereo sound. Until quite recently, object-based mixes were almost exclusively the preserve of larger major label artists. However, more and more studios and engineers have upgraded their systems, and both DAW manufacturers and DIY distributors have done a lot to pave the way for Dolby Atmos, Ambisonics, Sony 360 Reality Audio, Auro-3D and co. for indie and DIY artists as well. But how exactly do you go about it?
Whether you’re working with a dedicated engineer for immersive mixing or want to independently transfer your stereo mix into the three-dimensional world: This article aims to give you some insight into remixing in immersive audio formats so that your stereo song can shine in 3D.
Since there are now numerous resources on how to technically set up Dolby Atmos mixes, in this article I’ll focus on the specific processes and workflows involved in creating an immersive remix. Let’s go!
Preparation is everything
As with any mix, it’s the ingredients that determine the result – so we need good signals first! We generate these from the original stereo mix. Anyone who works with hip-hop, electronic, rock or pop music knows how essential the use of effects can be for the sound design of a song. But even when working with jazz or classical music, it makes sense to keep as much as possible of the original sound vision of the stereo production and not to completely reinvent the piece. Sensitivity is required because on the one hand the immersive space should be used in the best possible way, and on the other hand, the artistic intention should be preserved.
In general, what sounds great in stereo usually sounds great in 3D as well. So, we don’t start with multi-tracks and dry signals, but with stems and take the original effects with us. 3D mixes usually require a little more stems than you might use for stem mastering. About 12-24 stereo signals is a good orientation for an average Dolby Atmos pop mix.
In my work so far, it has worked well to keep all insert effects on the stems and to bounce all reverbs and echoes as separate stereo stems. In a 3D mix, the room and space effects can then be placed duplicated and varied according to taste, and you save yourself the trouble of recreating the room sound by ear. Sometimes I continue working directly with the original Pro Tools session so I can, for example, exchange a stereo reverb directly for a 7.1.2 reverb, etc.
De-mixing instead of upmixing
Hard disk broken? Original session no longer available? Mixed directly to tape? Single tracks overwritten? Unfortunately, good archiving is (or was) often neglected and the single tracks or stems of a production are not always trackable. What now?
Let’s start with the quick and dirty solution: place any upmix plugin on the stereo mix and voilà: 3D! Upmix plugins do have their uses, but on a stereo mix they rarely offer the creative flexibility needed for a professional music mix. After all, we don’t want to turn our song into a homogeneous sound cloud, but rather give each element its own place in the new arrangement!
The much better solution: de-mixing. The term “de-mixing” describes the process of separating a complete song into its stems, i.e. ideally undoing the mix. Until now, this has only been possible to a limited extent from a technical point of view and has always involved compromises. After all, signals in a mix mask each other and steep filters have to be used to separate the tracks from each other again. This can hardly be done manually, which is why artificial intelligence or machine learning is used for de-mixing. This is what a de-mixed song could look like:
I haven’t yet found the perfect algorithm that works ideally for all cases (and I don’t think there ever will be such a thing). Therefore, I like to test and combine different algorithms and techniques for different songs and signals. However, research in this area is still in full swing and in many cases, the most current algorithms have not yet been turned into purchasable products. So, anyone who is currently seriously involved with this topic should not be afraid of code and command lines.
If you are looking for a simpler way and don’t necessarily want to use or compare the latest algorithms, you can also buy various software that can split your song into stems (e.g. Audionamix Xtrax or iZotope RX Music Rebalance). There are even online services (e.g. lalal.ai) where the separation is not done on your own system and no software needs to be installed whatsoever.
Routing & templates
In this section, I would like to get a bit more specific and show you how a professional 3D mix can be created using my own template workflow. Because there is currently a high demand for Dolby Atmos and I myself mix for Dolby Atmos as well, I will limit myself to this format. The technical implementation differs of course, but the prerequisites and workflows can still be transferred if you are currently working with any other 3D format. Like most professional Atmos engineers, I work with Pro Tools and the Dolby Atmos Production Suite. I’ll spare us the technical details of the setup here, there’s more than enough material for that elsewhere. So let’s jump right into the practice of remixing for Atmos!
For the synchronization between renderer and DAW a timecode is needed (LTC), which is on the second channel in my template (right after the click track as you can see). It is switched to Solo Safe so that the timecode continues to run even when listening solo. This is also where the “Dolby Atmos Binaural Settings” plugin is located in my usual setup. Since it only delivers the binaural metadata to the renderer, the audio channel doesn’t matter in this case. By the way, I always use the same preset in which the binaural metadata is already chosen and the objects are named accordingly. This saves me the extra step of setting them individually because I simply choose an appropriate object (“near”, “mid”, “far” or “off”) right away. This workflow has proven itself especially for compiling albums in Dolby Atmos because no different settings are allowed to get in each other’s way.
The LFE channel is unfortunately often misunderstood by music creators. It is not a subwoofer channel that is supplied with the low frequencies of the other speaker channels by means of a crossover filter, but a completely independent channel that can be supplied with the full frequency spectrum. For reasons of translatability to different systems, however, it is advisable to limit the frequency response of the LFE channel by default – a simple low-pass filter between about 100 and 150 Hz does the job.
Since consumer systems differ greatly in terms of bass management, acoustics, etc., it also makes sense to slightly change the phase relationship between the bass component of the main speakers and the LFE channel. This reduces the likelihood that the bass will build up excessively on bass-managed systems. In my template, the Subharmonic Pro plugin is used for this purpose, which slightly changes the harmonic structure of the LFE channel. The LFE channel does not necessarily have to be used in every music mix, and the mix must still sound coherent even without LFE, but it can help to set dramaturgical points or to fatten the bass foundation of the mix.
Whether SSL, Neve, API, Fairchild, Manley or self-made – many rock and pop engineers swear by their sum compressor and of course don’t want to do without it in 3D. The only problem is: There is no sum! So, you have to come up with a workaround to get the popular compressor glue. Fortunately, almost all dynamic plug-ins offer us a sidechain input that we can use for this purpose. The workflow also works with hardware, but quickly becomes costly due to the high channel count. In my Atmos mixing template, there are compressor instances on all master channels that are controlled via sidechain from one and the same aux bus. Depending on context and taste, you can use post-fader send to create a stereo mix that matches the mix ratios of the Atmos mix, use pre-fader send to create completely individual ratios, or simply let the original stereo mix control the compression. I usually do the same with limiting, in case limiting is needed at all.
Granted: Individual compression using sidechain doesn’t sound exactly the same as summing compression, but it creates a very similar aesthetic and gives us considerably more freedom. By the way, I have linked the insert parameters of the Master Channels so that I can conveniently adjust all master compressors on one instance. If necessary, however, the link or group can of course be deactivated at any time to make individual settings possible. In my case, the external sidechain is routed and switched off by default, so it can be used at any time with a click if I want to turn the individual bus compression into a “false sum compression”:
In addition to reverbs, I use slapback echoes a lot in immersive mixes to give individual elements more spatial context and build a virtual room. In my template, there are three stereo slapbacks by default: one for the surround, one for the sides and one for the top channels. I tend to move them around though, depending on the musical context.
The slap channels are in fact very simple: a simple time delay and a little EQ-ing, if necessary, is usually enough. However, I often change the stereo width, swap the stereo channels or use a transient designer to focus the slap response more on transients or decay, that’s why the TrackControl plugin comes in particularly handy, and I have a transient designer plugin on bypass just in case.
Mastering & export
Dolby Atmos mastering causes a lot of confusion and misunderstanding. Since there is no sum in the classical sense, the conventional stereo mastering chains are of course invalid. Many conclude that there is no such thing as mastering for Dolby Atmos. I would disagree, but of course, it depends on how you define mastering so there’s no point in arguing.
In my opinion, mastering consists of 3 primary tasks:
- Compliance with all technical specifications and formats
- Sonic and artistic quality control
- Optimization of the translation to different listening systems
All three aspects are highly relevant to Dolby Atmos as well and in some cases even more critical than ever before.
Dolby Atmos has clear loudness specifications, and most companies are rather strict with this: A maximum of -18 LKFS and -1 dBTP. By the way, loudness is measured on a 5.1 re-render, because Dolby Atmos itself consists of up to 128 channels and is only interpreted by the renderer into an individual mix. This also means that re-rendering settings have an impact on loudness measurements by the way. The loudness measurement is integrated into the Dolby Atmos renderer, so it is easy to check. -18 LKFS is relatively quiet and gives more than enough headroom for beautifully dynamic music mixes. Of course, that doesn’t stop you from compressing your mix beyond good taste, but Dolby Atmos doesn’t know any Loudness War.
If you work for a label, there are usually even more detailed technical and creative specifications. Of course, you should read these carefully and adhere to them.
You should never deliver without quality control. Before delivery, someone should listen through everything in a professional studio environment and remove any resonances, technical errors, noise, etc. that may have been overlooked. Also, a tasteful polishing with compressors, EQs, saturation and co. is common. However, this is done on an object or bed basis as shown above since a sum in the sense does not exist.
Translation to the various monitoring systems has always been an important aspect of mastering, but I think it has taken on a whole new meaning with object-based formats. Since the format is interpreted individually on pretty much every conceivable speaker combination and even rendered binaurally for headphones, the differences between the various systems can be huge. It takes some experience to find the happy spot for all the various devices. Apple’s decision to use its own renderer instead of the official Dolby renderer further complicates this process. Specifically, you have to check the master on various studio speaker combinations, at least one soundbar, and on headphones with two different binauralizers (Dolby and Apple) and make sonic adjustments to the mix or master to ensure that the mix can shine everywhere. Often, quality control on the different devices and, if necessary, making further adjustments take up most of the work hours and often have the biggest impact on how your mix actually performs out there. So don’t ever think you can skip mastering (or however you want to call it) for Dolby Atmos mixes because there’s no sum.
Conclusion
Mixing in 3D may seem a bit more complicated than classic stereo at first since you have to deal with a lot more routing, but fortunately, you can set it up once and save it as a template, so your daily work will be a lot easier and faster. I hope this little overview helps you to understand the 3D remixing process even better and maybe gave you one or two new ideas to implement into your own workflow. The practical examples are of course only based on my personal workflow at this moment (which is constantly changing) and every engineer, of course, has his or her own ways of working. So, feel free to share your own workflows and plugin tips for 3D music mixing in the comments!
One Response
Nice work! I’m an algorithm engineer and trying to find a common solution to remix stereo pop music into 3D, my confusion is where to put the guitar, bass and vocal in the virtual 3D room. I’m not pro in remixing so it’s difficult to me. Are there any suggestions ?