Modify colors used in spectrograms to match industry standard

[Reaper10]

spectrum color redo

I think it would be good to change the audio spectrum color to something like ocenaudio or audition

examples

audacity

ocenaudio

audition

Why I think this would be good is for someone coming the for mentioned programs to tenacity doesn't need to relearn the spectrum color

[nyanpasu64]

Tenacity forked off Audacity after they already implemented a black-background color scheme in https://github.com/tenacityteam/tenacity/commit/474f05d4145dfce6205ef34d48c2660714a768ce. And this is a good thing, since I prefer when "no audio energy" is represented as "no light energy" or black, which isn't the case in the old theme with the gray background.

[RoaddogLabs]

The Audacity/Tenacity representation of an audio spectrogram is not the standard used in audio software. It's not an issue of being a theme revision. It's an issue of how the data is represented. In the pictured form above in terms of audio engineering the Audacity/Tenacity display is not usable in an application where professional results are required. The colors mean something. The background is used to differentiate the colors (which are translated values) so the information is presented in a usable format. What Audacity/Teanacity has now in the spectrogram are just colors with no meaningful data representation.

[emabrey]

I agree that the colors should be changed because I am also used to the Audition version of a spectrogram.

[nyanpasu64]

The colors have already been changed in Audacity and Tenacity master:

I noticed this screenshot has overall lighter colors than the above pictures. I'm not sure if the palette is lighter than the other programs shown above, or if Audacity's gain/range/treble boost curves are different from the above programs, or if the music is different.

[emabrey]

This is the color legend I was looking at implementing. Comments?

[nyanpasu64]

At first glance, there's not a lot of difference between Audacity's current colors, your suggested color scheme, the Ocenaudio/Audition color schemes, and Matplotlib or Seaborn (colormaps designed for data science visualization).

Matplotlib:

Seaborn: link

I feel one important factor when picking a colormap is making sure that it's perceptually uniform, providing a uniform increase in brightness from the lowest to highest-energy areas, with a uniform "local gradient" so fine detail is equally visible in all areas. (Though in the case of a spectrogram, it may be acceptable for light high-energy areas to have a steeper gradient than dark low-energy areas which are less audible in the first place, but the transition must still be smooth.) A discussion of measuring perceptual uniformity, and why it's important, is at this link. The Matplotlib and Seaborn colormaps should work well out of the box, having been designed specifically for perceptual uniformity.

Looking at your colormap, my impression is that there's a steep change in color between -40 dB and -48 dB, but almost none from -48 dB through -56-ish dB, making those colors difficult to distinguish on a spectrogram. (Note that I haven't tried measuring your colormap with a test image.)

Audacity's new colormap (link to source) seems to be designed with perceptual uniformity in mind (link to tests), resulting in relatively uniform brightness gradients and fairly visible fine detail throughout all colors.

Personally I'd stick with Audacity's new colormap, or port over a Matplotlib or Seaborn colormap (which takes extra work for minimal benefit, aside from coming with more choices of colors).

(My goal for this post is to spread knowledge that I find valuable. Did I succeed in doing that without bringing others down?)

[Sciss]

I don't find the "steps" that bad. It could be an advantage in that if you know the thresholds, you can immediately say that if you see red, that's in the -50 dB area, and when you see yellow, that must be stronger than -40 dB.

[Sciss]

One point to take perhaps into consideration, is how this works for people with constrained vision, say red-green color blindness. This would generally favour having a number of user selectable palettes, of course.

[nyanpasu64]

Personally I disagree that steps are harmless in a colormap for continuous data:

I do suspect there is a tradeoff between a uniform gradient (limited hue variation) and the ability to map color hues to absolute power levels (more hues is better), though colormaps with local regions of no contrast fail both criteria. However, Audacity's spectrogram has no legend, impairing the ability to translate from hues to absolute power levels (the best you get is comparing different points in a song). Turbo is a colormap designed to have a parabolic brightness and no regions of "similar color", though parabolic brightness means it's probably a poor choice for spectrograms where energy ranges from low to high.

Looking at the image in my previous comment, I believe that matplotlib's new colormaps are reasonably colorblind-friendly. Perhaps adding a choice of palette may be good for normal-vision or colorblind users. I don't know whether that's the case, or how to design a good palette for colorblind users, though more customization is always nice to have (aside from user/developer complexity).

EDIT: I found some criticism that the Matplotlib palettes were not designed to work well for colorblindness, but rather tested to "look good enough" based on outdated colorblindness simulations.

[Sciss]

well, I'm not advocating for solarised colours, or "perceptual dead zones", but rather saying that a jump might not be disadvantageous as such. For instance, compare how different audio applications colour the level meters. They often have clear steps, so you can immediately know whether some energy is above a particular threshold. I have no strong feelings here, anyway, all the last shown palettes look good to me.

[Sciss]

Another reference: https://support.ircam.fr/docs/AudioSculpt/3.0/co/Manipulation.html - good old audiosculpt. it lets you also intentionally clip the upper/lower threshold.

[nyanpasu64]

This appears to use the Jet color scheme, which has been criticized for inconsistent brightness variation, color banding, and dead regions, and I would not recommend using in Tenacity. The reddish colormaps (including Audacity's new colormap) are designed to fix all of these (at the cost of less distinct colors), and Turbo (which I linked earlier) is designed to act like Jet but fix color banding and dead regions.

EDIT: A colorblind person has some positive words to say about Jet (link).

[RoaddogLabs]

This is the color legend I was looking at implementing. Comments?

That's a good start but the representations for a spectrogram and a dB meter are two different applications. A spectrogram shows the energy within that particular audio segment. A dB meter only displays the level (amplitude) referenced to the scale indicated on the meter. In this case it's dBfs or decibels full scale. That's the standard for showing level on digital audio signals (though some allow the user to switch between other references).

For a meter, input or output, it's common that anything below the nominal level, in this case about -20 dBfs, would be green. Between about -20 and -10 or -5, give or take, would be yellow and just under absolute clip (0 dBfs) would be red. That's the standard used in pro audio since meters went to lighted graphs from mechanical gauges. Red always indicates a level at or near clip. Clipping in this case is the truncating of the amplitude of the signal that can result in distortion.

Most of the manufacturer pics/promo don't show level meters in use in the shots. I'll be in the theater in a few hours and will see about getting something that shows what a level meter would look like in real world use. As for the spectrogram I won't be able to get those tools going today as we have shows but I'll get something from our Smart system (a standard audio analyzer) to show what a standard looking spectrum render looks like.

[RoaddogLabs]

Tenacity should use standards and methods considered best practices in the audio business. This includes representations of data as well as best practice work flows. It doesn't matter which other methods are employed by different science disciplines for data presentation. What matters are the standards used in pro audio. Using something else or deciding to do something based on what looks best or what other disciplines do risks relegating Tenacity an a sub pro app in which features either won't be used in the pro arena or in some cases abandoned all together. I can say with great certainty the current spectrogram feature of Audacity is not used in any meaningful was by those in the pro audio world. Continuing to use a non standard (in terms of pro audio) color scheme will not only dissuade pro users from the feature but it won't be a realistic representation of information that can be used outside the program.

[CHJ85]

@emabrey I love it!

[Sciss]

I think making choices based on scientific data is actually "professional". Not everything in the market is automatically better, plus it changes according to fashion. One year everyone wants to have photo-realistic sliders, and so all "pro" audio applications are filled with photo-realistic knobs and sliders, even if they are a mismatch for screen and mouse, and their usability is bad :shrug:

[RoaddogLabs]

I think making choices based on scientific data is actually "professional". Not everything in the market is automatically better, plus it changes according to fashion. One year everyone wants to have photo-realistic sliders, and so all "pro" audio applications are filled with photo-realistic knobs and sliders, even if they are a mismatch for screen and mouse, and their usability is bad 🤷

You're conflating a UI element with the display of data. Audio engineering is indeed a science. As such there are standards for how data is displayed. A fader deck representation isn't one of those. A spectrogram, FFT, impulse response or even signal level are. It's not a marketing decision. It's a decision based on engineering specific to pro audio. In the case of audio engineering the colors correlate to the information that is displayed. To implement these features in a way that is usable one needs to understand what the data is and how it is used.

Here's a shot from industry standard Smaart v8 from Rational Acoustics. It's an analysis tool that is common in professional audio. Specific colors and patterns convey specific information. You can't see the mouse pointer in the shot (Grabber didn't capture it) but there is a cross hatch pointer that gives the frequency and amplitude displayed at the top of the window. Mouse around and it returns frequency/amplitude pairs from the data. This is the industry standard for how a spectrograph is displayed.

Here's a model of a system deployed in free space. It's from Meyer Sound MAPP 3D a well respected modeler for Meyer Sound Labs products. Other manufacturers has similar products for modeling their specific products. Notice how the energy is represented in color gradients similar to Smaart. There is a legend for the colors in the upper right of the screen. Both Meyer Sound Labs and Rational Acoustics are well respected within the audio engineering community and are using display elements that have existed since these types of measurement tools went color 25-30 years ago.

[maelh]

I very much disagree. I appreciate that Audacity/Tenacity uses the colors it does, it is much easier to differentiate. Good UI takes care of color vision differences, red and blue contrasts for most people, also those that have non-standard color vision.

The other pictures all have less contrast/pop out less than the very first picture. Please favor accessibility/usability by default. Color maps could still be selectable if people prefer another style.

[SS64]

The Audacity/Tenacity representation of an audio spectrogram is not the standard used in audio software. It's not an issue of being a theme revision. It's an issue of how the data is represented. In the pictured form above in terms of audio engineering the Audacity/Tenacity display is not usable in an application where professional results are required. The colors mean something. The background is used to differentiate the colors (which are translated values) so the information is presented in a usable format. What Audacity/Teanacity has now in the spectrogram are just colors with no meaningful data representation.

I think the issue Audacity has is that for some inexplicable reason, the Maximum frequency for the spectrogram is set to 8KHz. It can be changed to 22050 in preferences > spectrogram, really that should be the default.

[nyanpasu64]

The other pictures all have less contrast/pop out less than the very first picture. Please favor accessibility/usability by default.

Personally I find Audacity's old spectrogram theme subjectively less accessible/usable and intuitive (in addition to doing poorly in mathematical properties like perceptual uniformity), and prefer for low energy to be black and high energy to be bright, like in Audacity's new spectrum (which is Tenacity's current spectrum).

I think the issue Audacity has is that for some inexplicable reason, the Maximum frequency for the spectrogram is set to 8KHz. It can be changed to 22050 in preferences > spectrogram, really that should be the default.

I agree that 8KHz is a bad default (I change it on every new install), but why should 22050 be the default? The human hearing limit is 20 KHz, the 44.1 KHz Nyquist limit is 22.05 KHz, and the 48 KHz Nyquist limit is 24 KHz. I don't feel 44.1 KHz is worth "enshrining" in the default spectrum settings (though it could be seen as a compromise that shows a bit of frequency above the hearing limit so you can see aliasing, but not enough to waste lots of room in linear scaling mode). Personally I prefer setting a default of either 20 KHz, or 999999 (which later gets clamped to half the track's sampling rate).

Additionally I personally prefer to use ERB (or Mel/Bark) scales by default, since they better allocate the limited vertical space available in a track. Treble takes up less space than Linear, and low bass doesn't waste space with thick lines stretched vertically like Log. I don't know what professional spectrogram software uses though.