Equal loudness clipping protection

[BrechtDeMan]

If loudness equalisation introduces clipping (e.g. making something -10 LU would lead to a sample value being > 1.0), the test should not continue and an error message should be shown. I don't think anything like this happens now, and since it could lead to invalid results I'd call this a bug.

To help the experimenter choose a more appropriate loudness, debugging data should be available including the audio file and time where clipping occurs, and perhaps even the maximum loudness (i.e. rescale the audio so that the maximum absolute value is about -0.3dBFS, then calculate the corresponding loudness). (the latter is less critical, more of an enhancement/stretch goal)

[nickjillings]

Rescaling is a good way to resolve this. Afterall the LU is a relative scale, so when we say we want something at -10LU and another at -20LU, ultimately they just need to be 10LU appart.

The difficulty comes from having a scaling factor known for the entire test, across all pages, at run-time. The test loads audio asynchronously so that would have to be blocked to support this. Ultimately a popup could appear saying the test is invalid in some way by indicating that clipping would occur once the audio has been loaded.

[BrechtDeMan]

Agreed - and even more problematic if you have several tests which all need to be at -23 LU, for instance. No way for the system to know that the loudness is actually important. It is a relative scale, yes, but it is relative to something, in this case the full scale gain of the system, which may be fixed on purpose across the whole signal chain.

I think it is acceptable to have a popup saying the loudness has been rescaled to x LU, at which point the experimenter can decide to adjust the loudness units on the page, across the test, or across several tests (or of course edit the problematic sample). But since I thought it unlikely that anyone would like to ship a test with such a popup, I suggested the test doesn't need to continue at that point.

[deeuu]

Hi both,

Ideally, it would be good to have a 'loudness checker' which finds the minimum LUFS value across the entire audio and uses this to determine the gain for every stimulus. The experiment configuration option is then simply a checkbox denoting 'loudness normalise or not' (rather than a user-defined target loudness).

Otherwise, perhaps just stick with a 'safe' target loudness (not variable?) and then go with the pop up approach.

[nickjillings]

finds the minimum LUFS value across the entire audio and uses this to determine the gain for every stimulus

So every audio frame is reduced to match the minimum frame? That would work I suppose. The problem is the asynchronous loading. It is possible the first page is ready to run before all the audio fragments are loaded, so it is not possible to safely obtain that information.

The only other way is to normalise within a page, since they are more atomic. However then the loudness would swap from page to page. Hitting a target loudness makes the most sense but then we get into the clipping problems.

A blanket loudness normalisation does probably make more sense, and simply set to -23LU (or another target) to ensure the perceived loudness is consistent. The local gains can still be used to adjust for in-page differences if needed.

So the new loudness normalisation routine would be:

1. Calculate the integrated loudness of the audio
2. Determine the difference and respond with an appropriate playback gain to meet -23LU (if `normalise="true")
3. If the loudness would cause clipping, throw an error (a page may already have run by this point).
4. On local pages, use the local fragment gain node to further boost/cut the audio if required
5. If this causes clipping, throw an on-page error since it may no longer be perceptually consistent to alter page gains (the clipped audio could be on the last page, changing the audio would invalidate the test).

I think this is the neatest way to achieve this without having to break the asynchronous loading patterns.

[deeuu]

Hi Nick,

So every audio frame is reduced to match the minimum frame? That would work I suppose

I meant the minimum integrated loudness across all files.

If the master gain (I'm assuming that's what 'local fragment gain node' is) is an optional element then I think your proposal is best.

Also:

It is possible the first page is ready to run before all the audio fragments are loaded

Can't this be handled by checking the load state of every buffer? For example, lock the UI and playback until all buffer states are true?

[nickjillings]

I meant the minimum integrated loudness across all files.

Yeah we're talking about the same thing here, but the master / fragment gain can stll muck things up. There's not really a good way to handle that except to say the test creator 'this is a problem now'.

Can't this be handled by checking the load state of every buffer?

Yes we could lock the UI, but we wanted to avoid the situation where loading loads of files caused the system to hang before loading the first page. I'd prefer to not lock the UI because it could take easily 5-10 mins to load audio for several pages.

I think ultimately we just have to throw errors when they occur, to ensure that either a) the test creator normalises themselves or b) adjusts any gain structures that may occur.

[nickjillings]

I've been thinking on this a bit over the last few days and the best scenario I have is this.

Instead of doing the loudness normalisation on the fly, the loudness is computed ahead of time in the testCreator. That way the correct gains are known ahead of time, before the audio loads. That way a ready page can operate in the knowledge that no further gains need be adjusted.

However it would require the test creator tool to have WAAPI, the loudness system and everything that goes with it. There are a lot of other nice opportunities to do in there, although it means that directly modifying the XML will mean no automatic loudness normalisation, and therefore it will break syntax to go this way.

So the decision is this:

1. Don't break syntax. Throw errors which are test halting if the system thinks clipping has occured.
2. Break syntax. Push this back to v1.3.0, the point where any specification breaks occur.

Perhaps both are required, in which case I'll raise an issue for the v1.3.0 milestone.

Thoughts?

[BrechtDeMan]

I think the important points are

• the user needs control over the loudness, e.g. when setting up a system such that playback for each stimulus, page, and test is exactly -23 LUFS. So opt-out of automatic normalisation should be possible.
• when to throw errors isn't terribly important as people should pilot the test first and then notice some of the loudness equalisation introduces clipping

So upon any rescaling, check if clipping occurs, then throw errors.

I think auto-normalisation is a separate issue, which could indeed be built into the test creator (at which point it could set the test-wide loudness as '-19.4 LUFS' if that's the value that keeps all stimuli from clipping). Again this is an option.

So 1 for now, then 2 maybe later?

[nickjillings]

That's definitely a better way to think it out by having the test creator determine a suitable normalisation threshold!

It shouldn't be too hard to get the clipping to throw errors, since it's just gain stages.