mixed audio tracks sound fine on the timeline but are clipping in the export

[PontiacCZ]

Describe the bug: When I add music to our birthday celebration, it all sounds good while played back on the timeline. But audible clipping is in the export file.

Steps to reproduce the behavior:

1. Download these two audio files:

   • sound of people crowd - 20 s
   • Small guitar music (free download)

2. import them into a new project in OpenShot 2.5.0

3. choose project profile: HD 1080p 25 fps

4. put celebration to track 4 and the music to track 5. It should look like this:

5. play back first 20 secs of the timeline - you can hear that all sounds good in the OpenShot.

6. now export video: H265, profile HD 1080p 25 fps, quality "med", on the Advanced tab: start frame 1, end frame 500 (so you export only 20 seconds, we do not need more)

7. play back the exported file - you will hear pretty bad audio clipping, especially with the drum beats. My encode is here.

Expected behavior: I expect the audio in the exported file to sound the same way as in the timeline.

System Details:

• OpenShot Version: 2.5.0
• Operating System / Distro: Windows 10 Pro x64

I know there is already an issue about this problem, but I wanted be more specific, in my report, provide steps and files for you to work with.

[zackees]

Yup - i have the same problem.

I'm using an MP3 audio track

[cleveopiyo]

Having the same problem with audio clipping but with mp4 audio and only after I export

[ferdnyc]

If you're mixing two audio sources together at full volume, clipping is a likely result. Especially if either or both of them are already close to or at the limits of their amplitude range.

Loading the guitar track into Audacity, you can see that it's already clipping some, and that's without any other tracks mixed in:

(The red lines represent clipping, with View > Show Clipping enabled.)

So it's no surprise that when we mix in another fairly loud track at full volume, the clipping increases drastically:

And, yeah, it sounds pretty bad.

The trick is to simply reduce the volume of both tracks, so that they don't push each other beyond max amplitude when mixed together.

Here's the same mix, only this time I set Volume adjustment keyframes on the first frame of each clip (so the value applies for the entire length of the clip). I adjusted the guitar track to 0.75, and the crowd-noise track to 0.8:

There's still a little visible clipping, but just like with the original guitar track, none of it is audible.

The reason you don't hear clipping when previewing is, you most likely have the output volume on your computer set far below 100%. So, there's plenty of headroom left in the output hardware, and the signal doesn't come anywhere near maximum amplitude — never mind exceeding it. If you were to play the preview back with the volume set to 100%, you'd notice that both tracks were far too loud, leading to distortion every time their combined amplitude attempted to exceed the maximum levels for your audio hardware

Long story short, when mixing audio from multiple sources, you have to reduce the input levels to prevent clipping. Don't assume that there won't be clipping, and then plan to correct for it only if you hear distortion in the preview. Unless the track volumes are reduced, you KNOW there's going to be clipping, whether you hear it or not. So, just make the necessary adjustments right from the start. (And then plan to go back and tweak things some more, if the final result still doesn't sound exactly the way you planned.)

[SuslikV]

@ferdnyc the problem is slightly different, I think. At least, the last time I did my own test with the audio, the clipping itself wasn't an issue, but buffer data corruption caused major issue. The samples should be looked at high zoom before making conclusion. Clipping is just square signal that can cause distortion but buffer corruption is way different (random data). I want to say that your screenshots may not fully describe the original issue ( https://github.com/OpenShot/openshot-qt/issues/1069 ) that was mentioned in this thread.

[ferdnyc]

The other thing you can do, if you prefer not to deal with manually adjusting levels, is set the Volume Mixing mode for both (all) of your tracks to Average. In that mode, OpenShot will automatically average the volume of all playing tracks so that they don't exceed 1.0, which certainly works to prevent clipping. Here's the mix again, with both tracks set to Volume Mixing: Average and Volume: 1.0...

Here's a breakdown of what's happening there:

• OpenShot automatically sets the volume of both tracks to 0.5 while they are both playing.
• Because audio volume is measured on a logarithmic scale, two tracks playing at 0.5 barely ever add up to more than 0.7 combined volume. (In theory they could reach 1.0, but only if both tracks were at peak amplitude at exactly the same time.)
• However, due to the logarithmic scale, a volume level of 0.5 is perceptually only a small reduction from 1.0 — it certainly doesn't sound "half as loud", in fact it's hardly noticeable.
• Nevertheless, when the crowd noise track ends, the guitar track does audibly increase in volume. (which at times can be a desirable effect, allowing background music to swell to the foreground when the dialogue and other environmental sounds mixed over it drop away.)

Note, BTW, that Volume Mixing only operates on the Volume property for each track — it doesn't take into account the actual volume levels of the audio. Any two tracks mixed with Volume Mixing: Average will both be set to 0.5 volume, even if the audio in one track is much louder than the other.

Volume Mixing is, however, compatible with other adjustments to the Volume property. If two tracks are both set to 0.8 Volume, Average mixing, then when they're combined they will each have a Volume level of 0.5. (NOT 0.4. Remember that Volume Mixing: Average only reduces the volume of the tracks enough to prevent going over 1.0 total.) When the second track ends, the first track will return to 0.8 volume instead of 1.0, which can make that transition less audible. Here's what that looks like, for our mix.

(Upper waveform: 1.0,Average+1.0,Average; Lower waveform: 0.8,Average+0.8,Average):

As you can see, the levels are identical when both tracks are playing together, but the volume transition to the guitar track playing solo is less severe.

The reduction with Volume Mixing: Average may be a bit more severe than people would want, which is why it isn't enabled by default. It's better to manually balance the audio levels where you want them, when mixing audio from multiple clips. But it's there as an option, if you prefer not to deal with levels. Another nice thing is that Volume Mixing: Average has no effect whatsoever when the clip is the only one playing audio, since there's nothing to average.

(There's a third Volume Mixing mode, Reduce, which simply sets the max volume of the clip to 0.77. That mode is stupid and nobody should use it.)

[ferdnyc]

@SuslikV

Agreed, the audio mixing does appear to be doing some strange things when amplitude levels exceed 1.0, that makes the audio sound worse than normal clipping. That's a bug for libopenshot.

However, that corruption/distortion only happens when the audio levels exceed 1.0 (where there would otherwise be clipping anyway), so the practical solution to avoiding it is still just to prevent the levels from exceeding max volume. As long as the amplitude levels stay safely below 1.0, the audio mixing is flawless.

(When they exceed it you get this — lower waveform — which I agree shouldn't happen.)

[SuslikV]

... (There's a third Volume Mixing mode, Reduce, which simply sets the max volume of the clip to 0.77. That mode is stupid and nobody should use it.)

The best comment I ever seen here. @ferdnyc , let's hope that there is serious math and experience behind this ^_^

[PontiacCZ]

The reason you don't hear clipping when previewing is, you most likely have the output volume on your computer set far below 100%. So, there's plenty of headroom left in the output hardware, and the signal doesn't come anywhere near maximum amplitude — never mind exceeding it. If you were to play the preview back with the volume set to 100%, you'd notice that both tracks were far too loud, leading to distortion every time their combined amplitude attempted to exceed the maximum levels for your audio hardware

You're right in this one. My volumes are at about 30%:

So I did a test now: while playing back my timeline in OpenShot, I pulled the master volume up to nearly 90% (which was so loud that I started to worry about possible damage of my headphones and my ears but I did it only for a couple of seconds). No clipping appeared.

On the contrary, I played back the exported file clipping.mp4 on a very low volume so I could barely hear it. The clipping was still there.

I believe this is not about playback HW.

EDIT: the Average Volume Mixing mode works well, will use it on my home video, thanks for pointing that out. :)

[SuslikV]

@ferdnyc Well, I thought that this is far worse error but it looks that it is just mistake in the code. You may notice how it works, just from the screenshots. What a beautiful math:

On the screenshot above is OpenShot's: 1) aac-compressed track (result of 1 stereo pre-overloaded track export) 2) aac-compressed 2 tracks mix (result of 2 stereo not overloaded tracks mix export) - bottom one.

The input samples in uncompressed .wav float format are simple as Audacity main menu Generate>DTMF Tones... (3 sec) The overloaded sample in uncompressed .wav float format may be produced by Audacity as own mix of two of these tones.

[stale[bot]]

Thank you so much for submitting an issue to help improve OpenShot Video Editor. We are sorry about this, but this particular issue has gone unnoticed for quite some time. To help keep the OpenShot GitHub Issue Tracker organized and focused, we must ensure that every issue is correctly labelled and triaged, to get the proper attention. This issue will be closed, as it meets the following criteria: - No activity in the past 180 days - No one is assigned to this issue We'd like to ask you to help us out and determine whether this issue should be reopened. - If this issue is reporting a bug, please can you attempt to reproduce on the latest daily build to help us to understand whether the bug still needs our attention. - If this issue is proposing a new feature, please can you verify whether the feature proposal is still relevant. Thanks again for your help!

[weshinsley]

Two things I'd request as features from my experience with the above issues...

1) Regardless of all the clipping issues above, it is really common for me that it sounds perfect when putting the movie together in Openshot, and the big ugly clips only come out in the final exported MP4. That's hard to work with - I have to try and guess which channel is mainly responsible for the clipping at that moment, make the change in Openshot, re-export the video and see if I've got it. So, if audio when you're making the video could be the same as what gets exported, that would be amazing.

2) Rather than "Average" being applied per-channel, I'd like an overall "master" scaling done on the audio. I am usually mixing many videos together for a band - the largest number I've done is about 6 instruments and 12 voices. So 18 or so audio channels; averaging is all very well, but the final level of each track, and of the final output audio mix is extremely low. I have to pull the audio out in Audacity and normalise it up to a sensible level - but by that time I've lost quite a lot of signal. [Just want to emphasise that except for this point, Openshot does a great job of handling this]

Additionally, I find the audio much harder to mix in Openshot when the Average mode is enabled on the channels, since changes to one channel all affect the others in the average... which is very different from the "normal" audio mixing we're probably aspiring towards being able to do in Openshot.

So the best way to solve this, I think, would be to have an option to calculate the maximum mixed audio level across the whole mixed audio, find the scaling factor to get that max down to 1 (or 0.99), and then multiply all the audio tracks levels by that scaling when mixing down, so the sum wouldn't clip.

In the meantime, I currently play the movie in Openshot, simultaneously record the audio output in Audacity (which comes at a very sensible level), and then use that as my final audio...

[PontiacCZ]

Yep, I agree. I (as a user) do not need to understand the background or use some explicit type of mixing (e.g. average). I just expect that the sound in the exported file does not clip when it does not clip on the timeline. It should actually sound the same. Common sense.

[stale[bot]]

Thank you so much for submitting an issue to help improve OpenShot Video Editor. We are sorry about this, but this particular issue has gone unnoticed for quite some time. To help keep the OpenShot GitHub Issue Tracker organized and focused, we must ensure that every issue is correctly labelled and triaged, to get the proper attention.

This issue will be closed, as it meets the following criteria:

• No activity in the past 180 days
• No one is assigned to this issue

We'd like to ask you to help us out and determine whether this issue should be reopened.

• If this issue is reporting a bug, please can you attempt to reproduce on the latest daily build to help us to understand whether the bug still needs our attention.
• If this issue is proposing a new feature, please can you verify whether the feature proposal is still relevant.

Thanks again for your help!