Audio Artifacts In the Output

[Drive4Code]

I've made a detailed bug report to ffmpeg, which details the problem: https://trac.ffmpeg.org/ticket/10399#no3

In short, the audio presents crackling artifacts when it gets outputted.

[mmguero]

Interesting, thanks. I'll look at doing a short fade in/fade out to mitigate.

[mmguero]

I've been trying things with afade and can't quite get it to work/

First I tried this in CreateCleanSubAndMuteList:

        for timePair in newTimestampPairs:
            lineStart = (
                (timePair[0].hour * 60.0 * 60.0)
                + (timePair[0].minute * 60.0)
                + timePair[0].second
                + (timePair[0].microsecond / 1000000.0)
            )
            lineEnd = (
                (timePair[1].hour * 60.0 * 60.0)
                + (timePair[1].minute * 60.0)
                + timePair[1].second
                + (timePair[1].microsecond / 1000000.0)
            )
            self.muteTimeList.append("afade=t=out:st=" + format(lineStart, '.3f') + ":d=30ms")
            self.muteTimeList.append("afade=t=in:st=" + format(lineEnd, '.3f') + ":d=10ms")

Then I found this stackoverflow thread and I tried this:

        for timePair, timePairPeek in pairwise(newTimestampPairs):
            lineStart = (
                (timePair[0].hour * 60.0 * 60.0)
                + (timePair[0].minute * 60.0)
                + timePair[0].second
                + (timePair[0].microsecond / 1000000.0)
            )
            lineEnd = (
                (timePair[1].hour * 60.0 * 60.0)
                + (timePair[1].minute * 60.0)
                + timePair[1].second
                + (timePair[1].microsecond / 1000000.0)
            )
            lineStartPeek = (
                (timePairPeek[0].hour * 60.0 * 60.0)
                + (timePairPeek[0].minute * 60.0)
                + timePairPeek[0].second
                + (timePairPeek[0].microsecond / 1000000.0)
            )
            self.muteTimeList.append(
                "afade=enable='between(t,"
                + format(lineStart, '.3f')
                + ","
                + format(lineEnd, '.3f')
                + ")':t=out:st="
                + format(lineStart, '.3f')
                + ":d=30ms"
            )
            self.muteTimeList.append(
                "afade=enable='between(t,"
                + format(lineEnd, '.3f')
                + ","
                + format(lineStartPeek, '.3f')
                + ")':t=in:st="
                + format(lineEnd, '.3f')
                + ":d=10ms"
            )

which is (or what I'm trying to do, at least) is enable the fade only for the interval between it and the next action. This sort of worked, but it missed a lot of them, so I'm not sure what I'm doing wrong.

The strange thing about this bug is that I don't have the audio artifacts you are describing at all. For me, the original:

            self.muteTimeList.append(
                "volume=enable='between(t," + format(lineStart, '.3f') + "," + format(lineEnd, '.3f') + ")':volume=0"
            )

works just fine, despite what the reply to the ticket you logged said.

[mmguero]

I think we'll probably have to wait for this aoverlay filter the ffmpeg guy mentioned and fix it at that point in the future.

[mmguero]

Hmm, I may have another idea with the afade filter, I will try it out next time I get a sec.

[mmguero]

Okay, turns out I was dealing with an inaccurate SRT, I think the afade version actually is working. I'm going to put a release out, let me know if it works better for you without the artifacts.

[mmguero]

Going to mark as fixed, if there are issues found I'll open it back up again.

[Drive4Code]

After testing it (once) it looks fixed: the audio is now working as intended again. Because of this I don't see the need for #16 anymore. Thank you for the fix!

[mmguero]

@Drive4Code glad to hear it. From the half-dozen times or so I've used it since I've made the change it seems to be working for me as well without noticeable issues. Thanks for bringing the original issue to my attention.

[Drive4Code]

No worries. I'll keep monitoring for other issues. Stay safe!

[Drive4Code]

Update: The artifacts are still there. What changed is the overall audio quality now got better so that the artifacts aren't there consistently, yet only appear at random segements in the video. I'll post an example of the artifacts as soon as its uploaded to google drive

[Drive4Code]

@mmguero Here's the link, the electronic sound (artifact) plays already throughout the first minute

[mmguero]

Can you link me to the original source video and .srt file for your example? I'd like to try running the same thing on my system as you on your system. I have cleaned dozens of videos since the v1.5.0 release of this tool and never get any artifacts.

[Drive4Code]

Source and .srt

[mmguero]

Thanks, checking it out. I did find that I left a bit of the old volume filter in the command passed to ffmpeg (oops) so I'll remove that and see what I get with the video and srt you provided.

[mmguero]

I just released a v1.5.3 that removed the extra leftover volume filter that should have been removed when I commited the afade stuff. You can try that. I used it on your source and .srt from your comment and I don't get the audio distortion you're hearing, although I do hear it in the link you shared that you generated. That being the case I feel like it's got to be an issue with the ffmpeg you're running yourself, because if the code we're running is the same and I'm not getting the distortion with the same video file as you I don't know what else it could be except ffmpeg.

My ffmpeg I'm running locally is:

/usr/bin/ffmpeg
ffmpeg version 5.1.3-1 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 12 (Debian 12.2.0-14)
  configuration: --prefix=/usr --extra-version=1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libglslang --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librist --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --disable-sndio --enable-libjxl --enable-pocketsphinx --enable-librsvg --enable-libmfx --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-libplacebo --enable-librav1e --enable-shared
  libavutil      57. 28.100 / 57. 28.100
  libavcodec     59. 37.100 / 59. 37.100
  libavformat    59. 27.100 / 59. 27.100
  libavdevice    59.  7.100 / 59.  7.100
  libavfilter     8. 44.100 /  8. 44.100
  libswscale      6.  7.100 /  6.  7.100
  libswresample   4.  7.100 /  4.  7.100
  libpostproc    56.  6.100 / 56.  6.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'

and in the Docker container, which also works fine for me:

ffmpeg version 6.0 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 12.2.1 (Alpine 12.2.1_git20220924-r10) 20220924
  configuration: --pkg-config-flags=--static --extra-cflags=-fopenmp --extra-ldflags='-fopenmp -Wl,-z,stack-size=2097152' --toolchain=hardened --disable-debug --disable-shared --disable-ffplay --enable-static --enable-gpl --enable-version3 --enable-nonfree --enable-fontconfig --enable-gray --enable-iconv --enable-lcms2 --enable-libaom --enable-libaribb24 --enable-libass --enable-libbluray --enable-libdav1d --enable-libdavs2 --enable-libfdk-aac --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libkvazaar --enable-libmodplug --enable-libmp3lame --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-librabbitmq --enable-librav1e --enable-librtmp --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtheora --enable-libtwolame --enable-libuavs3d --enable-libvidstab --enable-libvmaf --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-openssl
  libavutil      58.  2.100 / 58.  2.100
  libavcodec     60.  3.100 / 60.  3.100
  libavformat    60.  3.100 / 60.  3.100
  libavdevice    60.  1.100 / 60.  1.100
  libavfilter     9.  3.100 /  9.  3.100
  libswscale      7.  1.100 /  7.  1.100
  libswresample   4. 10.100 /  4. 10.100
  libpostproc    57.  1.100 / 57.  1.100
Hyper fast Audio and Video encoder
usage: ffmpeg [options] [[infile options] -i infile]... {[outfile options] outfile}...

Use -h to get full help or, even better, run 'man ffmpeg'

[Drive4Code]

This Normal? Traceback (most recent call last): File ".venv/bin/cleanvid", line 8, in sys.exit(RunCleanvid()) File ".venv/lib/python3.9/site-packages/cleanvid/cleanvid.py", line 681, in RunCleanvid cleaner.CreateCleanSubAndMuteList() File ".venv/lib/python3.9/site-packages/cleanvid/cleanvid.py", line 338, in CreateCleanSubAndMuteList for sub, subPeek in itertools.pairwise(subs): AttributeError: module 'itertools' has no attribute 'pairwise'

[mmguero]

Ah, Python 3.9. I'll get it fixed.

[mmguero]

Restored pairwise for older python 3 versions.

[Drive4Code]

I finally managed to run it on docker with ffmpeg 6.0 and it definetely works much better. As a sidenote: how did you get ffmpeg 5 on your local machine? Mine only gets ffmpeg 4 even with snap (i'm using WSL)

[mmguero]

I'm running Debian Linux 12 natively, I've not run cleanvid in Windows at all before. I'm just using the ffmpeg package in the default repos. I'm glad you were able to get it outputting better sounding audio!