Audio and video not synched when trimming video after download

[MaciAC]

Hello, first of all, many thanks for your work, this library is truly useful.

In our use case we download short chunks from a long twitch stream. We are using this set of "DownloadOptions"

params = DownloadOptions( chapter=None, start=start_in_seconds, end=end_in_seconds, max_workers=20, format="mp4", output=output_path, overwrite=True, auth_token=None, rate_limit=None, quality=quality_to_download, dry_run=False, no_join=False, concat=False, skip_existing=False, keep=False, cache_dir=cache_dir, )

We are encountering that, when start_in_seconds does not coincide with how the VOD files are cut, we get a desyncronization between audio and video, basically the video stream from the first video file is not appearing in the output video while the audio is properly concatenated.

We have found a fix to this: in the _join_vods method, when creating the ffmpeg command, the -ss parameter should be placed between the first input file (playlists_path) and the second (metadata_path), so moving the if and its content from line 87 to line 84:

https://github.com/ihabunek/twitch-dl/blob/master/twitchdl/commands/download.py#L87

Let us know if you will add this of the next release or if you want us to open a PR.

Many thanks in advance

[ihabunek]

Thanks for reporting. I'm currently unable to reproduce this. Here's my test:

First video is clipped between VODs so ffmpeg uses -ss:

twitch-dl download 2248712439 --start 1:00:32 --end 1:01:44 -q 720p -o test1.mp4

ffmpeg -i /home/ihabunek/.cache/twitch-dl/videos/2248712439/720p30/playlist_downloaded.m3u8 -i /home/ihabunek/.cache/twitch-dl/videos/2248712439/720p30/metadata.txt -map_metadata 1 -ss 00:02 -t 01:12 -c copy -stats -loglevel warning file:test1.mp4

Second video is clipped at VOD boundary so ffmpeg does not use -ss:

twitch-dl download 2248712439 --start 1:00:30 --end 1:01:44 -q 720p -o test2.mp4

ffmpeg -i /home/ihabunek/.cache/twitch-dl/videos/2248712439/720p30/playlist_downloaded.m3u8 -i /home/ihabunek/.cache/twitch-dl/videos/2248712439/720p30/metadata.txt -map_metadata 1 -t 01:14 -c copy -stats -loglevel warning file:test2.mp4

Both downloaded videos, test1.mp4 and test2.mp4 have correctly synced audio and video.

If it's not a secret, could you share with me a command which produces unsynced audio? I'd love to learn what causes this. Thanks.

[MaciAC]

You are right, with the given command the result is not unsyced, but it has 2 seconds of black frames at the begining, these 2 seconds of black frames (actually no frames) leads us to an unsynced video when we apply transformations afterwards.

So the actual error is not an unsynced video but a video without the first frames of the video stream.

Is your downloaded video properly downloaded or does it also have these 2 seconds of black frames at the start?

[ihabunek]

That's interesting. I do not get the black frames...

Here's my results: https://bezdomni.net/test1.mp4 https://bezdomni.net/test2.mp4

I did test and moving -ss as you suggested produces the same result for me, so I would be up for changing that if it fixes your problem. But I would be interested to find under which conditions the problem occurs.

Could you paste the result of running twitch-dl env and ffmpeg -version here?

[MaciAC]

In the results you are sharing, test1.mp4 has 2 seconds of freeze frames at the begining, and when I am downloading it I see the black frames (I am using iMac preview to check it, when opening the video with VLC I also see the 2 seconds of freeze frames at the begining).

Anyways, here you can see the versions:

(base) ➜  /tmp twitch-dl env
twitch-dl 2.9.1
Python 3.11.7 (main, Dec 15 2023, 12:09:56) [Clang 14.0.6 ]
Platform: macOS-14.1-arm64-arm-64bit
Cache dir: /Users/macia/Library/Caches/twitch-dl

(base) ➜  /tmp ffmpeg -version 
ffmpeg version 7.0 Copyright (c) 2000-2024 the FFmpeg developers
built with Apple clang version 15.0.0 (clang-1500.3.9.4)
configuration: --prefix=/opt/homebrew/Cellar/ffmpeg/7.0_1 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags='-Wl,-ld_classic' --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libharfbuzz --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopenvino --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox --enable-neon
libavutil      59.  8.100 / 59.  8.100
libavcodec     61.  3.100 / 61.  3.100
libavformat    61.  1.100 / 61.  1.100
libavdevice    61.  1.100 / 61.  1.100
libavfilter    10.  1.100 / 10.  1.100
libswscale      8.  1.100 /  8.  1.100
libswresample   5.  1.100 /  5.  1.100
libpostproc    58.  1.100 / 58.  1.100

[ihabunek]

Curiouser and curiouser. I do not see the frozen frames in VLC or MPV on linux. :thinking: Now I tried Totem, the default video app, and I do see them there...

Anyways, can you pull the latest master and let me know if this fixes your problem?

[MaciAC]

I tried but the result is not correct, the -t option should be after the metadata parameter, this is how I accomplished the proper result:

        append("ffmpeg")

        append("-i", playlist_path)

        if crop_start:
            append("-ss", utils.format_time(crop_start))

        append("-i", metadata_path)
        append("-map_metadata", 1)

        if crop_duration:
            append("-t", utils.format_time(crop_duration))

        append("-c", "copy")
        append("-stats")
        append("-loglevel", "warning")
        append(f"file:{target}")

[ihabunek]

Oh man, ffmpeg... Done, test please!

[MaciAC]

Now it works perfectly! Many thanks for the fast response!!

[ihabunek]

Fix released in v2.9.2. Thanks for reporting, please let me know of any other issues you encounter.