devhotteok
commented
5 months ago Thank you very much for your detailed description. I will look into the issue of segments getting corrupted during download. However, implementing a solution to this problem requires significant changes to the current download process, which isn't a straightforward task. Therefore, I cannot guarantee an immediate fix. Nevertheless, I truly appreciate your thorough and helpful explanation. I will investigate this matter carefully.
Linden10
Hiya, I’m opening this issue in response to your comment on a issue you closed, I’ll be referencing my comment that I made on it so you don’t miss it. I come with a report on the big problem that TwitchLink has, please read it all and thank you.
Hello, I’m happy to see you finally return knowing how busy your life is! So I’ve been using TwitchLink every day for many months and now know the issue properly to the point that I switched from mp4 to ts format for the following reasons:
With ts format, I’ve written (with the help of ai) a custom python script to detect corrupt segments in a ts video file. If any corruption is detected, I re-download the vod if it doesn’t contain muted segments, however if the vod online does contain muted after a stream finishes then with the unmuted vod I have, I manually replace the corrupted segments with uncorrupted ones using ffmpeg.
How I locate the corrupted segments is with ffprobe which will report the pts timestamp of the corrupted segment.
Anyways I’ve been dealing with the corrupted segment issue with vods on a case by case basis, archiving many channels that I work with.
I can confirm that TwitchLink does not make any attempt to re-download or report (except one time where it aborted the download because of corruption) any segments that are corrupted due to not downloading a segment fully, thus leaving in a incomplete segment.
Yt-dlp and streamlink, two programs that can download from Twitch, have a function where they can re-download segments that didn’t download correctly, reporting the error as it attempts to re-download the segment properly.
With TwitchLink, no segments that are downloaded incorrectly are either reported (unless you’re downloading using mp4 as ffmpeg reports a corrupted segment in the log file) or re-downloaded to fix the issue.
I’ve downloaded tons of vods so I know the issue at hand. If you want to test this out, simply download any vod (preferably large like ironmouse) and check the logs for any corrupted segments. For ts files, you can use ffprobe to tedt for corrupted segments.
Anyways that is my report, I’ve downloaded close to 2TB worth of vods since October and hoping that you figure out a way for TwitchLink to check if each segment it downloads is not incomplete or corrupted like in yt-dlp and streamlink.
If you need anything, do tell and thank you! I personally love the Update Track feature as it helped me archive vods that would be muted after the stream finishes! Despite the bugs, I still love and use it to this day! Yup! 😁
Originally posted by @Linden10 in https://github.com/devhotteok/TwitchLink/issues/176#issuecomment-1986304713
I like to add that, they’re times that when downloading a large vod, it’s more prone to be corrupted vs when downloading a stream through update track as it downloads a few segments per update.
This can be fixed by introducing a check if a segment downloaded fully or not. You can look at either streamlink or yt-dlp for reference on that. Hope this helps and again if you need anything, just ask and thank you!