devhotteok / TwitchLink

Twitch Stream & Video & Clip Downloader/Recorder. The best GUI utility to download/record Broadcasts/VODs/Clips.
https://twitchlink.github.io
MIT License
389 stars 31 forks source link

Downloaded file is much shorter than it’s supposed to be, but retains its correct filesize #176

Closed 6xolotl closed 6 months ago

6xolotl commented 10 months ago

So for context, I’m trying to download parts of a subathon that one of my favourite streamers did. The day 1 vod is just shy of 47 hours long, but I’ve downloaded it twice now and both downloads have resulted in a video clocking in at about 20 hours, despite the filesize being correct. Am I going insane?? Am I the problem???

The filesize is ~126gb, if that matters. I have all the drive space I need, so that’s not the problem. This is around the 4th or 5th program I’ve tried to download this vod with, and it’s the only one that looks promising, so I’m hoping someone will be able to tell me what’s wrong so that I can fix it!!

Linden10 commented 10 months ago

I noticed something just now after downloading a bunch of vods, that sometimes the packets become corrupt thus data becomes lost/corrupted.

I don't know if this is the same situation that I'm having but to confirm it, check the logs for your vods and see if any "corrupt packet" comes up. If you see any in the logs, that's the reason and the solution is...I don't know.

I guess try downloading the vod in concat mode (not sure if that'll fix it, I'm trying it right now and plan on using yt-dlp's FixupM3u8 function to remux the vod offline) and see if that works...

EDIT: Actually yt-dlp only works with urls so for offline remuxing, use ffmpeg with this command to remux the vod properly: ffmpeg -i video.mp4 -movflags +faststart -vcodec copy -acodec copy -absf aac_adtstoasc output.mp4

Otherwise...I don't know, sorry. I will say though, I wish TwitchLink reported that the vod had corrupted packets because I only just found out now that ffmpeg sometimes messes up with the downloads!

devhotteok commented 6 months ago

This issue could have a variety of causes. It's possible that the file itself is corrupted, or there may be an issue with the video player you're using. If it's neither of these, it could be related to the FFmpeg library that our program uses internally. If you're experiencing this problem frequently with different videos, please let us know. I will consider updating the internal FFmpeg in a future release if possible.

Linden10 commented 6 months ago

This issue could have a variety of causes. It's possible that the file itself is corrupted, or there may be an issue with the video player you're using. If it's neither of these, it could be related to the FFmpeg library that our program uses internally. If you're experiencing this problem frequently with different videos, please let us know. I will consider updating the internal FFmpeg in a future release if possible.

Hello, I’m hqppy to see you finally return knowing how busy your life is! So I’ve been using TwitchLink every day for many months and now know the issue properly to the point that I switched from mp4 to ts format for the following reasons:

With ts format, I’ve written (with the help of ai) a custom python script to detect corrupt segments in a ts video file. If any corruption is detected, I re-download the vod if it doesn’t contain muted segments, however if the vod online does contain muted after a stream finishes then with the unmuted vod I have, I manually replace the corrupted segments with uncorrupted ones using ffmpeg.

How I locate the corrupted segments is with ffprobe which will report the pts timestamp of the corrupted segment.

Anyways I’ve been dealing with the corrupted segment issue with vods on a case by case basis, archiving many channels that I work with.

I can confirm that TwitchLink does not make any attempt to re-download or report (except one time where it aborted the download because of corruption) any segments that are corrupted due to not downloading a segment fully, thus leaving in a incomplete segment.

Yt-dlp and streamlink, two programs that can download from Twitch, have a function where they can re-download segments that didn’t download correctly, reporting the error as it attempts to re-download the segment properly.

With TwitchLink, no segments that are downloaded incorrectly are either reported (unless you’re downloading using mp4 as ffmpeg reports a corrupted segment in the log file) or re-downloaded to fix the issue.

I’ve downloaded tons of vods so I know the issue at hand. If you want to test this out, simply download any vod (preferably large like ironmouse) and check the logs for any corrupted segments. For ts files, you can use ffprobe to tedt for corrupted segments.

Anyways that my report, I’ve downloaded close to 2TB worth of vods since October and hoping that you figure out a way for TwitchLink to check if each segment it downloads is not incomplete or corrupted like in yt-dlp and streamlink.

If you need anything, do tell and thank you! I personally love the Update Track feature as it helped me archive vods that would be muted after the stream finishes! Despite the bugs, I still love and use it to this day! Yup! 😁