search

Kethsar / ytarchive

Garbage Youtube livestream downloader
MIT License
1.08k stars 87 forks source link

Missing fragments #66

Closed Berni11 closed 2 years ago

Berni11 commented 2 years ago

I usually use 5 threads to download a stream and noticed that sometimes 1 or more fragments are missing from the video, the audio or both. Increasing the threads number usually results in more fragments missed. This happens more often when I start the download when the stream has been running 2-3 hours,when the stream ends or when I use the --video-url and --audio-url option.

Kethsar commented 2 years ago

How does this end up manifesting in the final file? Is there an obvious skipped second, does it fail to mux, does it show which fragment got missed or that there are an unequal amount between audio and video? I've not run into this and the logic should be such that if a fragment actually doesn't get grabbed and something causes it to somehow skip it, it would stop the whole download instead. I guess it's possible the data somehow fails to transfer to the file but otherwise passes all the other steps. But I need more info to have any idea if that's the case.

Berni11 commented 2 years ago

The files are still able to mux, there's no error. It doesn't tell me which fragments are skipped but it says "Mismatched number of video and audio fragments. The files should still be mergable but data might be missing.". I haven't tried running it with --debug yet. I was able to compare an incomplete download with a complete one (I restarted the download after it was incomplete) and it seems that the missing fragments are at the end of the file. Playing the muxed file results in the audio or video being longer.

Kethsar commented 2 years ago

What's the most amount of time difference you've seen between audio and video from this so far? My first guess would be youtube not fully processing/sending out some of one or the other, given it happens at the end.

Has this happened since you opened the issue? Do you have any console output from such a case?

sebi-berlin commented 2 years ago

What's the advantage on using multiple threads on a single stream?

Kethsar commented 2 years ago

@sebi-berlin much quicker catch-up if you start downloading well after a stream has started. Also if the download is being throttled, higher chance you keep up with it instead of falling behind. That one depends on the number of threads and how hard it's being throttled.

Berni11 commented 2 years ago

Sorry for the late replay. Since I've opened the issue it happened less often but it happens from time to time.

I tried collecting some console outputs with the debug option on. I used the --video-url and --audio-url option for them because the issue happens more often when I use them.

https://gist.github.com/Berni11/bf9d00a82ec4bab118c7a3a4394e61c2 https://gist.github.com/Berni11/ef62dd08c96b3b85f31e694372bd54e8 https://gist.github.com/Berni11/456de05c55a73ef33ec68c47dbfe98f4 https://gist.github.com/Berni11/25180ba804465ed0da75d796123ea611

Kethsar commented 2 years ago

That is indeed quite odd. I have an idea of what might be causing this. I'll try to get a fix out here in a bit, but whether it works or not will probably take a while to test.

Kethsar commented 2 years ago

I've pushed out an update, you can download it from the releases page Let me know if it eventually pops up again.

Kethsar commented 2 years ago

Since there's been no new complaints about this (that I've seen), I'll close this for now.