bibanon / tubeup

Use yt-dlp to download video and upload to the Internet Archive with metadata.
https://pypi.python.org/pypi/tubeup/
GNU General Public License v3.0
408 stars 70 forks source link

Troubles facing video archival #317

Open vxbinaca opened 11 months ago

vxbinaca commented 11 months ago

Over the last 72 hours I've been triaging two or three issues that have either come up on Youtubes end, or with dependencies of yt-dlp.

The symptom of this issue is a item that has no video in it, this is due to the mux failure leaving .part files that we have safeties in place to not shove junk into IA. Tubeup users should be monitoring the output of their rips anyway. Removing the safety will merely upload junk .part files to IA, where the problem won't be immediately noticed until it's too late.

I would recommend signing up for Premium on a burner account, using a debit card you don't mind not being able to use if you're banned. Do not use your main accounts cookie.

mrpapersonic commented 1 month ago

One thing that has annoyed me for ages about tubeup is that when dealing with playlists, instead of doing the obvious thing (that is, downloading and then uploading each video) it downloads each video and then uploads each video. It's such a waste of disk space that could easily be overcome by just doing the obvious, and FWIW I get around this issue myself by scripting yt-dlp and tubeup together...

One issue that is possible is when there is a private video within the playlist. Tubeup already errors out on these, but it would be diverging behavior from previous releases to error out after uploading part of a playlist. But... in any case private or unavailable videos in playlists ought to be considered warnings at best anyway and tubeup should just skip over them. It's better to warn out and preserve more videos than error out and risk losing the archival of some videos just because one video was private. Of course the main issue would be these getting lost in the console; a simple and effective solution is providing a summary at exit that users can evaluate to make sure everything went smoothly and printing the URLs of videos that couldn't be scraped.

mrpapersonic commented 1 month ago

Actually, with private videos it should (?) already be handled the way I described (as a warning...): https://github.com/bibanon/tubeup/blob/4ca730ca3f10ca5fe7bc10c36fe0a71ece67aad5/tubeup/TubeUp.py#L120-L123

so maybe that can be disregarded. The part about providing a summary is still a good idea imo, and its what other downloaders like aria2c do as well.

vxbinaca commented 4 weeks ago

code something up and lets merge it

mrpapersonic commented 4 weeks ago

now if only I had the time... :)

vxbinaca commented 4 weeks ago

I understand business has picked up for me thankfully.