search

bibanon / tubeup

Use yt-dlp to download video and upload to the Internet Archive with metadata.
https://pypi.python.org/pypi/tubeup/
GNU General Public License v3.0
407 stars 70 forks source link

[Feature Request] Add command --concurrent-fragments #189

Open dunanight opened 2 years ago

dunanight commented 2 years ago

yt-dlp supports multi-threaded fragments downloads with --concurrent-fragments, I was wondering if it was possible to add this command for faster vod downloads.

brandongalbraith commented 2 years ago

@dunanight what are the potential downsides to using this argument? Any chance of blacklisting or rate limiting by a target service?

dunanight commented 2 years ago

I am not sure about any potential downsides to using this argument. I'd imagine if there is any issues it would be listed in the yt-dlp page. I personally have used the youtube-dl --external-downloader aria2c for multi-threaded downloads in the past on 100+ twitch vods and never had any problems.

vxbinaca commented 2 years ago

Turning it of by default will cause serious issues with providers like Instagram, who very aggressively throttle rip attempts if it thinks you're using a CLI downloader. I can look into it. What downloader does it use? Does it force aria2c? We'd have to update deps for that.

vxbinaca commented 2 years ago

this does speed up downloading for that one extractor/site, but only by a 25 percent in my testing.

Push up a PR with this as a flag and I'd be willing to merge it.

vxbinaca commented 1 year ago

I'm concerned this would break sites like youtube. I vote this issue be kept open and investigated though.