awebeer256
commented
3 years ago In other parallelism requests, someone mentioned the potential problem of the links passed to the external downloader expiring. Furthermore, it now occurs to me that with my initial proposal, youtube-dl wouldn't necessarily know when each download is finished, which could be desirable for multiple reasons. And while I think there are acceptable solutions to both of these (mostly involving having youtube-dl look at the hard drive), they could alternatively be simply avoided by compromising on my proposal. Currently, youtube-dl calls the external downloader once per file, and my initial proposal was to change this to once per invocation of youtube-dl itself, but there's also the option of calling it once per video.
Checklist
Description
Using aria2c with multiple connections allows me to actually use all of my bandwidth...except when it doesn't. For example, if I try to download a YouTube video whose audio and video are separate files, youtube-dl will first call aria2c with the video, which will download super fast, and then when that's finished, youtube-dl will call aria2c again with the audio, which will download at 2.5Mbps because the file is too small to be split.
So my request is this: when using youtube-dl with an external downloader, in such a way that will result in multiple files being downloaded (whether it's because multiple URLs were passed to youtube-dl, or a playlist URL was passed, or you're downloading one video in multiple "formats," or some combination of the above), youtube-dl should first determine all the URLs to be downloaded and then call the external downloader once, passing it all of those URLs. This will properly facilitate what is presumably the main reason external downloaders are supported in the first place: letting them handle parallelism. They can't properly do that if you only give them one URL at a time!