mikf / gallery-dl

Command-line program to download image galleries and collections from several image hosting sites
GNU General Public License v2.0
11.7k stars 953 forks source link

Ability to use youtube-dl forks #1330

Closed github-account1111 closed 3 years ago

github-account1111 commented 3 years ago

Kinda like what they're doing at mpv: https://github.com/mpv-player/mpv/pull/8207

For example, youtube-dlp lets you embed thumbnails and has a more robust video quality selection, so I'm thinking of moving to it.

Unfortunately, the executable is youtube-dlc.exe, so other tools like gallery-dl (or mpv) won't be able to see it in path.

Hrxn commented 3 years ago

[..] and has a more robust video quality selection [..]

How so?

rautamiekka commented 3 years ago

EDIT: Fixed a couple brain farts.

If all else fails, you can create a symlink inside the folder where the youtube-dl binary's expected: Window$: 1) Open Command Prompt or PowerShell as Admin. This is cuz by default Window$ doesn't allow {hard|sym}linking without Admin. 2) X:, where X is the drive letter. 3) CD TARGET, where TARGET is where the youtube-dl binary is. 4) MKLINK youtube-dl.exe youtube-dlc.exe, dropping the .exe if not using it. Notice that Window$ MKLINK reverses the order of params from the NIX ln: name of the link, name of the target

NIX (Linux, Mac, ...): 1) cd TARGET, where TARGET is the folder where youtube-dl should be. 2) ln -vs youtube-dlc youtube-dl, assuming the folder doesn't require root, in which case prepend sudo (or whatever the distro/platform equivalent's; I recall seeing some Linux distros use something else for the same effect).

mikf commented 3 years ago

gallery-dl uses the Python package, not the executable, but you can still do the same thing and create a youtube_dl symlink to the fork.

github-account1111 commented 3 years ago

Wait.. but I don't even have Python installed on this system. Is there really a difference between an executable with an entry inside path and a Python package? I thought it's the same thing. Like you call it the same way (youtube-dl -f etc. etc.), right?

github-account1111 commented 3 years ago

[..] and has a more robust video quality selection [..]

How so?

Format Sorting: The default format sorting options have been changed so that higher resolution and better codecs will be now preferred instead of simply using larger bitrate. Furthermore, you can now specify the sort order using -S. This allows for much easier format selection that what is possible by simply using --format

mikf commented 3 years ago

Is there really a difference between an executable with an entry inside path and a Python package?

It is only possible to import a Python module, not a Windows executable, and use its internal classes, functions, and data structures and that's how gallery-dl interacts with youtube-dl.

I guess you can't make use of this feature then, unless you built the executable with yt-dlp instead of youtube-dl inside, but you'd need to install a Python interpreter to do that and then you might as well use gallery-dl, youtube-dl, etc directly as Python modules themselves.

github-account1111 commented 3 years ago

I do have conda installed which comes with pip if that makes any difference, but I still prefer self-contained executables to packages and dependencies on Windows.

So then does the gallery-dl exe come with an embedded youtube-dl? If so, that wasn't very clear to me from the README. Does it also have an embedded ffmpeg? Seeing how the README lists both under "Optional", I assumed that if I had them in path (which I do), whatever functionality depends on them will be activated and working.

github-account1111 commented 3 years ago

Following up. I'm just not sure how to verify both youtube-dl and ffmpeg are being utilized by gallery-dl. Never used Python for anything other than programming. I have Conda directly on Windows and Python 3 on WSL (which I almost never use).

mikf commented 3 years ago

gallery-dl uses ffmpeg the same way youtube-dl does, i.e. you need to have ffmpeg.exe in PATH. When you can run ffmpeg in your shell, so can gallery-dl and youtube-dl. You can run the following as test:

$ gallery-dl --ugoira-conv -o ugoira=true https://danbooru.donmai.us/posts/1830576

For youtube-dl, the Python interpreter running gallery-dl needs to be able to import youtube_dl (or whatever youtube-dl fork you want to use), so python -c "import youtube_dl" needs to return without error. To test that:

$ gallery-dl -o videos=ytdl -o downloader.ytdl.module=youtube_dl https://twitter.com/perrypumas/status/1065692031626829824

module can be changed to anything you want, e.g. yt_dlp

github-account1111 commented 3 years ago

Thanks for bearing with me and giving such a thorough explanation! I did end up switching to python packages for both gallery-dl and yt-dlp. I thought it would be problematic given I use conda and didn't want more than one python interpreter in the system, but it turns out it's a non-issue. What I ended up doing (in case anyone is in a similar situation):

conda create -n my_env pip idna pycryptodome requests
conda activate my_env
python -m pip install --upgrade yt-dlp gallery-dl

If you do not care about minimizing the number of non-conda packages then 1st line could be just conda create -n my_env pip.

Then set downloader.ytdl.module to "yt_dlp" inside the conf file. Now in any script I just need to put conda activate my_env at the top, and it will work as before (but will use yt-dlp now).