Use Youtube-dl for video downloads

[heyhippari]

A few of the rippers overlap with Youtube-dl.

It would be interesting to provide a "Youtube-dl ripper" that would use youtube-dl as the downloader for all the websites it supports, as well as serve as a backend to download videos for other rippers.

This would make updating some rippers with website changes much easier, as youtube-dl has a lot of contributors and is updated pretty quickly.

The easiest way I could see would be to use Jython to invoke youtube-dl, as outlined in the Jython book.

Another potential way would be to use the system's youtube-dl, but then we'd have to parse its output, which isn't standardized.

Obviously, this would be a pretty substantial undertaking, but I think it's interesting to at least discuss the idea, as it would make some rippers much easier to maintain in the long term and add support for a whole bunch of websites in one go.

[rautamiekka]

There's no guarantee Jython will run the code. Someone needs to test that.

[cyian-1756]

Another potential way would be to use the system's youtube-dl, but then we'd have to parse its output, which isn't standardized.

Couldn't we just pipe the output right to the text box of the app and just use stderr to tell if failed or not?

Anyhow I agree with adding youtube-dl support

edit: According to this thread youtube-dl has (basic) jython support

[heyhippari]

@rautamiekka Yep, that would need to be checked. This is just a discussion to propose the idea, though.

@cyian-1756 Yep, we could do that. How would the dependency work on Windows, though ? It's in the public license, so it can be shipped with ripme directly, but on Linux we should try to use the system one first.
We could check if "youtube-dl" is in PATH (Works on everything), then have a configuration field in the preferences to use a custom binary, perhaps with a checkbox to activate/deactivate it that would switch to an embedded version.

Embedding youtube-dl would complicate the build process a little though, to keep the cross-platform stuff. Windows is easy (.exe), Linux/BSD as well (require it as a dependency in all packages), but I don't know how MacOS works.

For cross-platform stuff Apache Commons Lang could enable us to detect which OS we're running on and do OS-specific stuff.

This all complicates things a fair bit and is quite a big rewrite, though.
Still, I think in the end it'd be worth it, as it would make further rippers stupidly easy to add for video sites.

Edit: Java also has a os.name in System.getProperties(), but if it returns something different for every version of Windows or every distribution of Linux, it might not be maintainable.

[rautamiekka]

youtube-dl dev made very extensive effort at making his code run on many Python versions (they recently dropped support for some dead versions) on whatever system someone managed to sensibly run Python though, so if Jython doesn't do anything stupid it might work.

[heyhippari]

So, with Youtube-dl at least supporting Jython partially, I think a sensible course of action for execution would be:

1. Use the system's Youtube-dl (Check if it's in the PATH beforehand)
2. Use an embedded Jython version.

This would ensure that it works on Windows without having to install anything (Since it's embedded) and should work on Linux/BSD as well, as youtube-dl would be installed by the package manager as a dependency.
Embedding it with Jython should ensure cross-platform support, so no need to spaghetti code our way through this with platform detection and multiple binaries.

[rautamiekka]

^ Embedding would increase the JAR release rate by a lot since the software gets updated every so often.

[heyhippari]

The embedded would only be a fallback.
It would use the system one by default, then fallback to the embedded.

We could pop up a message on first run if youtube-dl isn't in PATH to warn the user that the embedded version might not be the latest one available and that they can (and should) install their own.

When TravisCI is up again, maintaining new JARs will be much faster and easier.
Basically, it will test and release every tagged commit, so we can setup the build system to pull youtube-dl and embed it automatically.

But being slightly outdated shouldn't be an issue if the first thing the software does is try to use the system version.
Ripme releases new JARs pretty often anyway, so users that can't be bothered won't be too penalized.

[ghost]

I would love it if this will work for individual videos, playlist videos, and all videos of a profile.

[lineofbestgit]

I would love it if this will work for individual videos, playlist videos, and all videos of a profile.

youtube-dl supports this very well

[heyhippari]

@tehloxely Yep, that's the idea :) If youtube-dl supports it, we would simply pass the URL along to it with a set of parameters.

I think before this is worked on, we'd need to make releasing a Jar easier. I'll look at what can be done with Maven, Git Tags and Travis on my fork, then I'll submit a PR for review and, hopefully, this'll make releasing easier (I'm also thinking of killing two birds with one stone and make Maven generate a changelog, but we'd need to standardize our commit messages somewhat, before that)