search

ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
132.22k stars 10.03k forks source link

Eliminate extractors for deceased websites #29241

Open alerikaisattera opened 3 years ago

alerikaisattera commented 3 years ago

Certain websites supported by youtube-dl (for instance, liveleak) have been closed. I suggest removing extractors for these websites, for they don't serve any purpose anymore.

pukkandan commented 3 years ago

You can't expect the maintainers to periodically go through the massive list of supported websites and check which are alive. I suggest you open an issue (or even better, a PR) instead when you find a dead website

rautamiekka commented 3 years ago

No need to go check the sites manually, it can be scripted (then manually googled for a little. Not a huge amount of work, but ytdl devs are already overly busy), even just receiving the HTTP response code is enough.

alerikaisattera commented 3 years ago

You can't expect the maintainers to periodically go through the massive list of supported websites and check which are alive. I suggest you open an issue (or even better, a PR) instead when you find a dead website

Fair point. Maybe I should try to check the supported sites list myself to see which ones are dead

alerikaisattera commented 3 years ago

No need to go check the sites manually, it can be scripted (then manually googled for a little. Not a huge amount of work, but ytdl devs are already overly busy), even just receiving the HTTP response code is enough.

This may be a good idea, but it would fail to distinguish a truly dead website from just a temporarily down one

pukkandan commented 3 years ago

Yes, @rautamiekka 's suggestion can help narrow down the potential cases, but at the end, a manual review of each website is needed

rautamiekka commented 3 years ago

No need to go check the sites manually, it can be scripted (then manually googled for a little. Not a huge amount of work, but ytdl devs are already overly busy), even just receiving the HTTP response code is enough.

This may be a good idea, but it would fail to distinguish a truly dead website from just a temporarily down one

Ya, it can't tell a temporarily dead one, but it does help to narrow them down massively.

With the temporary death in mind, the script could be extended to warn after, say, 5 failures over, say, 5 weeks (1 check/failure a week). If the site's still returning the same response after the a bit over a month of time it's quite likely staying down.