search

ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
132.6k stars 10.05k forks source link

Site support request: voe.sx #29232

Open CodeSpoof opened 3 years ago

CodeSpoof commented 3 years ago

Checklist

Example URLs

Description

Some videos on VOE are restricted to embed view, but I didn't find any of that kind by search, that would comply with the beforementioned rules for links. Those urls can be transformed, by simply adding/removing the "/e" in the url which makes them embed. Some videos also are restricted to embed view and restricted to watch by region. That can be bypassed sometimes, when one calls the non-embed video URL once and calls the embed URL afterwards in the same tab. Videos for both special cases, I found using search term "site:voe.sx", but those videos where either NSFW or they violated copyrights, so if you can't implement those special cases, because of the lack of safely obtainable information, it wouldn't be neccessary.

SuperSonicHub1 commented 3 years ago

@CodeSpoof I dunno man, maybe I'm losing it after attempting to triage too many youtube-dl issues in one day, but when I open up my DevTools console, I end up stuck in a loop of constantly running anonymous functions with debugger statements, effectively stopping me from investigating the site. If this is an anti-tamper measure, it's the most simple but genius I've seen yet. Since debugger statements only stop evaluation when DevTools is open, a normal user never notices what's happening. Pretty crazy. Could you tell me if this is happening to you, too?

SuperSonicHub1 commented 3 years ago

For anyone else who tries this, debugger statements can be disabled with the appropriate button in your Sources tab or hitting Ctrl-F8.

SuperSonicHub1 commented 3 years ago

How to Crack Voe

URL parsing

<id> => 12-character long jumble of lowercase letters and numbers
https://voe.sx/<id>
https://voe.sx/e/<id> [embed]

Download Video/Embed

Surprisingly easy. Someone should figure out that table thing though so we have all the metadata we can get. If I never get around to making the extractor, I leave it as an exercise to the reader.

SuperSonicHub1 commented 3 years ago

@CodeSpoof Now that a basic proof for extracting has been made, I would like for you to share those interesting edge cases with us, even if their contents are a bit dubious.

CodeSpoof commented 3 years ago

Edge case (region restriction in embed view, call not embedded, then embedded to view): https://voe.sx/e/6926ecve5aia https://voe.sx/e/ifprbv97b4u7

Region restriction only tested for:

Tried different accept-language headers, didn't seem to change anything but just in case: accept-language = de,en-US;q=0.7,en;q=0.3

SuperSonicHub1 commented 3 years ago

@CodeSpoof https://voe.sx/e/ifprbv97b4u7 works just fine for me without doing your trick, and https://voe.sx/e/6926ecve5aia doesn't work for me at all. I'm in the US, what region are you in?

CodeSpoof commented 3 years ago

Germany

nikolasr commented 3 years ago

Well, please feel free to ignore me :-) ,but I'm having issues with voe.sx, too. Not limited to anything embedded, and, yes, using a german ISP, too (Unitymedia/now Vodafone). To add another sample, https://voe.sx/2ek0wunngaqz doesn't work for me either.

euclid2021 commented 2 years ago

For me, using the .m3u8 link from the HTML works. Example:

curl https://voe.sx/e/5vit9t9dgwwm | grep m3u8

Gives me: "hls": "https://delivery-node-gaber.voe-network.net/hls/,6oarn4lxvi33cszcrzu5h3zqwglee2gl4c65rxvbaer6rysvfmr2qqmz3f5q,.urlset/master.m3u8",

Now I can just download that link using youtube-dl:

youtube-dl https://delivery-node-gaber.voe-network.net/hls/,6oarn4lxvi33cszcrzu5h3zqwglee2gl4c65rxvbaer6rysvfmrx2w4z3f5q,.urlset/master.m3u8

dirkf commented 2 years ago

Still good.

Michtdu commented 2 years ago

The solution from euclid2021 doesn't work anymore

the curl output says hls2 in the url so i guess thats the issue

curl https://voe.sx/e/5vit9t9dgwwm | grep m3u8

youtube-dl https://delivery-node-fadilah.voe-network.net/engine/hls2/01/00329/5vit9t9dgwwm_n/master.m3u8?t=LX2_bqXXcXceD7SbLQvBUpFUR3bLH4tAdXrc8Jp9uoE&s=1668709280&e=14400&f=1645965&node=delivery-node-fadilah.voe-network.net&i=46.57&sp=4500&asn=8437 --verbose

WARNING: Could not send HEAD request to https://delivery-node-fadilah.voe-network.net/engine/hls2/01/00329/5vit9t9dgwwm_n/master.m3u8?t=LX2_bqXXcXceD7SbLQvBUpFUR3bLH4tAdXrc8Jp9uoE: HTTP Error 403: Forbidden [generic] master: Downloading webpage ERROR: Unable to download webpage: HTTP Error 403: Forbidden (caused by <HTTPError 403: 'Forbidden'>); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see https://yt-dl.org/update on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. youtube-dl https://delivery-node-fadilah.voe-network.net/engine/hls2/01/00329/5vit9t9dgwwm_n/master.m3u8?t=LX2_bqXXcXceD7SbLQvBUpFUR3bLH4tAdXrc8Jp9uoE&s=1668709280&e=14400&f=1645965&node=delivery-node-fadilah.voe-network.net&i=46.57&sp=4500&asn=8437

dirkf commented 2 years ago

Remember to quote URLs passed to yt-dl, especially if they contain &. Also ensure that the entire URL from the page is passed.

But now, try the updated PR #29616.

$ python -m youtube_dl -F -v 'https://voe.sx/e/5vit9t9dgwwm'
[debug] System config: [u'--prefer-ffmpeg']
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-F', u'-v', u'https://voe.sx/e/5vit9t9dgwwm']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Git HEAD: 774b346f9
[debug] Python version 2.7.17 (CPython) - Linux-4.4.0-210-generic-i686-with-Ubuntu-16.04-xenial
[debug] exe versions: avconv 4.3, avprobe 4.3, ffmpeg 4.3, ffprobe 4.3
[debug] Proxy map: {}
[voe] 5vit9t9dgwwm: Downloading webpage
[voe] 5vit9t9dgwwm: Downloading m3u8 information
[info] Available formats for 5vit9t9dgwwm:
format code  extension  resolution note
0            mp4        720p       
353          m3u8       640x288     353k , avc1.640029, 23.974fps, mp4a.40.2 (best)
$
JanzenJohn commented 1 year ago

the PR does not seem to work for me because the url they redirect me to "scatch176duplicities.com" is not recognized (but seems to be just a voe.sx mirror)

dirkf commented 1 year ago

It's more helpful to comment in the PR itself: I've copied this over.

CRYST4LF1GHT3R commented 1 week ago

Any updates on this?

dirkf commented 1 week ago

Try the newly updated PR #29616, which fixes https://github.com/ytdl-org/youtube-dl/issues/29232#issuecomment-1518539826.

Also, please provide at least one current URL from this site where there is a possibility that the uploader has the right to do so (own content, fair use, etc).

In either case, pls respond in the PR.

nikolasr commented 5 days ago

@CRYST4LF1GHT3R You might want to take a look at this project: https://github.com/p4ul17/voe-dl :-)

nikolasr commented 5 days ago

Also note that youtube-dl fork yt-dlp https://github.com/yt-dlp/yt-dlp has declined to support voe.sx because it's often (but not always!) used for copyright infringement. So I don't know what the policy over here is for this kind of "dual-use" sites...

dirkf commented 5 days ago

Hence why I asked for test URLs where

... there is a possibility that the uploader has the right to do so (own content, fair use, etc).

See https://github.com/ytdl-org/youtube-dl/issues/32716#issuecomment-1927180948.

dirkf commented 5 days ago

In fact: https://voe.sx/abuse

The linked tool is a wrapper for yt-dl in what appears to be the normal case that only HLS formats are offered, and otherwise doesn't appear to add anything to the current PR extractor.

TBH it would be better for users to help complete the long-delayed PR, as requested.