search

ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
129.68k stars 9.78k forks source link

Discovery + How to download using new format? #28160

Open Danker1990 opened 3 years ago

Danker1990 commented 3 years ago

Checklist

Question

I want to use discovery+ however it isn't as clear to directly download a source due to the url sticking to in my case https://www.discoveryplus.co.uk/show/chasing-classic-cars but can't find the individual listing of the url. How do i get the individual listing?

Diegus83 commented 3 years ago

I'm not sure if the extractor is working for discoveryplus.co.uk, but at least I can confirm it is not working for discoveryplus.com

What is working for me in the US site and will probably work for you too, if you start playing the episode you want to download and open the browser Network tool (command+option+e on Firefox/mac) you can write 'm3u' in the filter field, and you will see several hits as a.m3u8, b.m3u8 and so on. The one with the highest letter is the maximum quality available. Usually you want i.m3u8 for the 1080p stream.

Copy the link for one of those results and pass it to youtube-dl like this: youtube-dl https://content-ause2-ur-discovery1.uplynk.com/9dffd9cd960b4836932c210/f.m3u8?pbs=aef6d5ba29134458

If you want an easier way to get a whole season or a whole show, I haven't found it yet. Let me know if this method works for you.

ohmybahgosh commented 3 years ago

The only thread for this is locked to contributers currently, but I'd like to supply any dev with the time to implement this, my credentials to login to discoveryplus.com

remitamine commented 3 years ago

@ohmybahgosh you can contact me via email address in my profile.

ohmybahgosh commented 3 years ago

@ohmybahgosh you can contact me via email address in my profile.

Just emailed the credentials

Mr-Jake commented 3 years ago

Thanks so much for adding discoveryplus.com

I didn't sign up for an account yet so I can't test it, but can the extractor use username/password or does it use cookies?

Diegus83 commented 3 years ago

I always used cookies, never got it to work with user/pass but I haven't tried the latest commit yet so I can't be 100% sure.

Edit: just tested the last commit and you definitely has to pass the cookies, username/password are not suported.

I did observed that for some reason is downloading episodes with ads while if I play the episode in the browser I get no ads.

coopermor commented 3 years ago

I have the subscription with ads and when I downloaded episodes the downloaded files did not have ads.

Diegus83 commented 3 years ago

When I extract the m3u manually as I described before the downloads have no ads either. For some reason the extractor is getting a stream with ads even with the correct cookies.

ohmybahgosh commented 3 years ago

I supplied @remitamine with my credentials, and those credentials were for a premium subscription. So if you do not have a premium subscription you should ask @remitamine if you can send over your free account credentials.

Back to the inclusion via this update, everything works great on my end via a premium sub!

Just remember to use a Netscape cookie and make sure you include cookie data from not just www.discoveryplus.com but the root domain as well discoveryplus.com

Diegus83 commented 3 years ago

We already figured it out, the venv where I was testing the code didn't had the PyCrypto package so youtube-dl was delegating the download to ffmpeg. It was a weird case because when passing the m3u8 directly to youtube-dl or ffmpeg (with ffmpeg -i) the download was fine (eg. no ads). Thanks so much @remitamine and everybody else for your time.

nestukh commented 3 years ago

@Diegus83 solution almost works for discoveryplus.it. The correct m3u8 url is the one with playlist.m3u8 in the name, the others are the unskippable ads, but the audio doesn't work (the resulting downloaded mp4 has no audio track). In the Response tab of this playlist.m3u8, there is additional info about different resolutions of the same video, but I haven't investigated it further.

Example link: https://www.discoveryplus.it/programmi/fratelli-di-crozza

Anyway, not providing single pages for each episode is the webdesign equivalent of hostile architecture. It's like they've got a fully staffed Banality of Evil Department for thinking such horrors, or something like that

Diegus83 commented 3 years ago

In the US site they don’t a page with episode information but when you click in one it does loads in it’s own URL like this https://www.discoveryplus.com/video/mythbusters/poppy-seed-drug-test

If you know how to run from the source code you can download the current repository and test the new extractor for discovery.

nestukh commented 3 years ago

@Diegus83 I can't see your link: it's geografically locked, it redirects to discoveryplus.it for me. In the Response tab of that playlist.m3u8 there is also info for audio tracks, but I haven't managed to download any of those partial URIs, somehow connected to dplaysouth-vod.akamaized.net

Hoping to help devs from abroad, here some data.

Example: https://www.discoveryplus.it/programmi/fratelli-di-crozza last episode

m3u8 URL: https://dplaysouth-vod.akamaized.net/dplaydni/202480/0/hls/10082639004/playlist.m3u8?hdnts=st=1613809501~exp=1613895901~acl=/dplaydni/202480/0/hls/10082639004/*~hmac=cda9685f4698b4e976d0798d11b7772485789d6a1faec0674d5d069a4a7bf77d

Response payload:

#EXTM3U
#EXT-X-VERSION:4
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="64000mp4a.40.2",LANGUAGE="eng",NAME="eng",AUTOSELECT=YES,DEFAULT=NO,CHANNELS="2",URI="exp=1613895902~acl=%2f*~data=hdntl~hmac=386d38681df50e2fcb61a063687a2da6a2a732c847ec24def0eebb6b508eeb12/994506352-prog_index.m3u8?version_hash=c2dcd1b9"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="160000mp4a.40.2",LANGUAGE="eng",NAME="eng",AUTOSELECT=YES,DEFAULT=NO,CHANNELS="2",URI="exp=1613895902~acl=%2f*~data=hdntl~hmac=386d38681df50e2fcb61a063687a2da6a2a732c847ec24def0eebb6b508eeb12/995756510-prog_index.m3u8?version_hash=c2dcd1b9"
#EXT-X-STREAM-INF:BANDWIDTH=1756864,AVERAGE-BANDWIDTH=1756864,RESOLUTION=960x540,FRAME-RATE=25.000,VIDEO-RANGE=SDR,CODECS="avc1.4D401F,mp4a.40.2",AUDIO="160000mp4a.40.2"
exp=1613895902~acl=%2f*~data=hdntl~hmac=386d38681df50e2fcb61a063687a2da6a2a732c847ec24def0eebb6b508eeb12/805799613-prog_index.m3u8?version_hash=c2dcd1b9
#EXT-X-STREAM-INF:BANDWIDTH=234112,AVERAGE-BANDWIDTH=234112,RESOLUTION=320x180,FRAME-RATE=25.000,VIDEO-RANGE=SDR,CODECS="avc1.42C015,mp4a.40.2",AUDIO="64000mp4a.40.2"
exp=1613895902~acl=%2f*~data=hdntl~hmac=386d38681df50e2fcb61a063687a2da6a2a732c847ec24def0eebb6b508eeb12/141335910-prog_index.m3u8?version_hash=c2dcd1b9
#EXT-X-STREAM-INF:BANDWIDTH=463976,AVERAGE-BANDWIDTH=463976,RESOLUTION=480x270,FRAME-RATE=25.000,VIDEO-RANGE=SDR,CODECS="avc1.42C01E,mp4a.40.2",AUDIO="64000mp4a.40.2"
exp=1613895902~acl=%2f*~data=hdntl~hmac=386d38681df50e2fcb61a063687a2da6a2a732c847ec24def0eebb6b508eeb12/1791633737-prog_index.m3u8?version_hash=c2dcd1b9
#EXT-X-STREAM-INF:BANDWIDTH=863528,AVERAGE-BANDWIDTH=863528,RESOLUTION=640x360,FRAME-RATE=25.000,VIDEO-RANGE=SDR,CODECS="avc1.4D401F,mp4a.40.2",AUDIO="64000mp4a.40.2"
exp=1613895902~acl=%2f*~data=hdntl~hmac=386d38681df50e2fcb61a063687a2da6a2a732c847ec24def0eebb6b508eeb12/938469724-prog_index.m3u8?version_hash=c2dcd1b9
#EXT-X-STREAM-INF:BANDWIDTH=3350528,AVERAGE-BANDWIDTH=3350528,RESOLUTION=1280x720,FRAME-RATE=25.000,VIDEO-RANGE=SDR,CODECS="avc1.64001F,mp4a.40.2",AUDIO="160000mp4a.40.2"
exp=1613895902~acl=%2f*~data=hdntl~hmac=386d38681df50e2fcb61a063687a2da6a2a732c847ec24def0eebb6b508eeb12/1034749509-prog_index.m3u8?version_hash=c2dcd1b9
#EXT-X-STREAM-INF:BANDWIDTH=6527864,AVERAGE-BANDWIDTH=6527864,RESOLUTION=1920x1080,FRAME-RATE=25.000,VIDEO-RANGE=SDR,CODECS="avc1.640029,mp4a.40.2",AUDIO="160000mp4a.40.2"
exp=1613895902~acl=%2f*~data=hdntl~hmac=386d38681df50e2fcb61a063687a2da6a2a732c847ec24def0eebb6b508eeb12/1024744653-prog_index.m3u8?version_hash=c2dcd1b9
#EXT-X-STREAM-INF:BANDWIDTH=64000,AVERAGE-BANDWIDTH=64000,CODECS="mp4a.40.2",AUDIO="64000mp4a.40.2"
exp=1613895902~acl=%2f*~data=hdntl~hmac=386d38681df50e2fcb61a063687a2da6a2a732c847ec24def0eebb6b508eeb12/994506352-prog_index.m3u8?version_hash=c2dcd1b9

#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=21264,AVERAGE-BANDWIDTH=21264,RESOLUTION=320x180,VIDEO-RANGE=SDR,CODECS="avc1.42C015",URI="exp=1613895902~acl=%2f*~data=hdntl~hmac=386d38681df50e2fcb61a063687a2da6a2a732c847ec24def0eebb6b508eeb12/141335910-iframe.m3u8?version_hash=c2dcd1b9"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=49997,AVERAGE-BANDWIDTH=49997,RESOLUTION=480x270,VIDEO-RANGE=SDR,CODECS="avc1.42C01E",URI="exp=1613895902~acl=%2f*~data=hdntl~hmac=386d38681df50e2fcb61a063687a2da6a2a732c847ec24def0eebb6b508eeb12/1791633737-iframe.m3u8?version_hash=c2dcd1b9"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=99941,AVERAGE-BANDWIDTH=99941,RESOLUTION=640x360,VIDEO-RANGE=SDR,CODECS="avc1.4D401F",URI="exp=1613895902~acl=%2f*~data=hdntl~hmac=386d38681df50e2fcb61a063687a2da6a2a732c847ec24def0eebb6b508eeb12/938469724-iframe.m3u8?version_hash=c2dcd1b9"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=199608,AVERAGE-BANDWIDTH=199608,RESOLUTION=960x540,VIDEO-RANGE=SDR,CODECS="avc1.4D401F",URI="exp=1613895902~acl=%2f*~data=hdntl~hmac=386d38681df50e2fcb61a063687a2da6a2a732c847ec24def0eebb6b508eeb12/805799613-iframe.m3u8?version_hash=c2dcd1b9"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=398816,AVERAGE-BANDWIDTH=398816,RESOLUTION=1280x720,VIDEO-RANGE=SDR,CODECS="avc1.64001F",URI="exp=1613895902~acl=%2f*~data=hdntl~hmac=386d38681df50e2fcb61a063687a2da6a2a732c847ec24def0eebb6b508eeb12/1034749509-iframe.m3u8?version_hash=c2dcd1b9"
#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=795983,AVERAGE-BANDWIDTH=795983,RESOLUTION=1920x1080,VIDEO-RANGE=SDR,CODECS="avc1.640029",URI="exp=1613895902~acl=%2f*~data=hdntl~hmac=386d38681df50e2fcb61a063687a2da6a2a732c847ec24def0eebb6b508eeb12/1024744653-iframe.m3u8?version_hash=c2dcd1b9"

Headers:

{"GET":{"scheme":"https","host":"dplaysouth-vod.akamaized.net","filename":"/dplaydni/202480/0/hls/10082639004/playlist.m3u8","query":{"hdnts":"st=1613809501~exp=1613895901~acl=/dplaydni/202480/0/hls/10082639004/*~hmac=cda9685f4698b4e976d0798d11b7772485789d6a1faec0674d5d069a4a7bf77d"},"remote":{"Address":"23.32.238.195:443"}}}

Security:

{"Connection:":{"Protocol version:":"TLSv1.2","Cipher suite:":"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256","Key Exchange Group:":"none","Signature Scheme:":"ECDSA-P256-SHA256"},"Host dplaysouth-vod.akamaized.net:":{"HTTP Strict Transport Security:":"Disabled","Public Key Pinning:":"Disabled"},"Certificate:":{"Issued To":{"Common Name (CN):":"a248.e.akamai.net","Organization (O):":"Akamai Technologies, Inc.","Organizational Unit (OU):":"<Not Available>"},"Issued By":{"Common Name (CN):":"DigiCert Secure Site ECC CA-1","Organization (O):":"DigiCert Inc","Organizational Unit (OU):":"www.digicert.com"},"Period of Validity":{"Begins On:":"July 15, 2020","Expires On:":"September 13, 2021"},"Fingerprints":{"SHA-256 Fingerprint:":"2E:71:F9:23:7F:DE:C2:89:48:6E:D9:3E:C1:35:6C:CA:47:E7:43:6C:6B:40:1F:06:A6:74:A9:FA:17:C3:8A:E4","SHA1 Fingerprint:":"AF:32:D5:A4:A0:9A:25:21:BC:3B:49:18:6E:29:7D:DF:29:43:47:5E"},"Transparency:":"<Not Available>"}}
nestukh commented 3 years ago

@Diegus83 I can git clone but I'm at loss for extracting a single episode with this new discoveryplus site, I will help if you will point me to some instructions

Diegus83 commented 3 years ago

@Diegus83 I can't see your link: it's geografically locked, it redirects to discoveryplus.it for me.

Makes sense, I wanted to get an episode off the Discovery UK website and it is the same situation.

When I pass your m3u8 URL to youtube-dl I can see the download starts but it only shows information for the video

Input #0, hls, from 'https://dplaysouth-vod.akamaized.net/dplaydni/202480/0/hls/10082639004/exp=1613941452~acl=%2f*~data=hdntl~hmac=647a07f413d21de530d72c11ebba76aa495ab520dab3a1a68724d105e1865310/1024744653-prog_index.m3u8?version_hash=c2dcd1b9':
  Duration: 01:04:31.84, start: 0.000000, bitrate: 0 kb/s
  Program 0 
    Metadata:
      variant_bitrate : 0
    Stream #0:0: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Metadata:
      variant_bitrate : 0
Output #0, mp4, to 'file:playlist-_~hmac=cda9685f4698b4e976d0798d11b7772485789d6a1faec0674d5d069a4a7bf77d.f6527.mp4.part':
  Metadata:
    encoder         : Lavf58.45.100
    Stream #0:0: Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 25 fps, 25 tbr, 90k tbn, 90k tbc
    Metadata:
      variant_bitrate : 0
Stream mapping:
  Stream #0:0 -> #0:0 (copy)

But if I pass it directly to ffmpeg like this

ffmpeg -i "https://dplaysouth-vod.akamaized.net/dplaydni/202480/0/hls/10082639004/playlist.m3u8?hdnts=st=1613809501~exp=1613895901~acl=/dplaydni/202480/0/hls/10082639004/*~hmac=cda9685f4698b4e976d0798d11b7772485789d6a1faec0674d5d069a4a7bf77d" -c copy test.mp4

It finds 6 different programs with audio/video like this:

Input #0, hls, from 'https://dplaysouth-vod.akamaized.net/dplaydni/202480/0/hls/10082639004/playlist.m3u8?hdnts=st=1613809501~exp=1613895901~acl=/dplaydni/202480/0/hls/10082639004/*~hmac=cda9685f4698b4e976d0798d11b7772485789d6a1faec0674d5d069a4a7bf77d':
  Duration: 01:04:31.84, start: 0.000000, bitrate: 0 kb/s
  Program 0 
    Metadata:
      variant_bitrate : 1756864
    Stream #0:1(eng): Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 163 kb/s
    Metadata:
      variant_bitrate : 6527864
      comment         : eng
    Stream #0:2: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p, 960x540 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Metadata:
      variant_bitrate : 1756864

I checked a couple minutes and it had audio and didn't had an ad at the beginning.

@Diegus83 I can git clone but I'm at loss for extracting a single episode with this new discoveryplus site, I will help if you will point me to some instructions

I don't know what platform you are in, this link has instructions that will work in linux/mac, but you may be able to adapt them if you are in windows https://twosortoftechguys.wordpress.com/2019/04/23/how-to-install-and-run-youtube-dl-from-actual-source-code-so-that-you-can-apply-patches-as-needed/

Basically once you have you local clone, move to the youtube-dl directory with your command line, and you can run it as a module with

python -m youtube_dl

Note the "_" in the name, that is the name of the actual directory in the repository that contains all the python scripts.

nestukh commented 3 years ago

The good news is that the url with playlist.m3u8 is downloadble with ffmpeg like you've said, yes! But the extractor in youtube_dl doesn't work.

Yes I'm on linux. I've followed the guide, with $HOME/ytdl/youtube-dl/youtube_dl/__main__.py --version i'm using 2021.02.10. But the command python -m youtube_dl "https://www.discoveryplus.it/programmi/fratelli-di-crozza" returns

[generic] fratelli-di-crozza: Requesting header
WARNING: Falling back on generic information extractor.
[generic] fratelli-di-crozza: Downloading webpage
[generic] fratelli-di-crozza: Extracting information
ERROR: Unsupported URL: https://www.discoveryplus.it/programmi/fratelli-di-crozza

In a browser, the example link in the dplay.py extractor, https://www.discoveryplus.it/videos/biografie-imbarazzanti/luigi-di-maio-la-psicosi-di-stanislawskij, just redirects to https://www.discoveryplus.it.

My problem is that there are no more https://www.discoveryplus.it/program/$program_name/$episode_name urls. You go to a tv program page, and if you start playing an episode, a player will open at top of the page and you can watch it. After 2 minutes of unskippable ads, even with uBlock Origin active. But no individual episode link is shown anywhere. At least, as soon as you click "play" on a video, the playlist.m3u8 file appears in the Network tab of the inspector, 2 minutes before you can actually watch it. But you can't do youtube-dl https://www.discoveryplus.it/program/$program_name/$episode_name anymore. There should be a youtube-dl mechanism to tell youtube-dl https://www.discoveryplus.it/program/$program_name/:S03E09 or youtube-dl https://www.discoveryplus.it/program/$program_name/:video+title or something. I'm at loss here.

The layout of discoveryplus.co.uk seems identical to discoveryplus.it, I can go to the website, but I need to be in the UK or Ireland to subscribe otherwise I can't watch even free episodes. discoveryplus.com just redirects to discoveryplus.it, so it's more undiscoverable than the British version, the irony

LordMike commented 3 years ago

I can report that discoveryplus.dk has the same new layout - there are no episode-specific url's anymore. Downloading the m3u8 seems to produce the full episode in the end, audio and all (I'm logged in).

So it's workable.. But as mentioned, the extractor doesn't work - although for this show there aren't any subtitles or anything extra.. :P

nestukh commented 3 years ago

[a bit offtopic comment] In the meantime, in case you'd like to watch web content on a TV and you've got a DSL connection or faster, I'd like everybody to know that the discoveryplus plugin for Kodi works with the new website on Kodi 18 (current stable version). You will be able to bookmark favorites for your shows and access the list of streamable episodes like it was a directory on you PC. Unironically more user-friendly than the Discovery+ app for Android TV. Just download the zip and install it from the AddOn section. You'll need to use cookies exported from Chrome/Chromium to fix login issues, you can easly create a cookies.txt file by the Get cookies.txt extension.

Kodi is a free software and open source media center (different from Plex, a personal streaming server & client solution). For the cheapest and easiest alternative for novices, install it on your Windows or macOS computer, connect TV with an HDMI cable, set kodi to open on a second display (on Settings -> System -> Display -> Display mode), boom done. Most linuxes are surely on 18. Ubuntu has Kodi 18, RaspBerry Pi OS has 18, same for Arch and Manjaro. Linux Mint can install 19 via PPA. Debian Stable (Buster) has 17 (previous version) and it's a mess to install the current one, but the next stable (Bullseye, out most likely later this year) will have Kodi 19 (next stable). For more professional solutions, kodiboxes are a good investment. If you have a raspberry or a compatible ARM board you can use OSMC or LibreELEC (both free, modified linux OSes with Kodi for ARM devices). There are good communities behind. OSMC a Kodi 18 OS based on Debian Buster, installing apt packages and to use common linux tools is extra nice. LibreELEC is slightly more up to date (still on 18) but not based on Debian. They will update to 19 later in the current year. Eitherway, youtube_dl in a virtualenv with some related cronjobs is super useful on a kodibox. Especially when the current FREE episodes on Discovery+ will be marked as PAY PER VIEW. [end of the offtopic comment, please don't ban me]

Diegus83 commented 3 years ago

My problem is that there are no more https://www.discoveryplus.it/program/$program_name/$episode_name urls.

I wonder if maybe the are loading the episodes in a frame but that seems unlikely, I think frames are no longer popular in web design.

If you hover over the button or image that starts the episode is there any sort of link that you can copy? Maybe that can trigger the loading of the m3u8 and then it can be extracted. Just spitballing here.

nestukh commented 3 years ago

No, no hover links. I've checked: the playlist.m3u8 appears in the Network tab 2 seconds after I click on an episode. In my case, only on free episodes, no m3u8 file appears if I click on paid ones.

The aforementioned kodi plugin has some working extractors in python, but depends on the Inputstream Adaptive plugin for Kodi. For what I see most streamable websites on Kodi have plugins based on this library. Should youtube_dl adapt that to its needs and join forces with them is probably a major dev decision that I can't suggest because it's too big. CORRECTION: maybe it's used only for playing content from urls, but every plugin has to pass correct urls.

LordMike commented 3 years ago

My problem is that there are no more https://www.discoveryplus.it/program/$program_name/$episode_name urls.

I wonder if maybe the are loading the episodes in a frame but that seems unlikely, I think frames are no longer popular in web design.

No iframes. Checked for that.

ohmybahgosh commented 3 years ago

@remitamine I set the password back to the one you have from me if you need it for testing

remitamine commented 3 years ago

@ohmybahgosh is there a new problem with support for discoveryplus.com?

felisucoibi commented 3 years ago

does this work with www.discoveryplus.es?

Diegus83 commented 3 years ago

@ohmybahgosh is there a new problem with support for discoveryplus.com?

I updated to the latest release today and I'm experiencing the same problem of the episodes downloading with the pre roll ads incldued.

What I did is remove the previous version with brew and install the new one

❯ brew uninstall youtube-dl
❯ brew cleanup
❯ brew install youtube-dl
❯ youtube-dl --version
2021.02.22
❯ youtube-dl --cookies cookies.txt https://www.discoveryplus.com/video/mythbusters-mini-myths/champagne-rocket-bottle
[DiscoveryPlus] mythbusters-mini-myths/champagne-rocket-bottle: Downloading JSON metadata
[DiscoveryPlus] 885875: Downloading JSON metadata
[DiscoveryPlus] mythbusters-mini-myths/champagne-rocket-bottle: Downloading m3u8 information
[hlsnative] Downloading m3u8 manifest
WARNING: hlsnative has detected features it does not support, extraction will be delegated to ffmpeg

Looking at the last line in the log I think the problem is the same I had when testing the PR. I have pyenv installed in my system and I think that is causing youtube-dl not to run the appropriate version of python and not finding the required dependencies.

Is there a way to know o select which version of python youtube-dl is using?

I have 2.7 that comes with the OS, 3.9.2 was installed by brew when I installed youtube-dl and I'm globally using 3.9.1 via pyenv. I have tried installing pycrypto via /usr/local/bin/pip3.9 (the brew path) and via the pyenv version but still experiencing the same issue.

Edit: Looking at the verbose log it seems to be running the python install from brew. Should not all the required packages be available then?

[debug] youtube-dl version 2021.02.22
[debug] Python version 3.9.2 (CPython) - macOS-10.14.6-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.3.2, ffprobe 4.3.2, rtmpdump 2.4
remitamine commented 3 years ago

@Diegus83 try the latest change(44603290e5002153f3ebad6230cc73aef42cc2cd).

Diegus83 commented 3 years ago

@Diegus83 try the latest change(4460329).

Ok, I tried that commit a couple ways and both worked (no ads). I'm still convinced brew, pyenv and youtube-dl are not a good combo. I installed the 2021.02.22 using pip instead of brew and everything works as expected.

Running it from the repository:

python -m youtube_dl --cookies cookies.txt --verbose https://www.discoveryplus.com/video/teslas-death-ray-a-murder-declassified/a-suspect-in-serbia -f hls-3280
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--cookies', 'cookies.txt', '--verbose', 'https://www.discoveryplus.com/video/teslas-death-ray-a-murder-declassified/a-suspect-in-serbia', '-f', 'hls-3280']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.02.22
[debug] Python version 3.9.1 (CPython) - macOS-10.14.6-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.3.2, ffprobe 4.3.2, rtmpdump 2.4
[debug] Proxy map: {}
[debug] Using fake IP 6.36.72.236 (US) as X-Forwarded-For.
[DiscoveryPlus] teslas-death-ray-a-murder-declassified/a-suspect-in-serbia: Downloading JSON metadata
[DiscoveryPlus] 85067: Downloading JSON metadata
[DiscoveryPlus] teslas-death-ray-a-murder-declassified/a-suspect-in-serbia: Downloading m3u8 information
[debug] Invoking downloader on 'https://content-ause2-ur-discovery1.uplynk.com/bb854e04014e40709162e57c111831e9/h.m3u8?pbs=ca848934279e4f208366428455fc9bf8'
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 612

I also tried adding the modified dplay.py to the latest version available via brew in ./Cellar/youtube-dl/2021.2.22/libexec/lib/python3.9/site-packages/youtube_dl/extractor/

youtube-dl --verbose --cookies cookies.txt https://www.discoveryplus.com/video/teslas-death-ray-a-murder-declassified/search-for-the-lost-lab
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--verbose', '--cookies', 'cookies.txt', 'https://www.discoveryplus.com/video/teslas-death-ray-a-murder-declassified/search-for-the-lost-lab']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.02.22
[debug] Git HEAD: a7f44c39
[debug] Python version 3.9.1 (CPython) - macOS-10.14.6-x86_64-i386-64bit
[debug] exe versions: ffmpeg 4.3.2, ffprobe 4.3.2, rtmpdump 2.4
[debug] Proxy map: {}
[debug] Using fake IP 6.23.151.50 (US) as X-Forwarded-For.
[DiscoveryPlus] teslas-death-ray-a-murder-declassified/search-for-the-lost-lab: Downloading JSON metadata
[DiscoveryPlus] 85137: Downloading JSON metadata
[DiscoveryPlus] teslas-death-ray-a-murder-declassified/search-for-the-lost-lab: Downloading m3u8 information
[debug] Default format spec: bestvideo+bestaudio/best
[debug] Invoking downloader on 'https://content-ause2-ur-discovery1.uplynk.com/8bbe8a8eb20c468899590cc423960bb5/i.m3u8?pbs=a181c842cab34137ac0ae5c9c68a2bfc'
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 613 (not including 283 ad)
Tb3986 commented 3 years ago

python -m youtube_dl --cookies cookies.txt --verbose https://www.discoveryplus.com/video/teslas-death-ray-a-murder-declassified/a-suspect-in-serbia -f hls-3280

is there any workaround for the latest youtube-dl?

coopermor commented 3 years ago

The extractor works fine in the latest version. The video you are trying to download does not have a hls-3280 format. 

python -m youtube_dl --cookies cookies.txt --verbose https://www.discoveryplus.com/video/teslas-death-ray-a-murder-declassified/a-suspect-in-serbia -F
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--cookies', 'cookies.txt', '--verbose', 'https://www.discoveryplus.com/video/teslas-death-ray-a-murder-declassified/a-suspect-in-serbia', '-F']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] youtube-dl version 2021.03.14
[debug] Git HEAD: fa6bf0a71
[debug] Python version 3.8.2 (CPython) - Windows-10-10.0.19041-SP0
[debug] exe versions: ffmpeg 3.4
[debug] Proxy map: {}
[debug] Using fake IP 6.204.91.117 (US) as X-Forwarded-For.
[DiscoveryPlus] teslas-death-ray-a-murder-declassified/a-suspect-in-serbia: Downloading JSON metadata
[DiscoveryPlus] 85067: Downloading JSON metadata
[DiscoveryPlus] teslas-death-ray-a-murder-declassified/a-suspect-in-serbia: Downloading m3u8 information
[info] Available formats for 85067:
format code  extension  resolution note
hls-270      mp4        256x144     270k , avc1.4d000d, 15.0fps, mp4a.40.5
hls-472      mp4        416x234     472k , avc1.4d000d, 30.0fps, mp4a.40.5
hls-967      mp4        640x360     967k , avc1.4d001e, 30.0fps, mp4a.40.5
hls-2172     mp4        960x540    2172k , avc1.4d001f, 30.0fps, mp4a.40.5
hls-3426     mp4        1280x720   3426k , avc1.64001f, 30.0fps, mp4a.40.5
hls-5283     mp4        1920x1080  5283k , avc1.640028, 30.0fps, mp4a.40.5 (best)

Use one of the formats available and it works fine.

Tb3986 commented 3 years ago

The extractor works fine in the latest version. The video you are trying to download does not have a hls-3280 format.

python -m youtube_dl --cookies cookies.txt --verbose https://www.discoveryplus.com/video/teslas-death-ray-a-murder-declassified/a-suspect-in-serbia -F
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--cookies', 'cookies.txt', '--verbose', 'https://www.discoveryplus.com/video/teslas-death-ray-a-murder-declassified/a-suspect-in-serbia', '-F']
[debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252
[debug] youtube-dl version 2021.03.14
[debug] Git HEAD: fa6bf0a71
[debug] Python version 3.8.2 (CPython) - Windows-10-10.0.19041-SP0
[debug] exe versions: ffmpeg 3.4
[debug] Proxy map: {}
[debug] Using fake IP 6.204.91.117 (US) as X-Forwarded-For.
[DiscoveryPlus] teslas-death-ray-a-murder-declassified/a-suspect-in-serbia: Downloading JSON metadata
[DiscoveryPlus] 85067: Downloading JSON metadata
[DiscoveryPlus] teslas-death-ray-a-murder-declassified/a-suspect-in-serbia: Downloading m3u8 information
[info] Available formats for 85067:
format code  extension  resolution note
hls-270      mp4        256x144     270k , avc1.4d000d, 15.0fps, mp4a.40.5
hls-472      mp4        416x234     472k , avc1.4d000d, 30.0fps, mp4a.40.5
hls-967      mp4        640x360     967k , avc1.4d001e, 30.0fps, mp4a.40.5
hls-2172     mp4        960x540    2172k , avc1.4d001f, 30.0fps, mp4a.40.5
hls-3426     mp4        1280x720   3426k , avc1.64001f, 30.0fps, mp4a.40.5
hls-5283     mp4        1920x1080  5283k , avc1.640028, 30.0fps, mp4a.40.5 (best)

Use one of the formats available and it works fine.

mmm ok but how to get the exactly link for the some tv show? For example. I'm trying with https://www.discoveryplus.it/programmi/i-segreti-delle-strutture

and youtube-dl say this: youtube_dl.utils.UnsupportedError: Unsupported URL:

coopermor commented 3 years ago

The extractor does not work for discoveryplus.it, see above comments on how to workaround by getting the m3u8. The extractor works for discoveryplus.com.

Diegus83 commented 3 years ago

https://www.discoveryplus.it/programmi/i-segreti-delle-strutture

You are trying to pass an entire show, you need to pass the individual episodes.

Tb3986 commented 3 years ago

https://www.discoveryplus.it/programmi/i-segreti-delle-strutture

You are trying to pass an entire show, you need to pass the individual episodes.

Ok i did but unfortunately m3u8 has been decrypted:

https://dplaysouth-vod.akamaized.net/dplaydni/169249/0/hls/9844245004/exp=1616324137~acl=%2f*~data=hdntl~hmac=501c49fbd0564aa5589a72332 1bfdc9a5a27935bdff7de9691858a7d171c0a36/995756510-prog_index.m3u8?version_hash=38d5caf4: Server returned 403 Forbidden (access denied)

Diegus83 commented 3 years ago

Server returned 403 Forbidden (access denied)

That’s an authentication error and has nothing to do with encryption. Are you passing cookies to YouTube-dl? The user/pass authentication doesn’t work so you need to pass it a valid set of cookies.

Tb3986 commented 3 years ago

Server returned 403 Forbidden (access denied)

That’s an authentication error and has nothing to do with encryption. Are you passing cookies to YouTube-dl? The user/pass authentication doesn’t work so you need to pass it a valid set of cookies.

yes with cookie.txt. I exported it using Get cookies.txt extension from chrome.

Diegus83 commented 3 years ago

yes with cookie.txt. I exported it using Get cookies.txt extension from chrome.

Can you post the complete command you are using and the output from youtube-dl?

Tb3986 commented 3 years ago

yes with cookie.txt. I exported it using Get cookies.txt extension from chrome.

Can you post the complete command you are using and the output from youtube-dl?

python -m youtube_dl --cookies cookies.txt --verbose https://dplaysouth-vod.akamaized.net/dplaydni/169249/0/hls/9844245004/exp=1616324137~acl=%2f*~data=hdntl~hmac=501c49fbd0564aa5589a723321bfdc9a5a27935bdff7de9691858a7d171c0a36/995756510-prog_index.m3u8?version_hash=38d5caf4 -F

Diegus83 commented 3 years ago

First thing I would try is log out and back in of Discovery in Google Chrome, export a fresh set of cookies and try again.

Edit: I just checked because I could not remember for sure and once you have the m3u8 URL for the CDN you don't need cookies anymore, at least on US site.

Have you tried any of the other m3u8 URLs that show up in the Network tab? Your URL looks different from the ones I have encountered before.

ghost commented 3 years ago

This appears to have stopped working recently. Now when I try to download from discoveryplus.com, I'm getting the error SAMPLE-AES encryption is not supported yet. Anyone else?

coopermor commented 3 years ago

Maybe it depends on the show? I just tried again and it worked fine.

[debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: [u'--verbose', u'--cookies', u'/Users/user/Downloads/cookies.txt', u'-f', u'bestvideo[height<=720]+bestaudio/best[height<=720]', u'https://www.discoveryplus.com/video/90-day-the-single-life/pt-down-and-dirty', u'-o', u'Pillow.Talk.S06E05.%(ext)s'] [debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8 [debug] youtube-dl version 2021.02.10 [debug] Git HEAD: 9fc5eafb8 [debug] Python version 2.7.16 (CPython) - Darwin-20.3.0-x86_64-i386-64bit [debug] exe versions: ffmpeg N-96942-g1812352d76-tessus [debug] Proxy map: {} [debug] Using fake IP 6.20.63.102 (US) as X-Forwarded-For. [DiscoveryPlus] 90-day-the-single-life/pt-down-and-dirty: Downloading JSON metadata [DiscoveryPlus] 1421665: Downloading JSON metadata [DiscoveryPlus] 90-day-the-single-life/pt-down-and-dirty: Downloading m3u8 information [debug] Invoking downloader on u'https://content-ause2-ur-discovery1.uplynk.com/a91e1dcde3aa4bfe91c432bacb50644b/e.m3u8?cid=a91e1dcde3aa4bfe91c432bacb50644b&ct=a&exp=1623434231&hlsver=7&rays=cdefghiba&rn=12356851209106638253&tc=1&sig=123cdfd052b18b2195c973f5113f07d05dc233c6fb32a0bdd22284681873a886&pbs=c0a4d68669364b4693d0acae24f7361e' [hlsnative] Downloading m3u8 manifest [hlsnative] Total fragments: 568 [download] Destination: Pillow.Talk.S06E05.mp4 [download] 0.2% of ~950.25MiB at 573.92KiB/s ETA 15:43

Diegus83 commented 3 years ago

I won't be surprised if newer shows are using encryption to make them harder to rip.

ghost commented 3 years ago

It's working now. I first tried the video page URL https://www.discoveryplus.com/video/anthony-bourdain-no-reservations/iceland which gave me an "invalid URL" error. So, I followed bits of advice from this thread and tried using the m3u8 URL instead. That's when I was getting the SAMPLE-AES encryption is not supported yet error.

I tried your example and it worked, so decided to try again with my video page URL and it is working now! I have no explanation.

youtube-dl --cookies cookies.txt https://www.discoveryplus.com/video/anthony-bourdain-no-reservations/iceland

Sorry for the non-issue issue! My Discovery+ account is brand new so perhaps that had something to do with it.

jack-sparrow1 commented 2 years ago

Hey Guys, I was trying to download videos from discoveryplus and it works well when the only audio is in English. Incase the show is available in multiple languages the downloaded file is more of a hit or miss. It might either download in English or might pickup any other language. In the region that I'm located in, I get only 4 .m3u8 files, and I download the one titled master.m3u8.

The youtube-dl query that I'm using is youtube-dl link_to_master.m3u8.

Would you suggest any other query to download the video in the preferred language. Below is the response that I get from the master.m3u8 file (just included the audio part).

#EXTM3U
#EXT-X-INDEPENDENT-SEGMENTS

#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="stereo_128000",NAME="Hindi",LANGUAGE="hi",AUTOSELECT=YES,DEFAULT=YES,URI="audio-hi-a57a7b6e-e20c-4d74-9ff4-b12a5a5c08ca/video-128000bps.m3u8"
#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="stereo_128000",NAME="English",LANGUAGE="en",AUTOSELECT=YES,URI="audio-en-989bf02e-15aa-4916-abd2-273c60b91bb1/video-128000bps.m3u8"

#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subtitle_group",NAME="English",LANGUAGE="en",AUTOSELECT=YES,CHARACTERISTICS="public.accessibility.describes-music-and-sound",URI="caption-en/captions.m3u8"

Any help would be greatly appreciated. Thanks.

jack-sparrow1 commented 2 years ago

Sure, no issues. I'll look around for a solution. If something works, I'll post it here.

sandersrc commented 2 years ago

Please Help. Looks like this is broken again. I can not find the .m3u8 information anymore.

And the command: youtube-dl --cookies cookies.txt --verbose https://www.discoveryplus.com/video/mythbusters/exploding-toilet

Returns: [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: ['--cookies', 'cookies.txt', '--verbose', 'https://www.discoveryplus.com/video/mythbusters/exploding-toilet'] [debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252 [debug] youtube-dl version 2021.12.17 [debug] Python version 3.4.4 (CPython) - Windows-10-10.0.19041 [debug] exe versions: none [debug] Proxy map: {} [debug] Using fake IP 6.9.143.11 (US) as X-Forwarded-For. [DiscoveryPlus] mythbusters/exploding-toilet: Downloading JSON metadata ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\extractor\common.py", line 634, in _request_webpage File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmpupik7c6w\build\youtube_dl\YoutubeDL.py", line 2288, in urlopen File "C:\Python\Python34\lib\urllib\request.py", line 470, in open File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response File "C:\Python\Python34\lib\urllib\request.py", line 508, in error File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

splyfof commented 1 year ago

Any updates for the latest discoveryplus thingy, also i think they have moved the country sites under the .com as discoveryplus.com/dk or it or whatever. Is it confirmed that the new shows have encryption, that would be sad if true.

dirkf commented 1 year ago

I encourage any interested subscribers to this issue to review and test PR #32235 where we are looking for valid URLs for DiscoveryPlus (and related sites).