yt-dlp / yt-dlp

A feature-rich command-line audio/video downloader
https://discord.gg/H5MNcFW63r
The Unlicense
87.06k stars 6.78k forks source link

Rtve.es Play #1346

Open Jiriss opened 3 years ago

Jiriss commented 3 years ago

The list of the supported sites say that Rtve Alacarta has supported. This site has replaced with Rtve Play and extractor doesn't work with new site. Please could you add support for Rtve Play? This is example to video https://www.rtve.es/play/videos/comando-actualidad/

nggfng commented 3 years ago

Someone suggested replacing /play/ for /alacarta/ in the URL. After that, I'm getting a "Payment Required" error with this video: https://www.rtve.es/alacarta/videos/la-caza/caza-monteperdido-capitulo-1-deshielo/5092259/

Jiriss commented 3 years ago

Wow, It works if video is free and it hasn't any restrictions as paywall or geo-blocking. For instance, this video works https://www.rtve.es/play/videos/lazos-de-sangre/t3-concha-piquer/5655688/ when replacing play for alacarta Yt-dlp will output a destination url with http:// prefix but it also works with https:// one.

hansworzt commented 3 years ago

Any video on RTVE Play - including the geo-restricted and paywalled ones - can be transformed into a freely accessible URL and downloaded with "yt-dlp", after some additional steps:

1) Example program URL: https://www.rtve.es/play/videos/somos-cine/somos-cine-altamira/5621023/ Copy this to https://www.piraminetlab.com/ to generate the video URL. Resulting video URL: https://rtve-mediavod.secure.footprint.net/resources/TE_GLESP/mp4/7/4/1594294885247.mp4?nvb=20211030172651&nva=20211030232651&token=047dbe539d7132ff9bc98 The video URL can sometimes also be found in the HAR file saved from the network tab in Firefox.

2) This geo-restricted video only plays from Spanish IP addresses. Transform into the geo-unrestricted URL: https://rtve-hlsvod.secure.footprint.net/resources/TE_NGVA/mp4/7/4/1594294885247.mp4/playlist.m3u8?idasset=5621023

3) yt-dlp -F ... shows the available format(s), etc.

No doubt some Python wizzard could implement this into an extractor.

hansworzt commented 3 years ago

Someone suggested replacing /play/ for /alacarta/ in the URL. After that, I'm getting a "Payment Required" error with this video: https://www.rtve.es/alacarta/videos/la-caza/caza-monteperdido-capitulo-1-deshielo/5092259/

This transformed URL works just fine: https://rtve-hlsvod.secure.footprint.net/resources/TE_NGVA/mp4/8/9/1553557872798.mp4/playlist.m3u8?idasset=5092259

Jiriss commented 2 years ago

When will the yt-dlp project be added this change to the production?

Jiriss commented 2 years ago

It isn't work anymore with the latest version 2022.03.08.1 although using workground replacing /play/ for /alacarta/ in the URL. Output listed below

$ yt-dlp --version 2022.03.08.1

$ yt-dlp -F https://www.rtve.es/alacarta/videos/telediario-2/telediario-21-horas-10-03-22/6431098/ Traceback (most recent call last): File "/usr/local/bin/yt-dlp/yt_dlp/extractor/common.py", line 615, in extract File "/usr/local/bin/yt-dlp/yt_dlp/extractor/common.py", line 523, in initialize File "/usr/local/bin/yt-dlp/yt_dlp/extractor/rtve.py", line 73, in _real_initialize KeyError: 'User-Agent'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 1389, in wrapper File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 1459, in __extract_info File "/usr/local/bin/yt-dlp/yt_dlp/extractor/common.py", line 647, in extract yt_dlp.utils.ExtractorError: 6431098: An extractor error has occurred. (caused by KeyError('User-Agent')); please report this issue on https://github.com/yt-dlp/yt-dlp , filling out the "Broken site" issue template properly. Confirm you are on the latest version using yt-dlp -U

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main return _run_code(code, main_globals, None, File "/usr/lib/python3.8/runpy.py", line 87, in _run_code exec(code, run_globals) File "/usr/local/bin/yt-dlp/main.py", line 19, in File "/usr/local/bin/yt-dlp/yt_dlp/init.py", line 864, in main File "/usr/local/bin/yt-dlp/yt_dlp/init.py", line 854, in _real_main File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 3254, in download File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 3227, in wrapper File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 1380, in extract_info File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 1407, in wrapper File "/usr/local/bin/yt-dlp/yt_dlp/utils.py", line 1088, in format_traceback TypeError: format_exception() missing 2 required positional arguments: 'value' and 'tb'

hansworzt commented 2 years ago

They changed it so new streams can no longer be downloaded. The https://www.piraminetlab.com/ site that used to offer the URLs of RTVE programmes is thus basically dead. To alleviate this never ending cat-and-mouse game, use software that can record played streams. I use TubeDigger (payware) for this. No matter what they change from now on, if it plays in a web browser it will be recorded losslessly with this software.

pukkandan commented 2 years ago

https://github.com/ytdl-org/youtube-dl/pull/29816

dirkf commented 2 years ago

... This is example to video https://www.rtve.es/play/videos/comando-actualidad/

But the actual episode that you can play from that page: https://www.rtve.es/play/videos/comando-actualidad/abandonados-bancos/6459510/

The corresponding series page: https://www.rtve.es/play/videos/modulos/capitulos/11332/?currentpage=pf_serie, from which we need to harvest these elements: <a class="goto_media" href="https://www.rtve.es/play/videos/comando-actualidad/abandonados-bancos/6459510/" title="Empezar a ver">.

alexanderadam commented 2 years ago

Apart from the play vs alacarta issue, it's also not possible to fetch/embed the subtitles (it seems that there was once another Python script for this). Is this related or should I create a new issue for this?

mildsunrise commented 2 years ago

I've tried all links to videos mentioned in this issue (except https://www.rtve.es/play/videos/comando-actualidad/, which as pointed by @dirkf is NOT the link to a video, but to a show's landing page), and I can confirm they all work with https://github.com/ytdl-org/youtube-dl/pull/31241 applied (from a spanish IP). They work regardless of play or alacarta in the URL.

Jiriss commented 2 years ago

Support for rtve.es is still unavailable

$ yt-dlp --version 2022.10.04

Tested with European IP address but outside of Spain.

$ yt-dlp -v -F https://www.rtve.es/play/videos/telediario-2/21-horas-04-10-22/6706793/ [debug] Command-line config: ['-v', '-F', 'https://www.rtve.es/play/videos/telediario-2/21-horas-04-10-22/6706793/'] [debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8 [debug] yt-dlp version 2022.10.04 [4e0511f] (zip) [debug] Python 3.8.10 (CPython 64bit) - Linux-5.4.0-126-generic-x86_64-with-glibc2.29 (glibc 2.31) [debug] Checking exe version: ffmpeg -bsfs [debug] Checking exe version: ffprobe -bsfs [debug] exe versions: ffmpeg 4.2.7, ffprobe 4.2.7, rtmpdump 2.4 [debug] Optional libraries: certifi-2019.11.28, pycrypto-2.6.1, pyxattr-0.6.1, secretstorage-2.3.1, sqlite3-2.6.0 [debug] Proxy map: {} [debug] Loaded 1690 extractors [debug] [generic] Extracting URL: https://www.rtve.es/play/videos/telediario-2/21-horas-04-10-22/6706793/ [generic] 6706793: Downloading webpage WARNING: [generic] Falling back on generic information extractor [generic] 6706793: Extracting information [debug] Looking for Brightcove embeds [debug] Looking for embeds ERROR: Unsupported URL: https://www.rtve.es/play/videos/telediario-2/21-horas-04-10-22/6706793/ Traceback (most recent call last): File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 1477, in wrapper return func(self, *args, **kwargs) File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 1553, in __extract_info ie_result = ie.extract(url) File "/usr/local/bin/yt-dlp/yt_dlp/extractor/common.py", line 672, in extract ie_result = self._real_extract(url) File "/usr/local/bin/yt-dlp/yt_dlp/extractor/generic.py", line 3062, in _real_extract raise UnsupportedError(url) yt_dlp.utils.UnsupportedError: Unsupported URL: https://www.rtve.es/play/videos/telediario-2/21-horas-04-10-22/6706793/

Situation is same although replaced /play/ with /alacarta/ in the URL.

$ yt-dlp -v -F https://www.rtve.es/alacarta/videos/telediario-2/21-horas-04-10-22/6706793/ [debug] Command-line config: ['-v', '-F', 'https://www.rtve.es/alacarta/videos/telediario-2/21-horas-04-10-22/6706793/'] [debug] Encodings: locale UTF-8, fs utf-8, pref UTF-8, out utf-8, error utf-8, screen utf-8 [debug] yt-dlp version 2022.10.04 [4e0511f] (zip) [debug] Python 3.8.10 (CPython 64bit) - Linux-5.4.0-126-generic-x86_64-with-glibc2.29 (glibc 2.31) [debug] Checking exe version: ffmpeg -bsfs [debug] Checking exe version: ffprobe -bsfs [debug] exe versions: ffmpeg 4.2.7, ffprobe 4.2.7, rtmpdump 2.4 [debug] Optional libraries: certifi-2019.11.28, pycrypto-2.6.1, pyxattr-0.6.1, secretstorage-2.3.1, sqlite3-2.6.0 [debug] Proxy map: {} [debug] Loaded 1690 extractors [rtve.es:alacarta] Fetching manager info [debug] [rtve.es:alacarta] Extracting URL: https://www.rtve.es/alacarta/videos/telediario-2/21-horas-04-10-22/6706793/ [rtve.es:alacarta] 6706793: Downloading JSON metadata [rtve.es:alacarta] 6706793: Downloading url information ERROR: not enough values to unpack (expected 2, got 1) Traceback (most recent call last): File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 1477, in wrapper return func(self, *args, **kwargs) File "/usr/local/bin/yt-dlp/yt_dlp/YoutubeDL.py", line 1553, in __extract_info ie_result = ie.extract(url) File "/usr/local/bin/yt-dlp/yt_dlp/extractor/common.py", line 672, in extract ie_result = self._real_extract(url) File "/usr/local/bin/yt-dlp/yt_dlp/extractor/rtve.py", line 144, in _real_extract formats = self._extract_png_formats(video_id) File "/usr/local/bin/yt-dlp/yt_dlp/extractor/rtve.py", line 118, in _extract_png_formats for quality, video_url in self._decrypt_url(png): File "/usr/local/bin/yt-dlp/yt_dlp/extractor/rtve.py", line 81, in _decrypt_url quality, url_data = text.split(b'%%') ValueError: not enough values to unpack (expected 2, got 1)

dincel commented 1 year ago

Manual way of retrieving direct content links from RTVE.es/play portal

sample URL: https://www.rtve.es/play/videos/carlos-rey-emperador/carlos-rey-emperador-capitulo-1/3312421/

  1. While playing the media, goto devtools (F12) > Network tab and search for ".m3u8" (refresh with F5 if necessary) there will be two results: https://rtvehlsvodlote7.rtve.es/mediavodv2/resources/TE_SCARLOS/mp4/5/9/1641318615695.mp4/video.m3u8?hls_no_audio_only=true&idasset=3312421 https://rtvehlsvodlote7.rtve.es/mediavodv2/resources/TE_SCARLOS/mp4/5/9/1641318615695.mp4/1641318615695-audio=192975-video=2759000.m3u8?idasset=3312421
  2. take either one and remove the part after .mp4
  3. change the URL's "https://rtvehlsvodlote7.rtve.es/mediavodv2/" part to "https://mediavod-lvlt.rtve.es/" (domain at the download links for radio RNE content) resulting the final working URL: https://mediavod-lvlt.rtve.es/resources/TE_SCARLOS/mp4/5/9/1641318615695.mp4
  4. Now the subtitles. As in step (1) this time search for ".vtt" and directly copy the link "https://www.rtve.es/resources/vtt/0/3/1648225256930.vtt" (you don't have to download it) add both links to your in your text editor and save as .m3u playlist (e.g. play.m3u) . Like this; ... https://mediavod-lvlt.rtve.es/resources/TE_SCARLOS/mp4/5/9/1641318615695.mp4 https://www.rtve.es/resources/vtt/0/3/1648225256930.vtt ...
  5. When you open Celluloid player it only plays .mp4 file. And you can copy the .vtt link address from the playlist items (F9) and open it wiht Ctrl + L shortcut. This will activate while playing, subtitles too.

This seems a bit player specific and but at least I found a way to retrieve both video and subs. But manually it takes too long. It is up to you programmers to apply this workaround to yt-dlp API and allow us to watch shows directly from the visible URLs like https://www.rtve.es/play/videos/carlos-rey-emperador/carlos-rey-emperador-capitulo-1/3312421/

hansworzt commented 1 year ago

There's a simpler manual method that maybe automated:

1) Copy the page URL: https://www.rtve.es/play/videos/carlos-rey-emperador/carlos-rey-emperador-capitulo-1/3312421/

2) Put this into https://www.descargavideos.tv/ and hit the search button.

3) Hit "Descargar (opción 1, supuestamente sin geobloqueo". This displays the video and shows the direct URL : https://mediavod-lvlt.rtve.es/resources/TE_SCARLOS/mp4/5/9/1641318615695.mp4 A shorter URL that doesn't bother with certificates is: http://www.rtve.es/resources/TE_SCARLOS/mp4/5/9/1641318615695.mp4 This can be downloaded with: yt-dlp -f mp4 http://www.rtve.es/resources/TE_SCARLOS/mp4/5/9/1641318615695.mp4 Or with: wget http://www.rtve.es/resources/TE_SCARLOS/mp4/5/9/1641318615695.mp4 There's another button on the "Descargar" page that gives you the direct subtitles link.