Open sebastienadam opened 2 years ago
Although the log is not exactly the same, see #30878: in particular https://github.com/ytdl-org/youtube-dl/issues/30878#issuecomment-1141949627.
@Zombo schrieb am 05.06.22 um 19:08 Uhr: Do they have any English stuff? I would be willing to look at this, but it needs to be something I might actually watch
Sure they do, so much wonderful stuff: https://www.arte.tv/en/
They even have full concerts, e.g. Rolling stones β A bigger bang https://www.arte.tv/de/videos/105608-000-A/rolling-stones-a-bigger-bang/
I'd really apprechiate you looking into this. :D
Greetz and thank you laryllian
I've found a very easy workaround for this arte.tv problem with their API. No need to do any updates, download new PY files, or switch to a different code/app. After hours and hours of googling, I found a casual remark that just said, "API 2 simply means that you need to find the m3u8 URL."
So basically, all one has to do is google how to find find the URL for a blob video in your specific browser (the routines are a little different in Firefox and Chrome, respectively). Once you've found it, you only need to fetch the URL for the m3u8 file and paste it into youtube-dl, done.
on firefox, open Developer Tools, go to Media tab, search for mp4 file, the copy the url
... Backporting from current yt-dlp
might be necessary to restore ARTE.tv downloads in youtube-dl
;
sample URL available (and working) at the time of writing:
https://www.arte.tv/en/videos/067103-000-A/mariupol/
yt-dlp -v -F "https://www.arte.tv/en/videos/067103-000-A/mariupol/" --ie ArteTV =>
[debug] Command-line config: ['--ffmpeg-location', '..', '--downloader-args', 'ffmpeg:-v 8 -stats', '-v', '-F', 'https://www.arte.tv/en/videos/067103-000-A/mariupol/']
[debug] Encodings: locale cp1253, fs utf-8, pref cp1253, out utf-8 (No VT), error utf-8 (No VT), screen utf-8 (No VT)
[debug] ytdl-patched/yt-dlp version 2022.09.19.40298 [bd21f4d] (win32_exe)
[debug] ** This build is unofficial daily builds, provided for ease of use.
[debug] ** Please do not ask for any support.
[debug] Python 3.7.9 (CPython 32bit) - Windows-Vista-6.0.6003-SP2
[debug] Checking exe version: "..\ffmpeg" -bsfs
[debug] Checking exe version: "..\ffprobe" -bsfs
[debug] exe versions: ffmpeg 4.4.1 (setts), ffprobe 4.4.1, phantomjs 2.1.1
[debug] Optional libraries: Cryptodome-3.15.0, brotli-1.0.9, certifi-2022.09.14, mutagen-1.45.1, sqlite3-2.6.0, websockets-10.3
[debug] Proxy map: {}
[debug] Loaded 1 extractors
[debug] [ArteTV] Extracting URL: https://www.arte.tv/en/videos/067103-000-A/mariupol/
[ArteTV] 067103-000-A: Downloading JSON metadata
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7),vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[info] Available formats for 067103-000-A:
ID EXT RESOLUTION FPS | FILESIZE TBR PROTO | VCODEC VBR ACODEC MORE INFO
----------------------------------------------------------------------------------------------------------------------------------------------
VO-STA-program_audio_0-VO mp4 audio only | m3u8 | audio only unknown [ru] German (Subtitles) [subDE]
VO-STE_ESP_-program_audio_0-VO mp4 audio only | m3u8 | audio only unknown [ru] Spanish (Subtitles) [OGsub-ESP]
VO-STE_ITA_-program_audio_0-VO mp4 audio only | m3u8 | audio only unknown [ru] Italian (Subtitles) [OGsub-ITA]
VO-STE_POL_-program_audio_0-VO mp4 audio only | m3u8 | audio only unknown [ru] Polish (Subtitles) [OGsub-POL]
VO-STF-program_audio_0-VO mp4 audio only | m3u8 | audio only unknown [ru] French (Subtitles) [subFR]
VO-STE_ANG_-program_audio_0-VO mp4 audio only | m3u8 | audio only unknown [ru] English (Subtitles) [OGsub-ANG]
VO-STA-426 mp4 384x216 25 | ~248.75MiB 426k m3u8 | avc1.42e00d 426k video only German (Subtitles) [subDE]
VO-STE_ESP_-426 mp4 384x216 25 | ~248.75MiB 426k m3u8 | avc1.42e00d 426k video only Spanish (Subtitles) [OGsub-ESP]
VO-STE_ITA_-426 mp4 384x216 25 | ~248.75MiB 426k m3u8 | avc1.42e00d 426k video only Italian (Subtitles) [OGsub-ITA]
VO-STE_POL_-426 mp4 384x216 25 | ~248.75MiB 426k m3u8 | avc1.42e00d 426k video only Polish (Subtitles) [OGsub-POL]
VO-STF-426 mp4 384x216 25 | ~248.75MiB 426k m3u8 | avc1.42e00d 426k video only French (Subtitles) [subFR]
VO-STA-927 mp4 640x360 25 | ~541.19MiB 927k m3u8 | avc1.4d401e 927k video only German (Subtitles) [subDE]
VO-STE_ESP_-927 mp4 640x360 25 | ~541.19MiB 927k m3u8 | avc1.4d401e 927k video only Spanish (Subtitles) [OGsub-ESP]
VO-STE_ITA_-927 mp4 640x360 25 | ~541.19MiB 927k m3u8 | avc1.4d401e 927k video only Italian (Subtitles) [OGsub-ITA]
VO-STE_POL_-927 mp4 640x360 25 | ~541.19MiB 927k m3u8 | avc1.4d401e 927k video only Polish (Subtitles) [OGsub-POL]
VO-STF-927 mp4 640x360 25 | ~541.19MiB 927k m3u8 | avc1.4d401e 927k video only French (Subtitles) [subFR]
VO-STA-1617 mp4 768x432 25 | ~943.97MiB 1617k m3u8 | avc1.4d401e 1617k video only German (Subtitles) [subDE]
VO-STE_ESP_-1617 mp4 768x432 25 | ~943.97MiB 1617k m3u8 | avc1.4d401e 1617k video only Spanish (Subtitles) [OGsub-ESP]
VO-STE_ITA_-1617 mp4 768x432 25 | ~943.97MiB 1617k m3u8 | avc1.4d401e 1617k video only Italian (Subtitles) [OGsub-ITA]
VO-STE_POL_-1617 mp4 768x432 25 | ~943.97MiB 1617k m3u8 | avc1.4d401e 1617k video only Polish (Subtitles) [OGsub-POL]
VO-STF-1617 mp4 768x432 25 | ~943.97MiB 1617k m3u8 | avc1.4d401e 1617k video only French (Subtitles) [subFR]
VO-STA-2313 mp4 1280x720 25 | ~ 1.32GiB 2313k m3u8 | avc1.4d401f 2313k video only German (Subtitles) [subDE]
VO-STE_ESP_-2313 mp4 1280x720 25 | ~ 1.32GiB 2313k m3u8 | avc1.4d401f 2313k video only Spanish (Subtitles) [OGsub-ESP]
VO-STE_ITA_-2313 mp4 1280x720 25 | ~ 1.32GiB 2313k m3u8 | avc1.4d401f 2313k video only Italian (Subtitles) [OGsub-ITA]
VO-STE_POL_-2313 mp4 1280x720 25 | ~ 1.32GiB 2313k m3u8 | avc1.4d401f 2313k video only Polish (Subtitles) [OGsub-POL]
VO-STF-2313 mp4 1280x720 25 | ~ 1.32GiB 2313k m3u8 | avc1.4d401f 2313k video only French (Subtitles) [subFR]
VO-STE_ANG_-426 mp4 384x216 25 | ~248.75MiB 426k m3u8 | avc1.42e00d 426k video only English (Subtitles) [OGsub-ANG]
VO-STE_ANG_-927 mp4 640x360 25 | ~541.19MiB 927k m3u8 | avc1.4d401e 927k video only English (Subtitles) [OGsub-ANG]
VO-STE_ANG_-1617 mp4 768x432 25 | ~943.97MiB 1617k m3u8 | avc1.4d401e 1617k video only English (Subtitles) [OGsub-ANG]
VO-STE_ANG_-2313 mp4 1280x720 25 | ~ 1.32GiB 2313k m3u8 | avc1.4d401f 2313k video only English (Subtitles) [OGsub-ANG]
I just figured out that https://github.com/yt-dlp/yt-dlp appears to work, just in case somebody needs a solution urgently. I hope, though, that the support can be added to youtube-dl proper.
I just figured out that https://github.com/yt-dlp/yt-dlp appears to work
... The sample ARTE.tv
link I used back in Sep of last year now produces different results with latest yt-dlp
:
[debug] Command-line config: ['--ffmpeg-location', '..\\FFmpeg', '--downloader-args', 'ffmpeg:-v 8 -stats', '-v', '-F', 'https://www.arte.tv/en/videos/067103-000-A/mariupol/', '--ie', 'ArteTV']
[debug] Encodings: locale cp1253, fs utf-8, pref cp1253, out utf-8 (No VT), error utf-8 (No VT), screen utf-8 (No VT)
[debug] yt-dlp version 2023.02.14 [8b37c58f8] (win_x86_exe)
[debug] Python 3.8.16 (CPython x86 32bit) - Windows-Vista-6.0.6003-SP2 (OpenSSL 1.1.1s 1 Nov 2022)
[debug] exe versions: ffmpeg n5.2-dev-2245-N-109649-gab8cde6 (setts), ffprobe n5.2-dev-2245-N-109649-gab8cde6, phantomjs 2.1.1, rtmpdump 2.4
[debug] Optional libraries: Cryptodome-3.17, brotli-1.0.9, certifi-2022.12.07, mutagen-1.46.0, sqlite3-2.6.0, websockets-10.4
[debug] Proxy map: {}
[debug] Extractor Plugins: AGB+NSIG (YoutubeIE)
[debug] Plugin directories: ['<redacted>\\yt-dlp-plugins\\YTAgeGateBypass.zip\\yt_dlp_plugins', '<redacted>\\yt-dlp-plugins\\YTNSigProxy.zip\\yt_dlp_plugins']
[debug] Loaded 1 extractors
[ArteTV] Extracting URL: https://www.arte.tv/en/videos/067103-000-A/mariupol/
[ArteTV] 067103-000-A: Downloading JSON metadata
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), channels, acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id
[info] Available formats for 067103-000-A:
ID EXT RESOLUTION FPS | FILESIZE TBR PROTO | VCODEC VBR ACODEC MORE INFO
------------------------------------------------------------------------------------------------------------------------------------
VO-STF-program_audio_0-VO mp4 audio only | m3u8 | audio only unknown [ru] French (Subtitles) [subFR]
VO-STF-426 mp4 384x216 25 | ~248.75MiB 426k m3u8 | avc1.42e00d 426k video only French (Subtitles) [subFR]
VO-STF-927 mp4 640x360 25 | ~541.19MiB 927k m3u8 | avc1.4d401e 927k video only French (Subtitles) [subFR]
VO-STF-1617 mp4 768x432 25 | ~943.97MiB 1617k m3u8 | avc1.4d401e 1617k video only French (Subtitles) [subFR]
VO-STF-2313 mp4 1280x720 25 | ~ 1.32GiB 2313k m3u8 | avc1.4d401f 2313k video only French (Subtitles) [subFR]
... And a "word" of caution:
yt-dlp "https://www.arte.tv/en/videos/067103-000-A/mariupol/" --list-subs =>
[ArteTV] Extracting URL: https://www.arte.tv/en/videos/067103-000-A/mariupol/
[ArteTV] 067103-000-A: Downloading JSON metadata
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[info] Available subtitles for 067103-000-A:
Language Formats
en vtt
fr vtt
de vtt
es vtt
pl vtt
it vtt
If you prefer your subs (like myself :wink:) in the old but trusted .srt
format,
yt-dlp "https://www.arte.tv/en/videos/067103-000-A/mariupol/" --skip-download --write-subs --sub-langs en --convert-subs srt =>
[ArteTV] Extracting URL: https://www.arte.tv/en/videos/067103-000-A/mariupol/
[ArteTV] 067103-000-A: Downloading JSON metadata
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[info] 067103-000-A: Downloading subtitles: en
[info] 067103-000-A: Downloading 1 format(s): VO-STF-2313+VO-STF-program_audio_0-VO
[info] Writing video subtitles to: Mariupol [067103-000-A].en.vtt
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 1
[download] Destination: Mariupol [067103-000-A].en.vtt
[download] 100% of 39.89KiB in 00:00:02 at 19.65KiB/s
[SubtitlesConvertor] Converting subtitles
Deleting original file Mariupol [067103-000-A].en.vtt (pass -k to keep)
... will leave you with an empty Mariupol [067103-000-A].en.srt
file π π π ...
This is because the specific webvtt
subs format ARTE serve is not palatable to FFmpeg, used for the conversion vtt => srt
...
Simply fetch the original vtt
subs:
yt-dlp "https://www.arte.tv/en/videos/067103-000-A/mariupol/" --skip-download --write-subs --sub-langs en
and use another tool to convert them to .srt
(I found fairly recent :wink: releases of SubtitleEdit to work OK for the task π ) ...
As the PR for the yt-dlp extractor has been merged, the only reason not to update the yt-dl extractor is that yt-dl doesn't have the subtitle extraction API used by yt-dlp, but I see that is problematic, pending a fix for the format conversion. So:
$ python -m youtube_dl -v -F 'https://www.arte.tv/en/videos/067103-000-A/mariupol/'
[debug] System config: [u'--prefer-ffmpeg']
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'-F', u'https://www.arte.tv/en/videos/067103-000-A/mariupol/']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Git HEAD: dd9aa74be
[debug] Python version 2.7.18 (CPython) - Linux-4.4.0-210-generic-i686-with-Ubuntu-16.04-xenial
[debug] exe versions: avconv 4.3, avprobe 4.3, ffmpeg 4.3, ffprobe 4.3
[debug] Proxy map: {}
[ArteTV] 067103-000-A: Downloading JSON metadata
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[info] Available formats for 067103-000-A:
format code extension resolution note
VO-STE_ANG_-program_audio_0-VO mp4 audio only [ru] English (Subtitles) [?]
VO-STE_ANG_-1617 mp4 768x432 English (Subtitles) [?] 1617k , avc1.4d401e, 25.0fps, video only
VO-STE_ANG_-2313 mp4 1280x720 English (Subtitles) [?] 2313k , avc1.4d401f, 25.0fps, video only
VO-STE_ANG_-927 mp4 640x360 English (Subtitles) [?] 927k , avc1.4d401e, 25.0fps, video only
VO-STE_ANG_-426 mp4 384x216 English (Subtitles) [?] 426k , avc1.42e00d, 25.0fps, video only (best)
$
If anyone has a replacement test video for ArteTVEmbed
, please submit it. It might be like this one that is no longer valid:
https://www.arte.tv/player/v5/index.php?json_url=https%3A%2F%2Fapi.arte.tv%2Fapi%2Fplayer%2Fv2%2Fconfig%2Fde%2F100605-013-A&lang=de&autoplay=true&mute=0100605-013-A
G doesn't admit to having indexed any pages with URL https://www.arte.tv/player/v5/...
.
Otherwise this extractor will be marked as not _WORKING
.
And see updated PR #29653.
The following video https://www.arte.tv/de/videos/060738-167-A/blow-up-rainer-werner-fassbinder-in-bildern/ (ID: 060738-167-A) is claimed to be available until 01.08.2114 (though it might change without notice, which renders such availability declarations ad absurdum).
... Apparently, a "fix" for (default) format sorting (according to increasing resolution) will also be needed:
(redacted for brevity)
[info] Available formats for 067103-000-A:
format code extension resolution note
VO-STE_ANG_-program_audio_0-VO mp4 audio only [ru] English (Subtitles) [?]
VO-STE_ANG_-1617 mp4 768x432 English (Subtitles) [?] 1617k , avc1.4d401e, 25.0fps, video only
VO-STE_ANG_-2313 mp4 1280x720 English (Subtitles) [?] 2313k , avc1.4d401f, 25.0fps, video only
VO-STE_ANG_-927 mp4 640x360 English (Subtitles) [?] 927k , avc1.4d401e, 25.0fps, video only
VO-STE_ANG_-426 mp4 384x216 English (Subtitles) [?] 426k , avc1.42e00d, 25.0fps, video only (best)
Notice how 216p
has been labeled as "(best)"?
Reminds me of a similar issue with the dwIE
I reported here...
see updated PR https://github.com/ytdl-org/youtube-dl/pull/29653.
At its latest commit, cdcb4af1c3257d955404ac5ea34becbb2bb1b1d3, works adequately well, without addressing the format sorting issue I reported above:
yt-dl -vF "https://www.arte.tv/en/videos/067103-000-A/mariupol/" =>
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--ffmpeg-location', '.\\FFmpeg', '--external-downloader-args', '-v 8 -stats', '-vF', 'https://www.arte.tv/en/videos/067103-000-A/mariupol/']
[debug] Encodings: locale cp1253, fs mbcs, out cp737, pref cp1253
[debug] youtube-dl version 2023.02.14.1919
[debug] Python version 3.4.4 (CPython) - Windows-Vista-6.0.6003-SP2
[debug] exe versions: ffmpeg n5.2-dev-2245-N-109649-gab8cde6, ffprobe n5.2-dev-2245-N-109649-gab8cde6, phantomjs 2.1.1, rtmpdump 2.4
[debug] Proxy map: {}
[ArteTV] 067103-000-A: Downloading JSON metadata
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[ArteTV] 067103-000-A: Downloading m3u8 information
[info] Available formats for 067103-000-A:
format code extension resolution note
VO-STF-program_audio_0-VO mp4 audio only [ru] French (Subtitles) [subFR]
VO-STF-1617 mp4 768x432 French (Subtitles) [subFR] 1617k , avc1.4d401e, 25.0fps, video only
VO-STF-2313 mp4 1280x720 French (Subtitles) [subFR] 2313k , avc1.4d401f, 25.0fps, video only
VO-STF-927 mp4 640x360 French (Subtitles) [subFR] 927k , avc1.4d401e, 25.0fps, video only
VO-STF-426 mp4 384x216 French (Subtitles) [subFR] 426k , avc1.42e00d, 25.0fps, video only (best)
... a "fix" for (default) format sorting (according to increasing resolution) will also be needed:
How true. You can imagine the back-porting process:
The PR was updated from the same code as used above and will shortly be brought into line.
Given the change in (1), a test for height
or some similar metadata item needs to be included in at least one test-case for each back-ported extractor.
The following video ... is claimed to be available ...
Thanks, but that's just a normal ArteTV
URL. I'm looking for a URL that matches the pattern r'https?://(?:www\.)?arte\.tv/player/v\d+/index\.php\?.*?\bjson_url=.+'
, eg has /player/v
after the domain name and a query parameter json_url
, as in the existing test URL shown above.
... I'm looking for a URL ...
Following works in a browser tab: https://www.arte.tv/player/v7/index.php?json_url=https://api.arte.tv/api/player/v2/config/de/060738-167-A Optional appending "?lang=de&autoplay=false&mute=0"
Thanks, both variants of that are working with the PR code. Actually I realised that you can invent these URLs from any non-embedded URL, so I used this:
https://www.arte.tv/player/v5/index.php?json_url=https%3A%2F%2Fapi.arte.tv%2Fapi%2Fplayer%2Fv2%2Fconfig%2Fpl%2F100103-000-A&lang=pl&autoplay=true&mute=100103-000-A
Checklist
Verbose log
Description
I was trying to download a video from website https://arte.tv/. Before,
youtube-dl
was working well with this site, but not anymore. I tried to re-download previously (successfully) downloaded videos, but I still receive the same error. I guess the page format changed.Thanks for your job.