[TF1] - This video is DRM protected.

ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites

http://ytdl-org.github.io/youtube-dl/

The Unlicense

130.06k stars 9.8k forks source link

[TF1] - This video is DRM protected. #32299

Open pingu1 opened 1 year ago

pingu1 commented 1 year ago

Checklist

[x] I'm reporting a broken site support
[x] I've verified that I'm running youtube-dl version 2021.12.17
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar issues including closed ones

Verbose log

> youtube-dl -f bestvideo+bestaudio https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/quotidien-du-8-juin-2023-14878832.html -o test.mp4 -v

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-f', 'bestvideo+bestaudio', 'https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/quotidien-du-8-juin-2023-14878832.html', '-o', 'test.mp4', '-v']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Python version 3.11.2 (CPython) - macOS-13.3.1-arm64-arm-64bit
[debug] exe versions: ffmpeg 5.1.2, ffprobe 5.1.2
[debug] Proxy map: {}
[TF1] quotidien-du-8-juin-2023-14878832: Downloading JSON metadata
[wat.tv] 13991716: Downloading JSON metadata
[wat.tv] 13991716: Downloading MPD manifest
ERROR: This video is DRM protected.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 816, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 837, in __extract_info
    ie_result = ie.extract(url)
                ^^^^^^^^^^^^^^^
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 535, in extract
    ie_result = self._real_extract(url)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/wat.py", line 90, in _real_extract
    raise ExtractorError('This video is DRM protected.', expected=True)
youtube_dl.utils.ExtractorError: This video is DRM protected.

Description

It looks like TF1 updated its played to either add real DRM protection or update something that makes downloads unavailable. The link pasted above worked when I used it on Friday June 9th, but is not working any more this week.

I tried with a bunch of video links from the TF1 website, and they're all resulting in the same error.

dirkf commented 1 year ago

If that's really the case, those links are out of scope for yt-dl.

If there is a video that plays in the browser with DRM disabled (see these yt-dlp instructions, equally applicable) but cannot be fetched with yt-dl, please report it.

dirkf commented 1 year ago

The API that the extractor uses to find the media for the problem link returns this, consistent with the error message:

{
  'media': {
    'id': '13991716',
    'type': 'video',
    'title': 'Quotidien du 8 juin 2023',
    'owner': 'quotidien-tmc',
    'preview': 'https://photos.tf1.fr/1280/720/vignette-p3-080623-29758b-aed945-0@1x.jpg',
    'sqPreview': 'https://photos.tf1.fr/400/400/vignette-square-ef727a-7063af-0@1x.jpg',
    'channel': 'tmc',
    'duration': 809,
    'sb': 'https://video-storyboards.tf1.fr/7b1697aa648d16106ecbf7401bd630e1e452d11f56ca1ed88fd372b255a20a37/lsb.vtt',
    'syndicable': false,
    'shortTitle': 'Quotidien du 8 juin 2023',
    'programName': 'Quotidien',
    'programSlug': 'quotidien-avec-yann-barthes',
    'attach': 'TMC'
  },
  ...
  'delivery': {
    'code': 200,
    'url': 'https://vod-das.cdn-0.diff.tf1.fr/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjaXAiOiI0Ni4yMDguNi4xMDciLCJjbWNkIjoiIiwiZXhwIjoxNjg2NTkzMTAwLCJnaWQiOiI5YzcxOTY0Ny05ZGQzLTQwNzMtOWQ0ZC1hNWQ1NjJmMmE0ZTEiLCJpYXQiOjE2ODY1Nzg3MDAsImlzcyI6ImRlbGl2ZXJ5IiwibWF4YiI6MjcwMDAwMCwic3RlbSI6Ii8yL1VTUC0weDAvMTcvMTYvMTM5OTE3MTYvc3NtL2UyOGJhOTA4MTQ0OGI3NTIwZDJlNzUzYTEzYTNjNzRjYjk3MjM4NDVmY2FmY2NlM2IyMTk0MDZjZjM1YzJkNmMuaXNtLzEzOTkxNzE2Lm1wZCIsInN1YiI6IjljNzE5NjQ3LTlkZDMtNDA3My05ZDRkLWE1ZDU2MmYyYTRlMSJ9.HdXZg04zp1qDeqX6FgNgMSoiwgeX1eTTQsl0Rt3-Rrc/2/USP-0x0/17/16/13991716/ssm/e28ba9081448b7520d2e753a13a3c74cb9723845fcafcce3b219406cf35c2d6c.ism/13991716.mpd',
    'country': 'GB',
    'format': 'dash',
    'drm': 'widevine',
    'drm-server': 'https://widevine-proxy-m.prod.p.tf1.fr/proxy?id=13991716&rn=1&ssc=&si=a5bb9ba5ccc9fe1f402bdc01ce6f85927577235c0d9a8f21e5cd5415440b231a&e=1686593100',
    'drms': [
      {
        'name': 'widevine',
        'url': 'https://widevine-proxy-m.prod.p.tf1.fr/proxy?id=13991716&rn=1&ssc=&si=a5bb9ba5ccc9fe1f402bdc01ce6f85927577235c0d9a8f21e5cd5415440b231a&e=1686593100'
      },
      {
        'name': 'playready',
        'url': 'https://playready-proxy.prod.p.tf1.fr/RightsManager.asmx?id=13991716&rn=1&ssc=&si=a5bb9ba5ccc9fe1f402bdc01ce6f85927577235c0d9a8f21e5cd5415440b231a&e=1686593100'
      }
    ],
    'stream-id': '13991716_dash_drm_nomax'
  }
}

pingu1 commented 1 year ago

Alright then, not much to do I guess. Thanks for investigating though

Vangelis66 commented 1 year ago

First of all, I'm not located within France, but my EU IP appears to be whitelisted 😜 ; trying below sample (2min worth) video clip (doesn't require a TF1 account):

https://www.tf1.fr/lci/le-temps-de-linfo/videos/mort-de-silvio-berlusconi-une-vie-de-succes-daffaires-et-de-polemiques-52186276.html

with latest yt-dl, I get (much like the OP):

yt-dl -vF "https://www.tf1.fr/lci/le-temps-de-linfo/videos/mort-de-silvio-berlusconi-une-vie-de-succes-daffaires-et-de-polemiques-52186276.html" => 

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--ffmpeg-location', '.\\FFmpeg', '--external-downloader-args', '-v 8 -stats', '-vF', 'https://www.tf1.fr/lci/le-temps-de-linfo/videos/mort-de-silvio-berlusconi-une-vie-de-succes-daffaires-et-de-polemiques-52186276.html']
[debug] Encodings: locale cp1253, fs mbcs, out cp737, pref cp1253
[debug] youtube-dl version 2023.06.12.1919 (single file build)
[debug] Python 3.4.4 (CPython x86 32bit) - Windows-Vista-6.0.6003-SP2 - OpenSSL 1.0.2d 9 Jul 2015
[debug] exe versions: ffmpeg n6.1-dev-1149-N-111033-gf11515c, ffprobe n6.1-dev-1149-N-111033-gf11515c, phantomjs 2.1.1, rtmpdump 2.4
[debug] Proxy map: {}
[TF1] mort-de-silvio-berlusconi-une-vie-de-succes-daffaires-et-de-polemiques-52186276: Downloading JSON metadata
[wat.tv] 13994339: Downloading JSON metadata
[wat.tv] 13994339: Downloading MPD manifest
ERROR: This video is DRM protected.
Traceback (most recent call last):
  File "YoutubeDL.py", line 825, in wrapper
  File "YoutubeDL.py", line 846, in __extract_info
  File "common.py", line 535, in extract
  File "D:\a\youtube-dl\youtube-dl\youtube_dl\extractor\wat.py", line 90, in _real_extract
youtube_dl.utils.ExtractorError: This video is DRM protected.

However, running:

yt-dl "https://www.tf1.fr/lci/le-temps-de-linfo/videos/mort-de-silvio-berlusconi-une-vie-de-succes-daffaires-et-de-polemiques-52186276.html" --dump-pages | FindStr /n mpd => 

FINDSTR: Line 3 is too long.
8:[wat.tv] Dumping request to https://vod-das.cdn-0.diff.tf1.fr/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjaXAiOiI0Ni4xMi4xNzEuMTIwIiwiY21jZCI6IiIsImV4cCI6MTY4NjYzNjc0NywiZ2lkIjoiNDk3MzE2OTQtMzdiZi00NWVlLWIwYmEtMDZlZjgzYzAzOGM2IiwiaWF0IjoxNjg2NjIyMzQ3LCJpc3MiOiJkZWxpdmVyeSIsIm1heGIiOjI3MDAwMDAsInN0ZW0iOiIvMi9VU1AtMHgwLzQzLzM5LzEzOTk0MzM5L3NzbS8xMzk5NDMzOXdkdi5pc20vMTM5OTQzMzkubXBkIiwic3ViIjoiNDk3MzE2OTQtMzdiZi00NWVlLWIwYmEtMDZlZjgzYzAzOGM2In0.Sxf2PsnDDZBFyB4jblRf0x7Cpkh61-ttelXlZsbNWZQ/2/USP-0x0/43/39/13994339/ssm/13994339wdv.ism/13994339.mpd
FINDSTR: Line 9 is too long.
ERROR: This video is DRM protected.

... one can get the URI of the cenc MPEG-DASH manifest:

https://vod-das.cdn-0.diff.tf1.fr/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjaXAiOiI0Ni4xMi4xNzEuMTIwIiwiY21jZCI6IiIsImV4cCI6MTY4NjYzNjc0NywiZ2lkIjoiNDk3MzE2OTQtMzdiZi00NWVlLWIwYmEtMDZlZjgzYzAzOGM2IiwiaWF0IjoxNjg2NjIyMzQ3LCJpc3MiOiJkZWxpdmVyeSIsIm1heGIiOjI3MDAwMDAsInN0ZW0iOiIvMi9VU1AtMHgwLzQzLzM5LzEzOTk0MzM5L3NzbS8xMzk5NDMzOXdkdi5pc20vMTM5OTQzMzkubXBkIiwic3ViIjoiNDk3MzE2OTQtMzdiZi00NWVlLWIwYmEtMDZlZjgzYzAzOGM2In0.Sxf2PsnDDZBFyB4jblRf0x7Cpkh61-ttelXlZsbNWZQ/2/USP-0x0/43/39/13994339/ssm/13994339wdv.ism/13994339.mpd

(another way to get this is to use URL sniffing in your preferred browser, or use specialised browser extensions that detect MPD/M3U8 manifests 😉 ); of course, this is still no good for yt-dl,

yt-dl -F "https://vod-das.cdn-0.diff.tf1.fr/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjaXAiOiI0Ni4xMi4xNzEuMTIwIiwiY21jZCI6IiIsImV4cCI6MTY4NjYzNjc0NywiZ2lkIjoiNDk3MzE2OTQtMzdiZi00NWVlLWIwYmEtMDZlZjgzYzAzOGM2IiwiaWF0IjoxNjg2NjIyMzQ3LCJpc3MiOiJkZWxpdmVyeSIsIm1heGIiOjI3MDAwMDAsInN0ZW0iOiIvMi9VU1AtMHgwLzQzLzM5LzEzOTk0MzM5L3NzbS8xMzk5NDMzOXdkdi5pc20vMTM5OTQzMzkubXBkIiwic3ViIjoiNDk3MzE2OTQtMzdiZi00NWVlLWIwYmEtMDZlZjgzYzAzOGM2In0.Sxf2PsnDDZBFyB4jblRf0x7Cpkh61-ttelXlZsbNWZQ/2/USP-0x0/43/39/13994339/ssm/13994339wdv.ism/13994339.mpd" => 

[generic] 13994339: Requesting header
WARNING: Falling back on generic information extractor.
[generic] 13994339: Downloading webpage
[generic] 13994339: Extracting information
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

... but if you remove three critical 😜 letters from that URI, this is no longer the case:

yt-dl -F "https://vod-das.cdn-0.diff.tf1.fr/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjaXAiOiI0Ni4xMi4xNzEuMTIwIiwiY21jZCI6IiIsImV4cCI6MTY4NjYzNjc0NywiZ2lkIjoiNDk3MzE2OTQtMzdiZi00NWVlLWIwYmEtMDZlZjgzYzAzOGM2IiwiaWF0IjoxNjg2NjIyMzQ3LCJpc3MiOiJkZWxpdmVyeSIsIm1heGIiOjI3MDAwMDAsInN0ZW0iOiIvMi9VU1AtMHgwLzQzLzM5LzEzOTk0MzM5L3NzbS8xMzk5NDMzOXdkdi5pc20vMTM5OTQzMzkubXBkIiwic3ViIjoiNDk3MzE2OTQtMzdiZi00NWVlLWIwYmEtMDZlZjgzYzAzOGM2In0.Sxf2PsnDDZBFyB4jblRf0x7Cpkh61-ttelXlZsbNWZQ/2/USP-0x0/43/39/13994339/ssm/13994339.ism/13994339.mpd" => 

[generic] 13994339: Requesting header
WARNING: Falling back on generic information extractor.
[generic] 13994339: Downloading webpage
[generic] 13994339: Extracting information
[info] Available formats for 13994339:
format code    extension  resolution note
audio=64000    m4a        audio only DASH audio   64k , m4a_dash container, mp4a.40.2 (48000Hz)
audio=128000   m4a        audio only DASH audio  128k , m4a_dash container, mp4a.40.2 (48000Hz)
video=201075   mp4        416x234    DASH video  201k , mp4_dash container, avc1.42C01E, 25fps, video only
video=401923   mp4        480x270    DASH video  401k , mp4_dash container, avc1.42C01E, 25fps, video only
video=602783   mp4        640x360    DASH video  602k , mp4_dash container, avc1.42C01E, 25fps, video only
video=1209132  mp4        1024x576   DASH video 1209k , mp4_dash container, avc1.4D401F, 25fps, video only
video=1712857  mp4        1024x576   DASH video 1712k , mp4_dash container, avc1.4D401F, 25fps, video only
video=2518509  mp4        1280x720   DASH video 2518k , mp4_dash container, avc1.4D401F, 25fps, video only (best)

If "I" can do it, I'm certain python code can do it, too 😄 ...

EDIT: This "hack" doesn't work for full-length episodes 😞 , which, I suppose, most are after ...

dirkf commented 1 year ago

Good spot: the existing extraction tactic does something similar, but as you note, and as the example JSON that I posted shows, it's not always applicable.

When it is, after appending .replace('wdv.ism/', '.ism/') to the existing .replace() in the WatIE extractor:

$ python -m youtube_dl -v -F 'https://www.tf1.fr/lci/le-temps-de-linfo/videos/mort-de-silvio-berlusconi-une-vie-de-succes-daffaires-et-de-polemiques-52186276.html'
[debug] System config: [u'--prefer-ffmpeg']
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'-F', u'https://www.tf1.fr/lci/le-temps-de-linfo/videos/mort-de-silvio-berlusconi-une-vie-de-succes-daffaires-et-de-polemiques-52186276.html']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Git HEAD: 32539c964
[debug] Python 2.7.18 (CPython i686 32bit) - Linux-4.4.0-210-generic-i686-with-Ubuntu-16.04-xenial - OpenSSL 1.1.1t  7 Feb 2023 - glibc 2.15
[debug] exe versions: avconv 4.3, avprobe 4.3, ffmpeg 4.3, ffprobe 4.3
[debug] Proxy map: {}
[TF1] mort-de-silvio-berlusconi-une-vie-de-succes-daffaires-et-de-polemiques-52186276: Downloading JSON metadata
[wat.tv] 13994339: Downloading JSON metadata
[wat.tv] 13994339: Downloading MPD manifest
[info] Available formats for 13994339:
format code         extension  resolution note
dash-audio=64000    m4a        audio only DASH audio   64k , m4a_dash container, mp4a.40.2 (48000Hz)
dash-audio=128000   m4a        audio only DASH audio  128k , m4a_dash container, mp4a.40.2 (48000Hz)
dash-video=201075   mp4        416x234    DASH video  201k , mp4_dash container, avc1.42C01E, 25fps, video only
dash-video=401923   mp4        480x270    DASH video  401k , mp4_dash container, avc1.42C01E, 25fps, video only
dash-video=602783   mp4        640x360    DASH video  602k , mp4_dash container, avc1.42C01E, 25fps, video only
dash-video=1209132  mp4        1024x576   DASH video 1209k , mp4_dash container, avc1.4D401F, 25fps, video only
dash-video=1712857  mp4        1024x576   DASH video 1712k , mp4_dash container, avc1.4D401F, 25fps, video only
dash-video=2518509  mp4        1280x720   DASH video 2518k , mp4_dash container, avc1.4D401F, 25fps, video only (best)
$

dirkf commented 1 year ago

Additionally:

yt-dlp is using pver=4020003 instead of pver=4001000, but with the same result
the WatIE extractor is obsolete (see http://www.wat.tv/video) and should be rolled into the TF1 extractor
for the problem episode 13991716, the returned JSON now says `country: 'GB', error: 'Permission insuffisante', with no delivery URL
for extractor test video https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/linfluence-meme-polska-et-tootatis-sont-en-manif-a-leur-facon-71394737.html (id 13970670), the delivery URL ending in .../USP-0x0/06/70/13970670/ssm/13970670wdv.ism/13970670.mpd is an alias for the one with the long hex string in the penultimate component, but the URL with the "3 characters" removed gives an empty result.

dirkf commented 1 year ago

Even more additionally, the only other extractor that redirects to WatIE is LCIIE which also appears to be obsolete: URLs redirect to tf1info.fr.

pingu1 commented 1 year ago

I'm guessing they're still working on the rollout of their new player, the very same URL that gave a This video is DRM protected. yesterday is now giving the same Permission insuffisante as @dirkf mentioned, whether I'm using a FR proxy or not.

I'm going to continue testing, full episode URLs, and will let you know here if something changes, but from what I understand, some additional work may be required on yt-dl side too?

dirkf commented 1 year ago

It seems that the current tactics used in the extractor, as amended above, will only give us a fraction of the available shows, so any research by local users (en France) as to the URLs accessed while starting to play a show would be useful. Even then, the extractor used to use a loophole similar to that shown by Vangelis66 to get playable media and there may not be an equivalent in the new player.

dirkf commented 1 year ago

... the very same URL that gave a This video is DRM protected. yesterday is now giving the same Permission insuffisante ...

The page says:

Dispo[nible] 3j[ours] + 30j avec MYTF1 MAX

Presumably a MAX account would restore access after 3 days.

pingu1 commented 1 year ago

After playing a bit with the page (I even created a free account), the web page seem to require accepting all cookies to access full episodes. If you reject cookies, or in Incognito Mode, the player won't load the video.

I will make tests with cookies and will update in case it works.

Vangelis66 commented 1 year ago

I'm guessing they're still working on the rollout of their new player, the very same URL that gave a This video is DRM protected. yesterday is now giving the same Permission insuffisante as dirkf mentioned, whether I'm using a FR proxy or not.

I can confirm 😭 ; 14hr ago, I saved this log:

yt-dl "https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/quotidien-du-8-juin-2023-14878832.html" --dump-pages | FindStr /n mpd =>

8:[wat.tv] Dumping request to https://vod-das.cdn-0.diff.tf1.fr/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjaXAiOiI0Ni4xMi4xNzEuMTIwIiwiY21jZCI6IiIsImV4cCI6MTY4NjYzOTA1MCwiZ2lkIjoiYzQzZjZkNzItMTY1ZS00ZGE5LThjMTktOTU1YTI1MDA4Nzc0IiwiaWF0IjoxNjg2NjI0NjUwLCJpc3MiOiJkZWxpdmVyeSIsIm1heGIiOjI3MDAwMDAsInN0ZW0iOiIvMi9VU1AtMHgwLzE3LzE2LzEzOTkxNzE2L3NzbS9lMjhiYTkwODE0NDhiNzUyMGQyZTc1M2ExM2EzYzc0Y2I5NzIzODQ1ZmNhZmNjZTNiMjE5NDA2Y2YzNWMyZDZjLmlzbS8xMzk5MTcxNi5tcGQiLCJzdWIiOiJjNDNmNmQ3Mi0xNjVlLTRkYTktOGMxOS05NTVhMjUwMDg3NzQifQ.la5x3nFzInh54u7wOZbG4cl_aAalIUeaLkhg_Gc522M/2/USP-0x0/17/16/13991716/ssm/e28ba9081448b7520d2e753a13a3c74cb9723845fcafcce3b219406cf35c2d6c.ism/13991716.mpd

but now it returns:

FINDSTR: Line 3 is too long.
ERROR: Permission insuffisante

FWIW, the .mpd above (unfortunately, the URI is uniquely tied to my IP address and has an expiration timestamp) is still valid now : 13970670.mpd.txt

I'm only speculating here, but perhaps the inadequate permission error is linked to them (TF1) now enforcing a user log-in to access "full-content", as opposed to episode excerpts/clips, rather than to a geo-IP ban; supposedly, this inadequate permission error could be mitigated via supplying cookies from a logged-in browser session (?) ...

Also 14hr ago, I experimented with the idea of a "mobile" UA; I ran:

yt-dl --user-agent "Mozilla/5.0 (iPhone; CPU iPhone OS 5_0 like Mac OS X) AppleWebKit/534.46 (KHTML, like Gecko) Version/5.1 Mobile/9A334 Safari/7534.48.3" "https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/quotidien-du-8-juin-2023-14878832.html" --dump-pages | FindStr /n m3u8

which yielded:

8:[wat.tv] Dumping request to https://vod-hls.cdn-0.diff.tf1.fr/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjaXAiOiI0Ni4xMi4xNzEuMTIwIiwiY21jZCI6IiIsImV4cCI6MTY4NjYzOTYxNywiZ2lkIjoiNWMyOTlmOGMtOWQxOS00N2JkLTkyMDctODcwYzg4YjMyOWE0IiwiaWF0IjoxNjg2NjI1MjE3LCJpc3MiOiJkZWxpdmVyeSIsIm1heGIiOjI3MDAwMDAsInN0ZW0iOiIvMi9VU1AtMHgwLzE3LzE2LzEzOTkxNzE2L3NzbS9lMjhiYTkwODE0NDhiNzUyMGQyZTc1M2ExM2EzYzc0Y2I5NzIzODQ1ZmNhZmNjZTNiMjE5NDA2Y2YzNWMyZDZjLmlzbS8xMzk5MTcxNi5tM3U4Iiwic3ViIjoiNWMyOTlmOGMtOWQxOS00N2JkLTkyMDctODcwYzg4YjMyOWE0In0.uDliq7GKSmalAmajyFj-vYcg5aMkkXtCwMASJL6dEMM/2/USP-0x0/17/16/13991716/ssm/e28ba9081448b7520d2e753a13a3c74cb9723845fcafcce3b219406cf35c2d6c.ism/13991716.m3u8

This HLS manifest (again, generated uniquely for my specific IP) is, sadly, Apple FairPlay DRMed: 13991716.m3u8.txt

So, I'm running out of further ideas 😢 ...

for extractor test video https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/linfluence-meme-polska-et-tootatis-sont-en-manif-a-leur-facon-71394737.html (id 13970670), the delivery URL ending in .../USP-0x0/06/70/13970670/ssm/13970670wdv.ism/13970670.mpd is an alias for the one with the long hex string in the penultimate component, but the URL with the "3 characters" removed gives an empty result.

... Can confirm on my side:

https://vod-das.cdn-0.diff.tf1.fr/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjaXAiOiI0Ni4xMi4xNzEuMTIwIiwiY21jZCI6IiIsImV4cCI6MTY4NjY4MzE4NiwiZ2lkIjoiNzdjNDYxM2EtZGE1OC00YzE2LWEwNzMtMGM4NjFmOTRhMGE2IiwiaWF0IjoxNjg2NjY4Nzg2LCJpc3MiOiJkZWxpdmVyeSIsIm1heGIiOjI4MDAwMDAsInN0ZW0iOiIvMi9VU1AtMHgwLzA2LzcwLzEzOTcwNjcwL3NzbS8zZjBlOGRmNzY4MmMzNGY3ODg4ZTJiYmE2ZDU1OTk4YThmOWI0NTZjNDJlM2IzYjUzZDk3ZWUxNDQ3M2JjMTFkLmlzbS8xMzk3MDY3MC5tcGQiLCJzdWIiOiI3N2M0NjEzYS1kYTU4LTRjMTYtYTA3My0wYzg2MWY5NGEwYTYifQ.DYtfj0NuCPQelUWGoNEBKAxSWnNYfHU6kWpSFrZ2amc/2/USP-0x0/06/70/13970670/ssm/13970670wdv.ism/13970670.mpd => 

https://vod-das.cdn-0.diff.tf1.fr/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjaXAiOiI0Ni4xMi4xNzEuMTIwIiwiY21jZCI6IiIsImV4cCI6MTY4NjY4MzE4NiwiZ2lkIjoiNzdjNDYxM2EtZGE1OC00YzE2LWEwNzMtMGM4NjFmOTRhMGE2IiwiaWF0IjoxNjg2NjY4Nzg2LCJpc3MiOiJkZWxpdmVyeSIsIm1heGIiOjI4MDAwMDAsInN0ZW0iOiIvMi9VU1AtMHgwLzA2LzcwLzEzOTcwNjcwL3NzbS8zZjBlOGRmNzY4MmMzNGY3ODg4ZTJiYmE2ZDU1OTk4YThmOWI0NTZjNDJlM2IzYjUzZDk3ZWUxNDQ3M2JjMTFkLmlzbS8xMzk3MDY3MC5tcGQiLCJzdWIiOiI3N2M0NjEzYS1kYTU4LTRjMTYtYTA3My0wYzg2MWY5NGEwYTYifQ.DYtfj0NuCPQelUWGoNEBKAxSWnNYfHU6kWpSFrZ2amc/2/USP-0x0/06/70/13970670/ssm/3f0e8df7682c34f7888e2bba6d55998a8f9b456c42e3b3b53d97ee14473bc11d.ism/13970670.mpd

while

https://vod-das.cdn-0.diff.tf1.fr/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjaXAiOiI0Ni4xMi4xNzEuMTIwIiwiY21jZCI6IiIsImV4cCI6MTY4NjY4MzE4NiwiZ2lkIjoiNzdjNDYxM2EtZGE1OC00YzE2LWEwNzMtMGM4NjFmOTRhMGE2IiwiaWF0IjoxNjg2NjY4Nzg2LCJpc3MiOiJkZWxpdmVyeSIsIm1heGIiOjI4MDAwMDAsInN0ZW0iOiIvMi9VU1AtMHgwLzA2LzcwLzEzOTcwNjcwL3NzbS8zZjBlOGRmNzY4MmMzNGY3ODg4ZTJiYmE2ZDU1OTk4YThmOWI0NTZjNDJlM2IzYjUzZDk3ZWUxNDQ3M2JjMTFkLmlzbS8xMzk3MDY3MC5tcGQiLCJzdWIiOiI3N2M0NjEzYS1kYTU4LTRjMTYtYTA3My0wYzg2MWY5NGEwYTYifQ.DYtfj0NuCPQelUWGoNEBKAxSWnNYfHU6kWpSFrZ2amc/2/USP-0x0/06/70/13970670/ssm/13970670.ism/13970670.mpd => 

[HTTP/1.1 404 Not Found]

octavioj commented 1 year ago

I hope someone finds a way soon. This new player of theirs broke a lot of things. Kodi plugins no longer work and streamlink also yields "permission insuffisante". TF1 seems intent on making things difficult to watch, or only watchable on browsers.

gamer191 commented 1 year ago

The API that the extractor uses to find the media for the problem link returns this, consistent with the error message:

If you remove context=MYTF1 you get this instead:

  "delivery": {
    "code": 200,
    "url": "REDACTED",
    "country": "AU",
    "format": "dash",
    "stream-id": "REDACTED_dash_nodrm_nomax"

I haven't tested that format, but I would note that the stream id contains nodrm!

PS: Thanks @dirkf for pointing out that the API mentions whether the formats are drm protected. Also, thanks @Vangelis66 for suggesting the mobile user agent trick. I accidentally discovered this because I was preparing to test that with the api, and was establishing which API parameters were essential and which were trackers

Vangelis66 commented 1 year ago

If you remove context=MYTF1 you get this instead: (redacted for brevity) I haven't tested that format, but I would note that the stream id contains nodrm!

You're certainly onto something there 😜 👍 ; are you being geo-fenced? Additionally, @bashonly wrote here:

video_data = self._download_json(
    'https://mediainfo.tf1.fr/mediainfocombo/' + video_id,
    video_id, query={'context': 'MYTF1', 'pver': '4020003'})
... I also didn't get any manifest URLs in the response. It's probable that I am geo-blocked

Well, as already posted above, I'm not being geo-blocked, so putting the above two hints into practice for the extractor test video:

https://mediainfo.tf1.fr/mediainfocombo/13970670?pver=4020003 =>

13970670.json

```json { "media": { "id": "13970670", "type": "video", "title": "L’Influencé : même Polska et Tootatis sont en manif (à leur façon)", "owner": "quotidien-tmc", "preview": "https://photos.tf1.fr/1280/720/linfluence-meme-polska-et-tootatis-sont-en-manif-a-leur-facon-4cd764-34be9c-0@1x.jpg", "sqPreview": "https://photos.tf1.fr/400/400/vignette-square-ef727a-7063af-0@1x.jpg", "channel": "tmc", "duration": 299, "sb": "https://video-storyboards.tf1.fr/0877cbfe294fd1c416dd4e22a86276558eaf9378ed45eac85a15c46c95e903cf/lsb.vtt", "syndicable": true, "shortTitle": "L’Influencé : même Polska et Tootatis sont en manif (à leur façon)", "programName": "Quotidien", "programSlug": "quotidien-avec-yann-barthes", "attach": "TMC" }, "content": { "title": "l-influence-:-meme-polska-et-tootatis-sont-en-manif-(a-leur-façon)", "emId": "2029088" }, "mediametrie": { "chapters": [ { "title": "13970670_PLAYLISTID_l-influence-:-meme-polska-et-tootatis-sont-en-manif-(a-leur-façon)", "estatTitle": "13970670_PLAYLISTID_l-influence-:-meme-polska-et-tootatis-sont-en-manif-(a-leur-façon)", "estatS1": "syndication", "estatS2": "quotidien-avec-yann-barthes", "estatS3": "extraits", "estatS4": "2023-03-24", "estatS5": "312781_2029088", "estatGenre": "desktop_browsing_windows_html5" } ], "id": "210010205672", "mediaId": "13970670", "estatMsDm": "TVOD", "estatMsCh": "861", "estatMsCid": "2029088" }, "richmedia": { "site": 577760, "mediaType": "video", "mediaLevel2": 0, "mediaLabel": { "label": "ext::[CONTENTTYPE]::quotidien_TMC::13970670_PLAYLISTID_l-influence-:-meme-polska-et-tootatis-sont-en-manif-(a-leur-façon)", "contentType": "vod_extraits" }, "refreshDuration": 60, "duration": 299, "isEmbedded": true, "broadcastMode": "clip", "webdomain": "" }, "fw": { "adManagerSwf": "https://mssl.fwmrm.net/p/tf1_flash_live/AdManager.swf", "id": 506334, "asset": "13970670", "profile": "506334:tf1_html5_desktop_live_mills", "url": "https://7b9de.v.fwmrm.net", "channel": "tmc", "page": "syndication", "section": "tmc_page_type_syndication_platform_desktop_media_desktop", "duration": 299 }, "googleAd": { "cmsid": "2576987", "vid": "f769c9e0-31fe-4234-a2fe-77ecafd2f0f4", "iu": "/169657134/MYTF1_VOD_Applicatif/TF1_VOD_Applicatif_Android_Tablette" }, "yb": { "accountCode": "tf1", "username": "unregistered", "transactionCode": "syndication", "media": { "title": "l-influence-:-meme-polska-et-tootatis-sont-en-manif-(a-leur-façon)", "title2": "quotidien", "duration": 299, "isLive": false }, "properties": { "content_id": "13970670", "genre": "desktop_browsing_windows_html5", "language": "FR", "year": "2023-03-24", "owner": "quotidien_TMC" }, "extraParams": { "param1": "syndication", "param2": "TMC", "param3": "DASH", "param4": "HTML5", "param5": "DESKTOP", "param6": "NO" }, "ads": { "campaign": "506334:tf1_html5_desktop_live_mills:tmc_page_type_syndication_platform_desktop_media_desktop" }, "enableAnalytics": true }, "streamroot": { "streamrootKey": "c1335d96-4e28-4b42-8065-9a5a8267d421", "backendUrl": "https://distribsr.tf1.fr", "backendHost": "https://strmt.tf1.fr", "libHost": "libstrmt.tf1.fr", "property": "ON" }, "delivery": { "code": 200, "url": "https://vod-das.cdn-0.diff.tf1.fr/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjaXAiOiI0Ni4xMi4xNzEuMTIwIiwiY21jZCI6IiIsImV4cCI6MTY4NjcxOTQ0NiwiZ2lkIjoiYmU3Mzg1NGYtYTRjZi00NGJmLWE3MjYtOTE1ZTE5NGU3MDZkIiwiaWF0IjoxNjg2NzA1MDQ2LCJpc3MiOiJkZWxpdmVyeSIsIm1heGIiOjIwMDAwMDAsInN0ZW0iOiIvMi9VU1AtMHgwLzA2LzcwLzEzOTcwNjcwL3NzbS8yNTc2NmEzN2FmZWE2YzRiMzcwODJkN2MxYWI1MDRhZDFhYzZjZDg1MmI5YTIzOWNjOTgzOWIzYTEwM2VlNWJkLmlzbS8xMzk3MDY3MC5tcGQiLCJzdWIiOiJiZTczODU0Zi1hNGNmLTQ0YmYtYTcyNi05MTVlMTk0ZTcwNmQifQ.C359IAMtW-_2Tx9ysXb9VWzDipldELO2B1IwKNnawLc/2/USP-0x0/06/70/13970670/ssm/25766a37afea6c4b37082d7c1ab504ad1ac6cd852b9a239cc9839b3a103ee5bd.ism/13970670.mpd", "country": "xx", "format": "dash", "stream-id": "13970670_dash_nodrm_nomax" } } ```

... and thus:

yt-dl -F "https://vod-das.cdn-0.diff.tf1.fr/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjaXAiOiI0Ni4xMi4xNzEuMTIwIiwiY21jZCI6IiIsImV4cCI6MTY4NjcxOTQ0NiwiZ2lkIjoiYmU3Mzg1NGYtYTRjZi00NGJmLWE3MjYtOTE1ZTE5NGU3MDZkIiwiaWF0IjoxNjg2NzA1MDQ2LCJpc3MiOiJkZWxpdmVyeSIsIm1heGIiOjIwMDAwMDAsInN0ZW0iOiIvMi9VU1AtMHgwLzA2LzcwLzEzOTcwNjcwL3NzbS8yNTc2NmEzN2FmZWE2YzRiMzcwODJkN2MxYWI1MDRhZDFhYzZjZDg1MmI5YTIzOWNjOTgzOWIzYTEwM2VlNWJkLmlzbS8xMzk3MDY3MC5tcGQiLCJzdWIiOiJiZTczODU0Zi1hNGNmLTQ0YmYtYTcyNi05MTVlMTk0ZTcwNmQifQ.C359IAMtW-_2Tx9ysXb9VWzDipldELO2B1IwKNnawLc/2/USP-0x0/06/70/13970670/ssm/25766a37afea6c4b37082d7c1ab504ad1ac6cd852b9a239cc9839b3a103ee5bd.ism/13970670.mpd" => 

[generic] 13970670: Requesting header
WARNING: Falling back on generic information extractor.
[generic] 13970670: Downloading webpage
[generic] 13970670: Extracting information
[info] Available formats for 13970670:
format code       extension  resolution note
audio_fra=64000   m4a        audio only [fr] DASH audio   64k , m4a_dash container, mp4a.40.2 (48000Hz)
audio_fra=128000  m4a        audio only [fr] DASH audio  128k , m4a_dash container, mp4a.40.2 (48000Hz)
video=200636      mp4        416x234    DASH video  200k , mp4_dash container, avc1.42C01E, 25fps, video only
video=401193      mp4        480x270    DASH video  401k , mp4_dash container, avc1.42C01E, 25fps, video only
video=601702      mp4        640x360    DASH video  601k , mp4_dash container, avc1.42C01E, 25fps, video only
video=1205422     mp4        1024x576   DASH video 1205k , mp4_dash container, avc1.4D401F, 25fps, video only
video=1707467     mp4        1024x576   DASH video 1707k , mp4_dash container, avc1.4D401F, 25fps, video only (best)

[generic] 13970670: Requesting header
WARNING: Falling back on generic information extractor.
[generic] 13970670: Downloading webpage
[generic] 13970670: Extracting information
[dashsegments] Total fragments: 39
[download] Destination: 13970670-13970670.fvideo=1205422.mp4
[download] 100% of 42.96MiB in 01:20
[dashsegments] Total fragments: 39
[download] Destination: 13970670-13970670.faudio_fra=128000.m4a
[download] 100% of 4.62MiB in 00:28
[ffmpeg] Merging formats into "13970670-13970670.mp4"
Deleting original file 13970670-13970670.fvideo=1205422.mp4 (pass -k to keep)
Deleting original file 13970670-13970670.faudio_fra=128000.m4a (pass -k to keep)

Sadly, only maxes out at SD (576p) ; the DRMed manifest affords HD (720p):

yt-dlp -F --allow-u "https://vod-das.cdn-0.diff.tf1.fr/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjaXAiOiI0Ni4xMi4xNzEuMTIwIiwiY21jZCI6IiIsImV4cCI6MTY4NjcyMTE0OSwiZ2lkIjoiNDAzMzhlOGQtYzYyYy00NWEwLWJiY2MtMzllYzgzMGIxZGE0IiwiaWF0IjoxNjg2NzA2NzQ5LCJpc3MiOiJkZWxpdmVyeSIsIm1heGIiOjI4MDAwMDAsInN0ZW0iOiIvMi9VU1AtMHgwLzA2LzcwLzEzOTcwNjcwL3NzbS8zZjBlOGRmNzY4MmMzNGY3ODg4ZTJiYmE2ZDU1OTk4YThmOWI0NTZjNDJlM2IzYjUzZDk3ZWUxNDQ3M2JjMTFkLmlzbS8xMzk3MDY3MC5tcGQiLCJzdWIiOiI0MDMzOGU4ZC1jNjJjLTQ1YTAtYmJjYy0zOWVjODMwYjFkYTQifQ.FNoK8L_SXVoyBoAD7m9PVh1a3l7GSy07E9zs9rWC8hE/2/USP-0x0/06/70/13970670/ssm/3f0e8df7682c34f7888e2bba6d55998a8f9b456c42e3b3b53d97ee14473bc11d.ism/13970670.mpd" => 

WARNING: You have asked for UNPLAYABLE formats to be listed/downloaded. This is a developer option intended for debugging.
         If you experience any issues while using this option, DO NOT open a bug report
[generic] Extracting URL: https://vod-das.cdn-0.diff.tf1.fr/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjaXAiOiI0Ni4xMi4xNzEuMT...11d.ism/13970670.mpd
[generic] 13970670: Downloading webpage
WARNING: [generic] Falling back on generic information extractor
[generic] 13970670: Extracting information
[info] Available formats for 13970670:
ID               EXT RESOLUTION FPS |   TBR PROTO | VCODEC        VBR ACODEC  ABR ASR MORE INFO
-----------------------------------------------------------------------------------------------------------------------
audio_fra=64000  m4a audio only     |   64k dash  | audio only        mp4a.40.2  64k 48k DRM [fr] DASH audio, m4a_dash
audio_fra=128000 m4a audio only     |  128k dash  | audio only        mp4a.40.2 128k 48k DRM [fr] DASH audio, m4a_dash
video=200636     mp4 416x234     25 |  201k dash  | avc1.42C01E  201k video only          DRM DASH video, mp4_dash
video=401193     mp4 480x270     25 |  401k dash  | avc1.42C01E  401k video only          DRM DASH video, mp4_dash
video=601702     mp4 640x360     25 |  602k dash  | avc1.42C01E  602k video only          DRM DASH video, mp4_dash
video=1205422    mp4 1024x576    25 | 1205k dash  | avc1.4D401F 1205k video only          DRM DASH video, mp4_dash
video=1707467    mp4 1024x576    25 | 1707k dash  | avc1.4D401F 1707k video only          DRM DASH video, mp4_dash
video=2509572    mp4 1280x720    25 | 2510k dash  | avc1.4D401F 2510k video only          DRM DASH video, mp4_dash

Still, SD >>> nothing 😜

gamer191 commented 1 year ago

are you being geo-fenced?

Nope, I was just short on time when I discovered this and too excited to not share it

I wonder why yt-dlp (and presumably yt-dl) currently send that parameter. It’s probably to get 720p formats, but I’ll do a git blame later

Sadly, only maxes out to SD (576p) ; the DRMed manifest affords HD (720p):

A manifest is just a list of URLs. I wonder if we can modify that manifest and get 720p :eyes:

dirkf commented 1 year ago

720 formats are available with some content.

However (and why I didn't find this when testing), the problem URL https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/quotidien-du-8-juin-2023-14878832.html still gives Permission insuffisante. This may just mean that it needs an account.

For the blame see https://github.com/yt-dlp/yt-dlp/issues/7303.

dirkf commented 1 year ago

Same issue with the problem video https://www.tf1.fr/tmc/une-nounou-d-enfer/videos/une-nounou-denfer-s03-e7-un-homme-metamorphose-01883134.html from https://github.com/yt-dlp/yt-dlp/issues/7303.

dirkf commented 1 year ago

With a playable show like the extractor test video where the highest non-DRM resolution is 576p, you can get the id (bandwidth) of the 720p content from the DRM DASH manifest and add the corresponding Representation to the non-DRM DASH manifest:

      <Representation
        id="video=2509572"
        bandwidth="2509572"
        width="1280"
        height="720"
        codecs="avc1.4D401F"
        scanType="progressive">
      </Representation>

To make a locally saved manifest playable, the BaseURL (typically dash/) must be completed from the manifest URL:

    <BaseURL>https://vod-das.cdn-0.diff.tf1.fr/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ.../2/USP-0x0/06/70/13970670/ssm/25766a37afea6c4b37082d7c1ab504ad1ac6cd852b9a239cc9839b3a103ee5bd.ism/dash/</BaseURL>

gamer191 commented 1 year ago

To make a locally saved manifest playable, the BaseURL (typically dash/) must be completed from the manifest URL:
    <BaseURL>https://vod-das.cdn-0.diff.tf1.fr/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ.../2/USP-0x0/06/70/13970670/ssm/25766a37afea6c4b37082d7c1ab504ad1ac6cd852b9a239cc9839b3a103ee5bd.ism/dash/</BaseURL>

thanks, I got stuck on that during my testing before. I shall return the favor by pointing out that that manifest can then be downloaded using yt-dlp with --enable-file-urls

dirkf commented 1 year ago

Indeed, that was my test:

Get manifest: yt-dl -g -f bestvideo ...
Find further ID(s) from DRM: yt-dlp -v -F --allow-unplayable ...
Edit manifest
Play: yt-dlp -v -f bestvideo -o - manifest --enable-file-urls | mpv -

Vangelis66 commented 1 year ago

dirkf, code has to be backported from yt-dlp so that when yt-dl is presented with a standalone DASH (.mpd) or HLS (.m3u8) manifest, it should clearly print a message about the eventual presence of cenc/DRM (the same way yt-dlp does); currently, the message:

ERROR: No video formats found; blahblahblah

produced is a "generic" one and the failure to detect "video formats" could be due to several other reasons (besides CENC):

yt-dl -F "https://cdn.bitmovin.com/content/assets/art-of-motion_drm/mpds/11331.mpd" => 

[generic] 11331: Requesting header
WARNING: Falling back on generic information extractor.
[generic] 11331: Downloading webpage
[generic] 11331: Extracting information
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . 
       Make sure you are using the latest version; type  youtube-dl -U  to update. 
       Be sure to call youtube-dl with the --verbose flag and include its complete output.

whereas:

yt-dlp -F "https://cdn.bitmovin.com/content/assets/art-of-motion_drm/mpds/11331.mpd" => 

[generic] Extracting URL: https://cdn.bitmovin.com/content/assets/art-of-motion_drm/mpds/11331.mpd
[generic] 11331: Downloading webpage
WARNING: [generic] Falling back on generic information extractor
[generic] 11331: Extracting information
ERROR: [generic] 11331: This video is DRM protected

yt-dlp is using pver=4020003 instead of pver=4001000, but with the same result

I'm not certain myself whether the ?pver= query is needed after all, since I get the same "delivery": results with both:

https://mediainfo.tf1.fr/mediainfocombo/13970670?pver=4001000

https://mediainfo.tf1.fr/mediainfocombo/13970670

... but, as I found out, the pver "matter" was taken care of in https://github.com/dirkf/youtube-dl/commit/081e6680af41516d6e96e45cfe73ebd044ff445e 😉 (not merged into master)

the watIE extractor is obsolete (see http://www.wat.tv/video) and should be rolled into the TF1 extractor

... and that was https://github.com/dirkf/youtube-dl/commit/b0a31cc6a490267d63e436f63e05a9a4b16901b2 + https://github.com/dirkf/youtube-dl/commit/7a37aa70a8906c9ff97faf4c1f955072c7f9e6e2 from a year ago...

so any research by local users (en France) as to the URLs accessed while starting to play a show would be useful.

After playing a bit with the page (I even created a free account), the web page seem to require accepting all cookies to access full episodes.

@pingu1 : Can you report any progress? What would be indeed useful is if you managed to decipher what extra queries (if any) can be used for accessing the Permission insuffisante full episodes... For "https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/quotidien-du-8-juin-2023-14878832.html" (id=13991716), "https://mediainfo.tf1.fr/mediainfocombo/13991716" now gives:

  "media": {
    "id": "13991716",
    "type": "video",
    "error_code": "DELIVERY_ERROR",
    "error_desc": "Erreur dans le delivery",
...
 },
  "delivery": {
    "code": 500,
    "error": "Erreur inconnue",
    "country": "xx",
    "format": "dash",
    "stream-id": "13991716_dash_nodrm_nomax"
  }

pingu1 commented 1 year ago

So far, what I tested is youtube-dl -F <test_url> with

dumping my cookies into a text file and use --cookies cookies.txt > ERROR: Permission insuffisante
adding my bearer token with --add-header "Authorizartion: Bearer <JWT>" >ERROR: Permission insuffisante`
using both the Auth header and the cookies > same thing
the 3 above but behind a VPN > same error

However, the web page loads the video properly. So, here are a few requests made by the web page with their respective responses https://mediainfo.tf1.fr/mediainfocombo/69e7b229-f421-4881-9539-a0409120ddfd?pver=5010000&context=MYTF1&topDomain=www.tf1.fr&platform=web&device=desktop&os=macOS&osVersion=10.15.7&playerVersion=5.10.0&productName=mytf1&productVersion=2.59.1&browser=chrome&browserVersion=113

{
   "media":{
      "id":"13991716",
      "type":"video",
      "title":"Quotidien du 8 juin 2023",
      "owner":"quotidien-tmc",
      "preview":"https://photos.tf1.fr/1280/720/vignette-p3-080623-29758b-aed945-0@1x.jpg",
      "sqPreview":"https://photos.tf1.fr/400/400/vignette-square-ef727a-7063af-0@1x.jpg",
      "channel":"tmc",
      "duration":809,
      "sb":"https://video-storyboards.tf1.fr/7b1697aa648d16106ecbf7401bd630e1e452d11f56ca1ed88fd372b255a20a37/lsb.vtt",
      "syndicable":false,
      "shortTitle":"Quotidien du 8 juin 2023",
      "programName":"Quotidien",
      "programSlug":"quotidien-avec-yann-barthes",
      "attach":"TMC"
   },
   "content":{
      "title":"quotidien-du-8-juin-2023",
      "emId":"2029136"
   },
   "mediametrie":{
      "chapters":[
         {
            "title":"13991716_PLAYLISTID_quotidien-du-8-juin-2023",
            "estatTitle":"13991716_PLAYLISTID_quotidien-du-8-juin-2023",
            "estatS1":"MYTF1",
            "estatS2":"quotidien-avec-yann-barthes",
            "estatS3":"catch-up",
            "estatS4":"2023-06-08",
            "estatS5":"312781_2029136",
            "estatGenre":"desktop_browsing_windows_html5"
         }
      ],
      "id":"210010205672",
      "mediaId":"13991716",
      "estatMsDm":"TVOD",
      "estatMsCh":"861",
      "estatMsCid":"2029136"
   },
   "richmedia":{
      "site":553584,
      "mediaType":"video",
      "mediaLevel2":104,
      "mediaLabel":{
         "label":"int::[CONTENTTYPE]::quotidien_TMC::13991716_PLAYLISTID_quotidien-du-8-juin-2023",
         "contentType":"vod_catch-up"
      },
      "refreshDuration":60,
      "duration":809,
      "isEmbedded":false,
      "broadcastMode":"clip",
      "webdomain":""
   },
   "fw":{
      "adManagerSwf":"https://mssl.fwmrm.net/p/tf1_flash_live/AdManager.swf",
      "id":506334,
      "asset":"13991716",
      "profile":"506334:tf1_html5_desktop_live_mills",
      "url":"https://7b9de.v.fwmrm.net",
      "channel":"tmc",
      "page":"sites",
      "section":"tmc_page_type_sites_platform_desktop_media_desktop",
      "duration":809
   },
   "googleAd":{
      "description_url":"https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/quotidien-du-8-juin-2023-14878832.html",
      "cmsid":"2576987",
      "vid":"69e7b229-f421-4881-9539-a0409120ddfd",
      "iu":"/169657134/MYTF1_VOD_Applicatif/TF1_VOD_Applicatif_Android_Tablette"
   },
   "yb":{
      "accountCode":"tf1",
      "username":"unregistered",
      "transactionCode":"MYTF1",
      "media":{
         "title":"quotidien-du-8-juin-2023",
         "title2":"quotidien",
         "duration":809,
         "isLive":false
      },
      "properties":{
         "content_id":"13991716",
         "genre":"desktop_browsing_windows_html5",
         "language":"FR",
         "year":"2023-06-08",
         "owner":"quotidien_TMC"
      },
      "extraParams":{
         "param1":"MYTF1",
         "param2":"TMC",
         "param3":"DASH",
         "param4":"HTML5",
         "param5":"DESKTOP",
         "param6":"YES"
      },
      "ads":{
         "campaign":"506334:tf1_html5_desktop_live_mills:tmc_page_type_sites_platform_desktop_media_desktop"
      },
      "enableAnalytics":true
   },
   "streamroot":{
      "streamrootKey":"c1335d96-4e28-4b42-8065-9a5a8267d421",
      "backendUrl":"https://distribsr.tf1.fr",
      "backendHost":"https://strmt.tf1.fr",
      "libHost":"libstrmt.tf1.fr",
      "property":"ON"
   },
   "delivery":{
      "code":200,
      "url":"https://vod-das.cdn-0.diff.tf1.fr/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjaXAiOiIxMDcuMTkwLjM0LjM2IiwiY21jZCI6IiIsImV4cCI6MTY4Njg0Nzc3NCwiZ2lkIjoiNjRlNDc2NDE1YzVlNGI0YWEyNGMxZGFiYjQ5NjQ2MjYiLCJpYXQiOjE2ODY4MzMzNzQsImlzcyI6ImRlbGl2ZXJ5IiwibWF4YiI6MjgwMDAwMCwic3RlbSI6Ii8yL1VTUC0weDAvMTcvMTYvMTM5OTE3MTYvc3NtL2UyOGJhOTA4MTQ0OGI3NTIwZDJlNzUzYTEzYTNjNzRjYjk3MjM4NDVmY2FmY2NlM2IyMTk0MDZjZjM1YzJkNmMuaXNtLzEzOTkxNzE2Lm1wZCIsInN1YiI6IjY0ZTQ3NjQxNWM1ZTRiNGFhMjRjMWRhYmI0OTY0NjI2In0.tWfYjGdANJBW-j67pdCoWTpANxeI68iGYr9ILbwbrg4/2/USP-0x0/17/16/13991716/ssm/e28ba9081448b7520d2e753a13a3c74cb9723845fcafcce3b219406cf35c2d6c.ism/13991716.mpd",
      "country":"CA",
      "format":"dash",
      "drm":"widevine",
      "drm-server":"https://widevine-proxy-m.prod.p.tf1.fr/proxy?id=13991716&rn=1&ssc=e7ac95bc-a0b5-4b95-a299-4d84a7938627&si=b059e44c89ee7219040bf32693ecb6266cb10c04f1aee6b1ee8aecd52bbf0bb8&e=1686847774",
      "drms":[
         {
            "name":"widevine",
            "url":"https://widevine-proxy-m.prod.p.tf1.fr/proxy?id=13991716&rn=1&ssc=e7ac95bc-a0b5-4b95-a299-4d84a7938627&si=b059e44c89ee7219040bf32693ecb6266cb10c04f1aee6b1ee8aecd52bbf0bb8&e=1686847774"
         },
         {
            "name":"playready",
            "url":"https://playready-proxy.prod.p.tf1.fr/RightsManager.asmx?id=13991716&rn=1&ssc=e7ac95bc-a0b5-4b95-a299-4d84a7938627&si=b059e44c89ee7219040bf32693ecb6266cb10c04f1aee6b1ee8aecd52bbf0bb8&e=1686847774"
         }
      ],
      "level":"Basic",
      "stream-id":"69e7b229-f421-4881-9539-a0409120ddfd_dash_drm_nomax"
   }
}

(you will notice this is showing I'm in Canada, but it's the same response if I'm using a FR proxy) Note: trying youtube-dl -F <mpd url> gives a ERROR: No video formats found; as mentioned in other messages

Web page then makes a call to https://vod-das.cdn-0.diff.tf1.fr/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjaXAiOiIxMDcuMTkwLjM0LjM2IiwiY21jZCI6IiIsImV4cCI6MTY4Njg0Nzc3NCwiZ2lkIjoiNjRlNDc2NDE1YzVlNGI0YWEyNGMxZGFiYjQ5NjQ2MjYiLCJpYXQiOjE2ODY4MzMzNzQsImlzcyI6ImRlbGl2ZXJ5IiwibWF4YiI6MjgwMDAwMCwic3RlbSI6Ii8yL1VTUC0weDAvMTcvMTYvMTM5OTE3MTYvc3NtL2UyOGJhOTA4MTQ0OGI3NTIwZDJlNzUzYTEzYTNjNzRjYjk3MjM4NDVmY2FmY2NlM2IyMTk0MDZjZjM1YzJkNmMuaXNtLzEzOTkxNzE2Lm1wZCIsInN1YiI6IjY0ZTQ3NjQxNWM1ZTRiNGFhMjRjMWRhYmI0OTY0NjI2In0.tWfYjGdANJBW-j67pdCoWTpANxeI68iGYr9ILbwbrg4/2/USP-0x0/17/16/13991716/ssm/e28ba9081448b7520d2e753a13a3c74cb9723845fcafcce3b219406cf35c2d6c.ism/13991716.mpd and THIS returns the following XML

<?xml version="1.0" encoding="utf-8"?>
<!-- Created with Unified Streaming Platform  (version=1.11.3-24483) -->
<MPD
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xmlns="urn:mpeg:dash:schema:mpd:2011"
  xmlns:cenc="urn:mpeg:cenc:2013"
  xmlns:mspr="urn:microsoft:playready"
  xsi:schemaLocation="urn:mpeg:dash:schema:mpd:2011 http://standards.iso.org/ittf/PubliclyAvailableStandards/MPEG-DASH_schema_files/DASH-MPD.xsd"
  type="static"
  mediaPresentationDuration="PT13M28.200S"
  maxSegmentDuration="PT9S"
  minBufferTime="PT10S"
  profiles="urn:mpeg:dash:profile:isoff-live:2011,urn:com:dashif:dash264">
  <Period
    id="1"
    duration="PT13M28.200S">
    <BaseURL>dash/</BaseURL>
    <AdaptationSet
      id="1"
      group="1"
      contentType="audio"
      lang="fr"
      minBandwidth="64000"
      maxBandwidth="128000"
      segmentAlignment="true"
      audioSamplingRate="48000"
      mimeType="audio/mp4"
      codecs="mp4a.40.2"
      startWithSAP="1">
      <AudioChannelConfiguration
        schemeIdUri="urn:mpeg:dash:23003:3:audio_channel_configuration:2011"
        value="2" />
      <!-- Common Encryption -->
      <ContentProtection
        schemeIdUri="urn:mpeg:dash:mp4protection:2011"
        value="cenc"
        cenc:default_KID="DE97A029-21CC-5588-A180-B1FBDBD5D5A4">
      </ContentProtection>
      <!-- PlayReady -->
      <ContentProtection
        schemeIdUri="urn:uuid:9A04F079-9840-4286-AB92-E65BE0885F95"
        value="MSPR 2.0">
        <cenc:pssh>AAACsHBzc2gAAAAAmgTweZhAQoarkuZb4IhflQAAApCQAgAAAQABAIYCPABXAFIATQBIAEUAQQBEAEUAUgAgAHgAbQBsAG4AcwA9ACIAaAB0AHQAcAA6AC8ALwBzAGMAaABlAG0AYQBzAC4AbQBpAGMAcgBvAHMAbwBmAHQALgBjAG8AbQAvAEQAUgBNAC8AMgAwADAANwAvADAAMwAvAFAAbABhAHkAUgBlAGEAZAB5AEgAZQBhAGQAZQByACIAIAB2AGUAcgBzAGkAbwBuAD0AIgA0AC4AMAAuADAALgAwACIAPgA8AEQAQQBUAEEAPgA8AFAAUgBPAFQARQBDAFQASQBOAEYATwA+ADwASwBFAFkATABFAE4APgAxADYAPAAvAEsARQBZAEwARQBOAD4APABBAEwARwBJAEQAPgBBAEUAUwBDAFQAUgA8AC8AQQBMAEcASQBEAD4APAAvAFAAUgBPAFQARQBDAFQASQBOAEYATwA+ADwASwBJAEQAPgBLAGEAQwBYADMAcwB3AGgAaQBGAFcAaABnAEwASAA3ADIAOQBYAFYAcABBAD0APQA8AC8ASwBJAEQAPgA8AEMASABFAEMASwBTAFUATQA+ACsAeABKAHkAYgB5AC8AUABiADcAOAA9ADwALwBDAEgARQBDAEsAUwBVAE0APgA8AEwAQQBfAFUAUgBMAD4AaAB0AHQAcABzADoALwAvAGQAcgBtAC0AcABsAGEAeQAuAHQAZgAxAC4AZgByAC8AcABsAGEAeQByAGUAYQBkAHkALwByAGkAZwBoAHQAcwBtAGEAbgBhAGcAZQByAC4AYQBzAG0AeAA8AC8ATABBAF8AVQBSAEwAPgA8AC8ARABBAFQAQQA+ADwALwBXAFIATQBIAEUAQQBEAEUAUgA+AA==</cenc:pssh>
        <mspr:IsEncrypted>1</mspr:IsEncrypted>
        <mspr:IV_Size>8</mspr:IV_Size>
        <mspr:kid>KaCX3swhiFWhgLH729XVpA==</mspr:kid>
        <mspr:pro>kAIAAAEAAQCGAjwAVwBSAE0ASABFAEEARABFAFIAIAB4AG0AbABuAHMAPQAiAGgAdAB0AHAAOgAvAC8AcwBjAGgAZQBtAGEAcwAuAG0AaQBjAHIAbwBzAG8AZgB0AC4AYwBvAG0ALwBEAFIATQAvADIAMAAwADcALwAwADMALwBQAGwAYQB5AFIAZQBhAGQAeQBIAGUAYQBkAGUAcgAiACAAdgBlAHIAcwBpAG8AbgA9ACIANAAuADAALgAwAC4AMAAiAD4APABEAEEAVABBAD4APABQAFIATwBUAEUAQwBUAEkATgBGAE8APgA8AEsARQBZAEwARQBOAD4AMQA2ADwALwBLAEUAWQBMAEUATgA+ADwAQQBMAEcASQBEAD4AQQBFAFMAQwBUAFIAPAAvAEEATABHAEkARAA+ADwALwBQAFIATwBUAEUAQwBUAEkATgBGAE8APgA8AEsASQBEAD4ASwBhAEMAWAAzAHMAdwBoAGkARgBXAGgAZwBMAEgANwAyADkAWABWAHAAQQA9AD0APAAvAEsASQBEAD4APABDAEgARQBDAEsAUwBVAE0APgArAHgASgB5AGIAeQAvAFAAYgA3ADgAPQA8AC8AQwBIAEUAQwBLAFMAVQBNAD4APABMAEEAXwBVAFIATAA+AGgAdAB0AHAAcwA6AC8ALwBkAHIAbQAtAHAAbABhAHkALgB0AGYAMQAuAGYAcgAvAHAAbABhAHkAcgBlAGEAZAB5AC8AcgBpAGcAaAB0AHMAbQBhAG4AYQBnAGUAcgAuAGEAcwBtAHgAPAAvAEwAQQBfAFUAUgBMAD4APAAvAEQAQQBUAEEAPgA8AC8AVwBSAE0ASABFAEEARABFAFIAPgA=</mspr:pro>
      </ContentProtection>
      <!-- Widevine -->
      <ContentProtection
        schemeIdUri="urn:uuid:EDEF8BA9-79D6-4ACE-A3C8-27DCD51D21ED">
        <cenc:pssh>AAAAMHBzc2gAAAAA7e+LqXnWSs6jyCfc1R0h7QAAABAiCDEzOTkxNzE2SOPclZsG</cenc:pssh>
      </ContentProtection>
      <Label>Français</Label>
      <Role schemeIdUri="urn:mpeg:dash:role:2011" value="main" />
      <SegmentTemplate
        timescale="48000"
        initialization="e28ba9081448b7520d2e753a13a3c74cb9723845fcafcce3b219406cf35c2d6c-$RepresentationID$.dash"
        media="e28ba9081448b7520d2e753a13a3c74cb9723845fcafcce3b219406cf35c2d6c-$RepresentationID$-$Time$.dash">
        <SegmentTimeline>
          <S t="0" d="384000" r="99" />
          <S d="394239" />
        </SegmentTimeline>
      </SegmentTemplate>
      <Representation
        id="audio_fra=64000"
        bandwidth="64000">
      </Representation>
      <Representation
        id="audio_fra=128000"
        bandwidth="128000">
      </Representation>
    </AdaptationSet>
    <AdaptationSet
      id="2"
      group="2"
      contentType="video"
      par="16:9"
      minBandwidth="200253"
      maxBandwidth="2502747"
      maxWidth="1280"
      maxHeight="720"
      segmentAlignment="true"
      sar="1:1"
      frameRate="25"
      mimeType="video/mp4"
      startWithSAP="1">
      <!-- Common Encryption -->
      <ContentProtection
        schemeIdUri="urn:mpeg:dash:mp4protection:2011"
        value="cenc"
        cenc:default_KID="DE97A029-21CC-5588-A180-B1FBDBD5D5A4">
      </ContentProtection>
      <!-- PlayReady -->
      <ContentProtection
        schemeIdUri="urn:uuid:9A04F079-9840-4286-AB92-E65BE0885F95"
        value="MSPR 2.0">
        <cenc:pssh>AAACsHBzc2gAAAAAmgTweZhAQoarkuZb4IhflQAAApCQAgAAAQABAIYCPABXAFIATQBIAEUAQQBEAEUAUgAgAHgAbQBsAG4AcwA9ACIAaAB0AHQAcAA6AC8ALwBzAGMAaABlAG0AYQBzAC4AbQBpAGMAcgBvAHMAbwBmAHQALgBjAG8AbQAvAEQAUgBNAC8AMgAwADAANwAvADAAMwAvAFAAbABhAHkAUgBlAGEAZAB5AEgAZQBhAGQAZQByACIAIAB2AGUAcgBzAGkAbwBuAD0AIgA0AC4AMAAuADAALgAwACIAPgA8AEQAQQBUAEEAPgA8AFAAUgBPAFQARQBDAFQASQBOAEYATwA+ADwASwBFAFkATABFAE4APgAxADYAPAAvAEsARQBZAEwARQBOAD4APABBAEwARwBJAEQAPgBBAEUAUwBDAFQAUgA8AC8AQQBMAEcASQBEAD4APAAvAFAAUgBPAFQARQBDAFQASQBOAEYATwA+ADwASwBJAEQAPgBLAGEAQwBYADMAcwB3AGgAaQBGAFcAaABnAEwASAA3ADIAOQBYAFYAcABBAD0APQA8AC8ASwBJAEQAPgA8AEMASABFAEMASwBTAFUATQA+ACsAeABKAHkAYgB5AC8AUABiADcAOAA9ADwALwBDAEgARQBDAEsAUwBVAE0APgA8AEwAQQBfAFUAUgBMAD4AaAB0AHQAcABzADoALwAvAGQAcgBtAC0AcABsAGEAeQAuAHQAZgAxAC4AZgByAC8AcABsAGEAeQByAGUAYQBkAHkALwByAGkAZwBoAHQAcwBtAGEAbgBhAGcAZQByAC4AYQBzAG0AeAA8AC8ATABBAF8AVQBSAEwAPgA8AC8ARABBAFQAQQA+ADwALwBXAFIATQBIAEUAQQBEAEUAUgA+AA==</cenc:pssh>
        <mspr:IsEncrypted>1</mspr:IsEncrypted>
        <mspr:IV_Size>8</mspr:IV_Size>
        <mspr:kid>KaCX3swhiFWhgLH729XVpA==</mspr:kid>
        <mspr:pro>kAIAAAEAAQCGAjwAVwBSAE0ASABFAEEARABFAFIAIAB4AG0AbABuAHMAPQAiAGgAdAB0AHAAOgAvAC8AcwBjAGgAZQBtAGEAcwAuAG0AaQBjAHIAbwBzAG8AZgB0AC4AYwBvAG0ALwBEAFIATQAvADIAMAAwADcALwAwADMALwBQAGwAYQB5AFIAZQBhAGQAeQBIAGUAYQBkAGUAcgAiACAAdgBlAHIAcwBpAG8AbgA9ACIANAAuADAALgAwAC4AMAAiAD4APABEAEEAVABBAD4APABQAFIATwBUAEUAQwBUAEkATgBGAE8APgA8AEsARQBZAEwARQBOAD4AMQA2ADwALwBLAEUAWQBMAEUATgA+ADwAQQBMAEcASQBEAD4AQQBFAFMAQwBUAFIAPAAvAEEATABHAEkARAA+ADwALwBQAFIATwBUAEUAQwBUAEkATgBGAE8APgA8AEsASQBEAD4ASwBhAEMAWAAzAHMAdwBoAGkARgBXAGgAZwBMAEgANwAyADkAWABWAHAAQQA9AD0APAAvAEsASQBEAD4APABDAEgARQBDAEsAUwBVAE0APgArAHgASgB5AGIAeQAvAFAAYgA3ADgAPQA8AC8AQwBIAEUAQwBLAFMAVQBNAD4APABMAEEAXwBVAFIATAA+AGgAdAB0AHAAcwA6AC8ALwBkAHIAbQAtAHAAbABhAHkALgB0AGYAMQAuAGYAcgAvAHAAbABhAHkAcgBlAGEAZAB5AC8AcgBpAGcAaAB0AHMAbQBhAG4AYQBnAGUAcgAuAGEAcwBtAHgAPAAvAEwAQQBfAFUAUgBMAD4APAAvAEQAQQBUAEEAPgA8AC8AVwBSAE0ASABFAEEARABFAFIAPgA=</mspr:pro>
      </ContentProtection>
      <!-- Widevine -->
      <ContentProtection
        schemeIdUri="urn:uuid:EDEF8BA9-79D6-4ACE-A3C8-27DCD51D21ED">
        <cenc:pssh>AAAAMHBzc2gAAAAA7e+LqXnWSs6jyCfc1R0h7QAAABAiCDEzOTkxNzE2SOPclZsG</cenc:pssh>
      </ContentProtection>
      <Role schemeIdUri="urn:mpeg:dash:role:2011" value="main" />
      <SegmentTemplate
        timescale="600"
        initialization="e28ba9081448b7520d2e753a13a3c74cb9723845fcafcce3b219406cf35c2d6c-$RepresentationID$.dash"
        media="e28ba9081448b7520d2e753a13a3c74cb9723845fcafcce3b219406cf35c2d6c-$RepresentationID$-$Time$.dash">
        <SegmentTimeline>
          <S t="0" d="4800" r="99" />
          <S d="4920" />
        </SegmentTimeline>
      </SegmentTemplate>
      <Representation
        id="video=200253"
        bandwidth="200253"
        width="416"
        height="234"
        codecs="avc1.42C01E"
        scanType="progressive">
      </Representation>
      <Representation
        id="video=400512"
        bandwidth="400512"
        width="480"
        height="270"
        codecs="avc1.42C01E"
        scanType="progressive">
      </Representation>
      <Representation
        id="video=600609"
        bandwidth="600609"
        width="640"
        height="360"
        codecs="avc1.42C01E"
        scanType="progressive">
      </Representation>
      <Representation
        id="video=1201467"
        bandwidth="1201467"
        width="1024"
        height="576"
        codecs="avc1.4D401F"
        scanType="progressive">
      </Representation>
      <Representation
        id="video=1701995"
        bandwidth="1701995"
        width="1024"
        height="576"
        codecs="avc1.4D401F"
        scanType="progressive">
      </Representation>
      <Representation
        id="video=2502747"
        bandwidth="2502747"
        width="1280"
        height="720"
        codecs="avc1.4D401F"
        scanType="progressive">
      </Representation>
    </AdaptationSet>
  </Period>
</MPD>

Then the fragments start getting downloaded via GET sent to https://vod-das.cdn-0.diff.tf1.fr/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjaXAiOiIxMDcuMTkwLjM0LjM2IiwiY21jZCI6IiIsImV4cCI6MTY4Njg0Nzc3NCwiZ2lkIjoiNjRlNDc2NDE1YzVlNGI0YWEyNGMxZGFiYjQ5NjQ2MjYiLCJpYXQiOjE2ODY4MzMzNzQsImlzcyI6ImRlbGl2ZXJ5IiwibWF4YiI6MjgwMDAwMCwic3RlbSI6Ii8yL1VTUC0weDAvMTcvMTYvMTM5OTE3MTYvc3NtL2UyOGJhOTA4MTQ0OGI3NTIwZDJlNzUzYTEzYTNjNzRjYjk3MjM4NDVmY2FmY2NlM2IyMTk0MDZjZjM1YzJkNmMuaXNtLzEzOTkxNzE2Lm1wZCIsInN1YiI6IjY0ZTQ3NjQxNWM1ZTRiNGFhMjRjMWRhYmI0OTY0NjI2In0.tWfYjGdANJBW-j67pdCoWTpANxeI68iGYr9ILbwbrg4/2/USP-0x0/17/16/13991716/ssm/e28ba9081448b7520d2e753a13a3c74cb9723845fcafcce3b219406cf35c2d6c.ism/dash/e28ba9081448b7520d2e753a13a3c74cb9723845fcafcce3b219406cf35c2d6c-audio_fra=128000.dash

Now, I'm not an expert of YTDL, I'm not sure how I could help you more, so if you have any specific commands for me to run / test locally, let me know.

dirkf commented 1 year ago

If you can run those tests with a version that sends pver with 50100000 (in extractor/wat.py), that would determine whether sending the latest player ID is important.

Vangelis66 commented 1 year ago

I shall return the favor by pointing out that that manifest can then be downloaded using yt-dlp with --enable-file-urls

ffmpeg will also download a locally saved MPD manifest with a valid (full, not "relative" to the master MPD one) BaseURL , if your ffmpeg build has been compiled with an enabled dash demuxer; that last part requires you configure a third-party XML parser, like --enable-libxml2, when building; older ffmpeg versions (< 5.0 ?) would also need the option -protocol_whitelist "file,http,tcp,https,tls"...

Even then, I've found out that you have to manually specify the desired video+audio "Representations" to fetch via -map flags for video+audio streams, because ffmpeg isn't very good itself at automatically picking the best "qualities" available 😞 ...

Each input or output url can, in principle, contain any number of streams of different types (video/audio/subtitle/attachment/data). The allowed number and/or types of streams may be limited by the container format. Selecting which streams from which inputs will go into which output is either done automatically or with the -map option (see the Stream selection chapter).

To refer to input files in options, you must use their indices (0-based). E.g. the first input file is 0, the second is 1, etc. Similarly, streams within a file are referred to by their indices. E.g. 2:3 refers to the fourth stream in the third input file. Also see the Stream specifiers chapter.

Vangelis66 commented 1 year ago

Merci pingu1 for your voluminous report 👍 ; the type of JSON response with valid delivery url:

...
   },
   "delivery":{
      "code":200,
      "url":"https://vod-das.cdn-0.diff.tf1.fr/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjaXAiOiIxMDcuMTkwLjM0LjM2IiwiY21jZCI6IiIsImV4cCI6MTY4Njg0Nzc3NCwiZ2lkIjoiNjRlNDc2NDE1YzVlNGI0YWEyNGMxZGFiYjQ5NjQ2MjYiLCJpYXQiOjE2ODY4MzMzNzQsImlzcyI6ImRlbGl2ZXJ5IiwibWF4YiI6MjgwMDAwMCwic3RlbSI6Ii8yL1VTUC0weDAvMTcvMTYvMTM5OTE3MTYvc3NtL2UyOGJhOTA4MTQ0OGI3NTIwZDJlNzUzYTEzYTNjNzRjYjk3MjM4NDVmY2FmY2NlM2IyMTk0MDZjZjM1YzJkNmMuaXNtLzEzOTkxNzE2Lm1wZCIsInN1YiI6IjY0ZTQ3NjQxNWM1ZTRiNGFhMjRjMWRhYmI0OTY0NjI2In0.tWfYjGdANJBW-j67pdCoWTpANxeI68iGYr9ILbwbrg4/2/USP-0x0/17/16/13991716/ssm/e28ba9081448b7520d2e753a13a3c74cb9723845fcafcce3b219406cf35c2d6c.ism/13991716.mpd",
...

on a se connecter-only TF1 free title is exactly what yt-dl needs to get its hands on to overcome the Permission insuffisante hurdle... Sadly, https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/quotidien-du-8-juin-2023-14878832.html now shows:

Vous devez être connecté et abonné à MYTF1 MAX pour profiter de cette vidéo.

i.e., AIUI, now requires a paid subscription plan to be watched (which perplexes things even more 😠 ) ...

https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/quotidien-deuxieme-partie-du-14-juin-2023-35738352.html

is available for free for the next 6 days 😜 ...

dumping my cookies into a text file and use --cookies cookies.txt =>ERROR: Permission insuffisante

Are you certain that the needed cookies (exported from your logged-in TF1 session) were in the "legacy" Netscape HTTP Cookie format? Because that's the only type of a cookies.txt file that yt-dl can successfully parse...

However, the web page loads the video properly.

... Thus, it should be possible, at least in theory, to replicate the browser's behaviour in yt-dl itself; some times, passing an exported cookies.txt file isn't just enough; have you also tried passing to yt-dl the exact user-agent string of the browser (Google Chrome 113?) the cookies were extracted from? Thus, you'd issue:

youtube-dl --user-agent "UA string of Chrome 113 x64 on macOS-13.3.1-arm64-arm-64bit" --cookies "path/to/cookies.txt" "TF1-show-URI"

If you do get access, then you'd probably hit again the original This video is DRM protected. error that was current on June 12th, before they enforced the mandatory log-in on June 13th (😠) ...

adding my bearer token with --add-header "Authorizartion: Bearer " => ERROR: Permission insuffisante`

This was also mentioned (en francais) in the Replays MyTF1 issue thread, here and here ; FWIW, JWT online decrypter: "https://jwt.io/"

https://mediainfo.tf1.fr/mediainfocombo/69e7b229-f421-4881-9539-a0409120ddfd?pver=5010000&context=MYTF1&topDomain=www.tf1.fr&platform=web&device=desktop&os=macOS&osVersion=10.15.7&playerVersion=5.10.0&productName=mytf1&productVersion=2.59.1&browser=chrome&browserVersion=113

... Without being logged-in to TF1, this simple request URI doesn't produce a delivery url for me,

...
  "delivery": {
    "code": 403,
    "error": "Permission insuffisante",
...

so additional queries/request headers are needed to extort the sought-after MPD URI 😭 ...

I also noted that the web page doesn't use the media id ("id":"13991716") of the show (yt-dl does use that), but rather the "vid":"69e7b229-f421-4881-9539-a0409120ddfd" when making the API request, but, probably, this isn't a deal-breaker here...

For freely and anonymously accessible TF1 clips, I compiled locally a yt-dl build with https://github.com/dirkf/youtube-dl/commit/081e6680af41516d6e96e45cfe73ebd044ff445e merged-in, and on top of it the "hack" found by @gamer191 (👍):

yt-dl -F "https://www.tf1.fr/lci/le-temps-de-linfo/videos/mort-de-silvio-berlusconi-une-vie-de-succes-daffaires-et-de-polemiques-52186276.html" => 

[tf1.fr] mort-de-silvio-berlusconi-une-vie-de-succes-daffaires-et-de-polemiques-52186276: Downloading webpage
[tf1.fr] mort-de-silvio-berlusconi-une-vie-de-succes-daffaires-et-de-polemiques-52186276: Downloading JSON metadata
[tf1.fr] mort-de-silvio-berlusconi-une-vie-de-succes-daffaires-et-de-polemiques-52186276: Downloading MPD manifest
[tf1.fr] mort-de-silvio-berlusconi-une-vie-de-succes-daffaires-et-de-polemiques-52186276: Downloading JSON metadata
[info] Available formats for 9fc76431-f588-4c87-b7ec-cca1d578ff71:format code         extension  resolution note
dash-audio=64000    m4a        audio only DASH audio   64k , m4a_dash container, mp4a.40.2 (48000Hz)
dash-audio=128000   m4a        audio only DASH audio  128k , m4a_dash container, mp4a.40.2 (48000Hz)
dash-video=201075   mp4        416x234    DASH video  201k , mp4_dash container, avc1.42C01E, 25fps, video only
dash-video=401923   mp4        480x270    DASH video  401k , mp4_dash container, avc1.42C01E, 25fps, video only
dash-video=602783   mp4        640x360    DASH video  602k , mp4_dash container, avc1.42C01E, 25fps, video only
dash-video=1209132  mp4        1024x576   DASH video 1209k , mp4_dash container, avc1.4D401F, 25fps, video only
dash-video=1712857  mp4        1024x576   DASH video 1712k , mp4_dash container, avc1.4D401F, 25fps, video only
dash-video=2518509  mp4        1280x720   DASH video 2518k , mp4_dash container, avc1.4D401F, 25fps, video only (best)

As discussed already, some of those (DRM-free) max out at 576p (id=13970670, vid=f769c9e0-31fe-4234-a2fe-77ecafd2f0f4), others (like the one above, id=13994339, vid=9fc76431-f588-4c87-b7ec-cca1d578ff71) go all the way up to 720p...

(I even created a free account),

If it comes down to it, pingu1 may have to share that free account, at least with the current maintainer 😉 ; BTW, "downstream" have this to say: https://github.com/yt-dlp/yt-dlp/blob/master/CONTRIBUTING.md#are-you-willing-to-share-account-details-if-needed

pingu1 commented 1 year ago

Alright, so, from what I'm seeing with the program that I watch

smaller clips are "free to watch" as long as they are on the website
full length episodes require an account. A free account gives you 7 days to watch the episode
if you try to watch a full length episode after 7 days, you need a paid account that lets you watch replays for 30 days

So yeah, the video link I provided last week already expired for "free accounts".

I usually use Brave, but the DRM protection of the video is currently totally messing up the browser, even if I allow the streaming of protected content. To watch videos from a connected account, I now use Chrome (Version 113.0.5672.126) on MacOS (13.3.1) and use the Get cookies.txt LOCALLY Chrome extension to export all my cookies in the Netscape format.

I tried exporting my cookies only for tf1 domain, as well as for all cookies for all domains, but using

youtube-dl -v --user-agent "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36" --cookies cookies.txt -F https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/quotidien-premiere-partie-du-15-juin-2023-56085642.html

always gives the same error

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '--user-agent', 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36', '--cookies', 'cookies.txt', '-F', 'https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/quotidien-premiere-partie-du-15-juin-2023-56085642.html']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Python version 3.11.2 (CPython) - macOS-13.3.1-arm64-arm-64bit
[debug] exe versions: ffmpeg 5.1.2, ffprobe 5.1.2
[debug] Proxy map: {}
[TF1] quotidien-premiere-partie-du-15-juin-2023-56085642: Downloading JSON metadata
[wat.tv] 13993907: Downloading JSON metadata
ERROR: Permission insuffisante
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 816, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 837, in __extract_info
    ie_result = ie.extract(url)
                ^^^^^^^^^^^^^^^
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 535, in extract
    ie_result = self._real_extract(url)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/wat.py", line 67, in _real_extract
    raise ExtractorError(error_desc, expected=True)
youtube_dl.utils.ExtractorError: Permission insuffisante

From the web browser, I was able to get the associated mpd link, but not much luck with it either

youtube-dl -v --user-agent "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36" --cookies cookies.txt -F "https://vod-das.cdn-0.diff.tf1.fr/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjaXAiOiIxMDcuMTkwLjM0LjM2IiwiY21jZCI6IiIsImV4cCI6MTY4NjkzNjQzOSwiZ2lkIjoiNjRlNDc2NDE1YzVlNGI0YWEyNGMxZGFiYjQ5NjQ2MjYiLCJpYXQiOjE2ODY5MjIwMzksImlzcyI6ImRlbGl2ZXJ5IiwibWF4YiI6MjgwMDAwMCwic3RlbSI6Ii8yL1VTUC0weDAvMzkvMDcvMTM5OTM5MDcvc3NtL2Y5OTMwODJhMzAxYjg5ZjJmNjBiOWE4NDFmMzMxNjhlYWE4OWFjNzlkYmM3YWYxZmVkMmY5MGM4NjFiM2EyY2IuaXNtLzEzOTkzOTA3Lm1wZCIsInN1YiI6IjY0ZTQ3NjQxNWM1ZTRiNGFhMjRjMWRhYmI0OTY0NjI2In0.juIxHUxcZCWA6T_HTJvLXNQzR5m0bn4ZaY00dOeZYV4/2/USP-0x0/39/07/13993907/ssm/f993082a301b89f2f60b9a841f33168eaa89ac79dbc7af1fed2f90c861b3a2cb.ism/13993907.mpd"

results in

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', '--user-agent', 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36', '--cookies', 'cookies.txt', '-F', 'https://vod-das.cdn-0.diff.tf1.fr/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjaXAiOiIxMDcuMTkwLjM0LjM2IiwiY21jZCI6IiIsImV4cCI6MTY4NjkzNjQzOSwiZ2lkIjoiNjRlNDc2NDE1YzVlNGI0YWEyNGMxZGFiYjQ5NjQ2MjYiLCJpYXQiOjE2ODY5MjIwMzksImlzcyI6ImRlbGl2ZXJ5IiwibWF4YiI6MjgwMDAwMCwic3RlbSI6Ii8yL1VTUC0weDAvMzkvMDcvMTM5OTM5MDcvc3NtL2Y5OTMwODJhMzAxYjg5ZjJmNjBiOWE4NDFmMzMxNjhlYWE4OWFjNzlkYmM3YWYxZmVkMmY5MGM4NjFiM2EyY2IuaXNtLzEzOTkzOTA3Lm1wZCIsInN1YiI6IjY0ZTQ3NjQxNWM1ZTRiNGFhMjRjMWRhYmI0OTY0NjI2In0.juIxHUxcZCWA6T_HTJvLXNQzR5m0bn4ZaY00dOeZYV4/2/USP-0x0/39/07/13993907/ssm/f993082a301b89f2f60b9a841f33168eaa89ac79dbc7af1fed2f90c861b3a2cb.ism/13993907.mpd']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Python version 3.11.2 (CPython) - macOS-13.3.1-arm64-arm-64bit
[debug] exe versions: ffmpeg 5.1.2, ffprobe 5.1.2
[debug] Proxy map: {}
[generic] 13993907: Requesting header
WARNING: Falling back on generic information extractor.
[generic] 13993907: Downloading webpage
[generic] 13993907: Extracting information
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 816, in wrapper
    return func(self, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 837, in __extract_info
    ie_result = ie.extract(url)
                ^^^^^^^^^^^^^^^
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 535, in extract
    ie_result = self._real_extract(url)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/generic.py", line 2693, in _real_extract
    self._sort_formats(info_dict['formats'])
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 1375, in _sort_formats
    raise ExtractorError('No video formats found')
youtube_dl.utils.ExtractorError: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

dirkf commented 1 year ago

From this

https://mediainfo.tf1.fr/mediainfocombo/69e7b229-f421-4881-9539-a0409120ddfd?pver=5010000&context=MYTF1&topDomain=www.tf1.fr&platform=web&device=desktop&os=macOS&osVersion=10.15.7&playerVersion=5.10.0&productName=mytf1&productVersion=2.59.1&browser=chrome&browserVersion=113

we get the query data:

{
    'pver': '5010000',
    'context': 'MYTF1',
    'topDomain': 'www.tf1.fr',
    'platform': 'web',
    'device': 'desktop',
    'os': 'macOS',
    'osVersion': '10.15.7',
    'playerVersion': '5.10.0'
    'productName': 'mytf1',
    'productVersion': '2.59.1',
    'browser': 'chrome',
    'browserVersion': '113'
}

For some reason they want the player version twice (in different formats) and the context and productName parameters are essentially duplicates.

The current yt-dl master extractor

uses the Wat ID instead of the UUID
just sends
```
{
'context': 'MYTF1',
'pver': '4001000'
}
```
Let's look at https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/quotidien-deuxieme-partie-du-14-juin-2023-35738352.html (at least for the next 6j).

yt-dl release uses Wat ID 13993604 and returns Permission insuffisante. The result is returned with a version equivalent to yt-dl master and with a new version that sends pver as 5010000, and when sending all the parameters listed above. Even getting the Video ID (93cc5844-be74-4343-8df9-0730b76dddf6) and using that instead of the Wat ID (so apparently replicating the request shown above) gets the same answer.

Some additional headers must be required.

OT: enemies of the Académie will be happy to see the use of "miracle" and "stars" to describe the slimming drug that the US stars (f.) are mad for:

...le nouveau médicament « miracle » pour maigrir que s’arrachent les stars américaines ...

Zovi3U commented 1 year ago

Hi @pingu1 , the autorization header with the right token is mandatory but this header dont pass the proxy or the VPN on my test.. please try to make a echo of your request to see the result on your side (https://echo.hoppscotch.io/ or http://httpbin.org/anything).

enigmathix commented 1 year ago

Here is some code to get the tokens:

import requests
import json

apiKey = '3_hWgJdARhz_7l1oOp3a8BDLoR9cuWZpUaKG4aqF7gum9_iK3uTZ2VlDBl8ANf8FVk'

formData = {
'loginID': 'USERNAME', #TODO
'password': 'PASSWORD', #TODO
'APIKey': apiKey,
}

tf1Session = requests.Session()

# get login info
r = tf1Session.post('https://compte.tf1.fr/accounts.login', data=formData)
if r.status_code == 200:
    js = json.loads(r.content)

    formData = {
    'consent_ids': [ "1", "2", "3", "4", "10001", "10003", "10005", "10007", "10013", "10015", "10017", "10019", "10009", "10011", "13002", "13001", "10004", "10014", "10016", "10018", "10020", "10010", "10012", "10006", "10008"],
    'signature': js['UIDSignature'],
    'timestamp': int(js['signatureTimestamp']),
    'uid': js['UID'],
    }

    # get tokens
    r = tf1Session.post('https://www.tf1.fr/token/gigya/web', json=formData)

    if r.status_code == 200:
        js = json.loads(r.content)
        print(js)

The token should be passed to the header "Autorization: Bearer TOKEN" to obtain the list of videos. The refresh_token could be used to get a new token but it's not necessary here because the script does a login every time.

Edit: simplified code, bootstrap is not necessary with limited login parameters.

Vangelis66 commented 1 year ago

The day before yesterday I caved in ( 😜 ) and registered a free MYTF1 account, which would satisfy below proviso:

full length episodes require an account. A free account gives you 7 days to watch the episode

FYI, you'd need:

a (French?) surname
provide a (adult?) birth date
a (temp) working e-mail address which has to be verified within 24h, for the account to be activated
compose a password containing at least: 1 capital letter, one small letter, one Arabic numeral and 1 "special" character
a French postal code

The "episode" (actually, it's the last part [3] of a full emission lasting 1h40m in total 😉 ) I used for testing is:

https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/quotidien-du-14-juin-2023-34621006.html

(id=13993616, vid=f9d23aaf-7d06-4107-828f-d28e518f8032)

For dissecting this, I just used my browser's Developer Tools...

The crucial web request to MYTF1's API looks like this:

https://mediainfo.tf1.fr/mediainfocombo/f9d23aaf-7d06-4107-828f-d28e518f8032?pver=5010000&context=MYTF1&topDomain=unknown&platform=web&device=desktop&os=windows&osVersion=[*]&playerVersion=5.10.0&productName=mytf1&productVersion=2.59.1&browser=[*]&browserVersion=[*]

[*]: redacted for privacy

My testing has revealed that the only absolutely necessary query of that default set is ?context=MYTF1; when being logged-in, you additionally have to pass as Request Header the gigya-issued authorization, which can be grabbed from your browser; in my case, it looks like below:

authorization: Bearer eyJhbGciOiJFUzI1NiIsImtpZCI6ImRlZmF1bHQiLCJ0eXAiOiJKV1QifQ.eyJhYlRlc3QiOnt9LCJleHAiOjE2ODcxNzQ1NjcsImZjZmEiOjAsImZ0Ijp7fSwiaWF0IjoxNjg3MDczNzY3LCJpZGQiOjAsImlzcyI6ImdpZ3lhIiwibHZsIjoxLCJtc2NiIjpmYWxzZSwibm9BdWRpZW5jZSI6ZmFsc2UsIm5vQ3VzdG9taXphdGlvbiI6ZmFsc2UsIm5vUHJvZmlsaW5nIjpmYWxzZSwicGx0Ijoid2ViIiwic3ViIjoiNTZhNjE5ZWYwZjdlNGJkZDhiOGQ1YmYyNDU5MWI3MWYifQ.TBjAs8vxt7t6qrFEPv1sRdNmcJVIujiBC5M7662CPcUQMJJmxbCctuQzwhySKFm00N-P9clmGRGuD88U46H3VQ

If you decode the Bearer string up to the last dot (.), you get:

{
  "alg": "ES256",
  "kid": "default",
  "typ": "JWT"
}

{
  "abTest": {},
  "exp": 1687174567,
  "fcfa": 0,
  "ft": {},
  "iat": 1687073767,
  "idd": 0,
  "iss": "gigya",
  "lvl": 1,
  "mscb": false,
  "noAudience": false,
  "noCustomization": false,
  "noProfiling": false,
  "plt": "web",
  "sub": "56a619ef0f7e4bdd8b8d5bf24591b71f"
}

Lifespan of the authorization is ca. 24h since first generation...

Then:

MYTF1-WR1

which returns this:

MYTF1-WR2

The cenc MPEG-DASH manifest received looks like below (parts of it removed, for privacy concerns):

https://vod-das.cdn-0.diff.tf1.fr/eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.[*]ZCI6IiIsImV4cCI6MTY4NzA4OTA2NywiZ2lkIjoiNTZhNjE5ZWYwZjdlNGJkZDhiOGQ1YmYyNDU5MWI3MWYiLCJpYXQiOjE2ODcwNzQ2NjcsImlzcyI6ImRlbGl2ZXJ5IiwibWF4YiI6MjgwMDAwMCwic3RlbSI6Ii8yL1VTUC0weDAvMzYvMTYvMTM5OTM2MTYvc3NtL2YyYWM3MTI3ZTAwZDBlOGE4MjI5MjNkMzgxOTg4N2Q3YzA3NzQ4OWM2OWY3YTE1Zjk4MGFkM2FjODQxODhmMjQuaXNtLzEzOTkzNjE2Lm1wZCIsInN1YiI6IjU2YTYxOWVmMGY3ZTRiZGQ4YjhkNWJmMjQ1OTFiNzFmIn0.J8bifZ0dwi6HbJ95cE_90lA6z5GUL6z0b7QX7IFod1I/2/USP-0x0/36/16/13993616/ssm/f2ac7127e00d0e8a822923d3819887d7c077489c69f7a15f980ad3ac84188f24.ism/13993616.mpd

This one contains an embedded type of "JWT", between the first and second "/" of the URI:

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.[*]ZCI6IiIsImV4cCI6MTY4NzA4OTA2NywiZ2lkIjoiNTZhNjE5ZWYwZjdlNGJkZDhiOGQ1YmYyNDU5MWI3MWYiLCJpYXQiOjE2ODcwNzQ2NjcsImlzcyI6ImRlbGl2ZXJ5IiwibWF4YiI6MjgwMDAwMCwic3RlbSI6Ii8yL1VTUC0weDAvMzYvMTYvMTM5OTM2MTYvc3NtL2YyYWM3MTI3ZTAwZDBlOGE4MjI5MjNkMzgxOTg4N2Q3YzA3NzQ4OWM2OWY3YTE1Zjk4MGFkM2FjODQxODhmMjQuaXNtLzEzOTkzNjE2Lm1wZCIsInN1YiI6IjU2YTYxOWVmMGY3ZTRiZGQ4YjhkNWJmMjQ1OTFiNzFmIn0.J8bifZ0dwi6HbJ95cE_90lA6z5GUL6z0b7QX7IFod1I

If you decode this string up to the second dot (.), you get:

{
  "alg": "HS256",
  "typ": "JWT"
}

{
  "cip": "redacted client IP",
  "cmcd": "",
  "exp": 1687089067,
  "gid": "56a619ef0f7e4bdd8b8d5bf24591b71f",
  "iat": 1687074667,
  "iss": "delivery",
  "maxb": 2800000,
  "stem": "/2/USP-0x0/36/16/13993616/ssm/f2ac7127e00d0e8a822923d3819887d7c077489c69f7a15f980ad3ac84188f24.ism/13993616.mpd",
  "sub": "56a619ef0f7e4bdd8b8d5bf24591b71f"
}

Lifespan of this MPD is ca. 4h since generation 😉 ...

If you feed it to yt-dlp --allow-u -F, you get:

[generic] 13993616: Extracting information
[info] Available formats for 13993616:
ID               EXT RESOLUTION FPS |   TBR PROTO | VCODEC        VBR ACODEC  ABR ASR MORE INFO
-----------------------------------------------------------------------------------------------------------------------
audio_fra=64000  m4a audio only     |   64k dash  | audio only        mp4a.40.2  64k 48k DRM [fr] DASH audio, m4a_dash
audio_fra=128000 m4a audio only     |  128k dash  | audio only        mp4a.40.2 128k 48k DRM [fr] DASH audio, m4a_dash
video=200155     mp4 416x234     25 |  200k dash  | avc1.42C01E  200k video only          DRM DASH video, mp4_dash
video=400305     mp4 480x270     25 |  400k dash  | avc1.42C01E  400k video only          DRM DASH video, mp4_dash
video=600438     mp4 640x360     25 |  600k dash  | avc1.42C01E  600k video only          DRM DASH video, mp4_dash
video=1200769    mp4 1024x576    25 | 1201k dash  | avc1.4D401F 1201k video only          DRM DASH video, mp4_dash
video=1701076    mp4 1024x576    25 | 1701k dash  | avc1.4D401F 1701k video only          DRM DASH video, mp4_dash
video=2501640    mp4 1280x720    25 | 2502k dash  | avc1.4D401F 2502k video only          DRM DASH video, mp4_dash

Moving forward requires you (somehow) download the encrypted MPD and then circumvent DRM protection by (somehow) acquiring decryption key[s], which is a strict NO-NO 🚫 on this issue tracker...

To conclude,

yt-dl -vF "https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/quotidien-du-14-juin-2023-34621006.html" --add-header "authorization:Bearer eyJhbGciOiJFUzI1NiIsImtpZCI6ImRlZmF1bHQiLCJ0eXAiOiJKV1QifQ.eyJhYlRlc3QiOnt9LCJleHAiOjE2ODcxNzQ1NjcsImZjZmEiOjAsImZ0Ijp7fSwiaWF0IjoxNjg3MDczNzY3LCJpZGQiOjAsImlzcyI6ImdpZ3lhIiwibHZsIjoxLCJtc2NiIjpmYWxzZSwibm9BdWRpZW5jZSI6ZmFsc2UsIm5vQ3VzdG9taXphdGlvbiI6ZmFsc2UsIm5vUHJvZmlsaW5nIjpmYWxzZSwicGx0Ijoid2ViIiwic3ViIjoiNTZhNjE5ZWYwZjdlNGJkZDhiOGQ1YmYyNDU5MWI3MWYifQ.TBjAs8vxt7t6qrFEPv1sRdNmcJVIujiBC5M7662CPcUQMJJmxbCctuQzwhySKFm00N-P9clmGRGuD88U46H3VQ" => 

[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--ffmpeg-location', '.\\FFmpeg', '--external-downloader-args', '-v 8 -stats', '-vF', 'https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/quotidien-du-14-juin-2023-34621006.html', '--add-header', 'authorization:Bearer eyJhbGciOiJFUzI1NiIsImtpZCI6ImRlZmF1bHQiLCJ0eXAiOiJKV1QifQ.eyJhYlRlc3QiOnt9LCJleHAiOjE2ODcxNzQ1NjcsImZjZmEiOjAsImZ0Ijp7fSwiaWF0IjoxNjg3MDczNzY3LCJpZGQiOjAsImlzcyI6ImdpZ3lhIiwibHZsIjoxLCJtc2NiIjpmYWxzZSwibm9BdWRpZW5jZSI6ZmFsc2UsIm5vQ3VzdG9taXphdGlvbiI6ZmFsc2UsIm5vUHJvZmlsaW5nIjpmYWxzZSwicGx0Ijoid2ViIiwic3ViIjoiNTZhNjE5ZWYwZjdlNGJkZDhiOGQ1YmYyNDU5MWI3MWYifQ.TBjAs8vxt7t6qrFEPv1sRdNmcJVIujiBC5M7662CPcUQMJJmxbCctuQzwhySKFm00N-P9clmGRGuD88U46H3VQ']
[debug] Adding header from command line option authorization:Bearer eyJhbGciOiJFUzI1NiIsImtpZCI6ImRlZmF1bHQiLCJ0eXAiOiJKV1QifQ.eyJhYlRlc3QiOnt9LCJleHAiOjE2ODcxNzQ1NjcsImZjZmEiOjAsImZ0Ijp7fSwiaWF0IjoxNjg3MDczNzY3LCJpZGQiOjAsImlzcyI6ImdpZ3lhIiwibHZsIjoxLCJtc2NiIjpmYWxzZSwibm9BdWRpZW5jZSI6ZmFsc2UsIm5vQ3VzdG9taXphdGlvbiI6ZmFsc2UsIm5vUHJvZmlsaW5nIjpmYWxzZSwicGx0Ijoid2ViIiwic3ViIjoiNTZhNjE5ZWYwZjdlNGJkZDhiOGQ1YmYyNDU5MWI3MWYifQ.TBjAs8vxt7t6qrFEPv1sRdNmcJVIujiBC5M7662CPcUQMJJmxbCctuQzwhySKFm00N-P9clmGRGuD88U46H3VQ
[debug] Encodings: locale cp1253, fs mbcs, out cp737, pref cp1253
[debug] youtube-dl version 2023.06.18.40298 (single file build)
[debug] Python 3.4.4 (CPython x86 32bit) - Windows-Vista-6.0.6003-SP2 - OpenSSL 1.0.2d 9 Jul 2015
[debug] exe versions: ffmpeg n6.1-dev-1149-N-111033-gf11515c, ffprobe n6.1-dev-1149-N-111033-gf11515c, phantomjs 2.1.1, rtmpdump 2.4
[debug] Proxy map: {}
[TF1] quotidien-du-14-juin-2023-34621006: Downloading JSON metadata
[wat.tv] 13993616: Downloading JSON metadata
[wat.tv] 13993616: Downloading MPD manifest
ERROR: This video is DRM protected.
Traceback (most recent call last):
  File "YoutubeDL.py", line 825, in wrapper
  File "YoutubeDL.py", line 846, in __extract_info
  File "common.py", line 535, in extract
  File "D:\a\youtube-dl\youtube-dl\youtube_dl\extractor\wat.py", line 90, in _real_extract
youtube_dl.utils.ExtractorError: This video is DRM protected.

... and this is the furthest you can go with yt-dl (and simply posted to demonstrate how you can override the Permission insuffisante obstacle 😉 ) ...

As to the if "clear" (DRM-free) streams can be obtained by applying the @gamer191 find, I don't have good news to share 😢 :

MYTF1-WR3

which, sadly, returns:

MYTF1-WR4

Indeed, using my youtube-dl local build (from source https://github.com/dirkf/youtube-dl/commit/081e6680af41516d6e96e45cfe73ebd044ff445e but with the gamer191 "hack" applied):

youtube-dl -F "https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/quotidien-du-14-juin-2023-34621006.html" --add-header "authorization:Bearer eyJhbGciOiJFUzI1NiIsImtpZCI6ImRlZmF1bHQiLCJ0eXAiOiJKV1QifQ.eyJhYlRlc3QiOnt9LCJleHAiOjE2ODcxNzQ1NjcsImZjZmEiOjAsImZ0Ijp7fSwiaWF0IjoxNjg3MDczNzY3LCJpZGQiOjAsImlzcyI6ImdpZ3lhIiwibHZsIjoxLCJtc2NiIjpmYWxzZSwibm9BdWRpZW5jZSI6ZmFsc2UsIm5vQ3VzdG9taXphdGlvbiI6ZmFsc2UsIm5vUHJvZmlsaW5nIjpmYWxzZSwicGx0Ijoid2ViIiwic3ViIjoiNTZhNjE5ZWYwZjdlNGJkZDhiOGQ1YmYyNDU5MWI3MWYifQ.TBjAs8vxt7t6qrFEPv1sRdNmcJVIujiBC5M7662CPcUQMJJmxbCctuQzwhySKFm00N-P9clmGRGuD88U46H3VQ" => 

[tf1.fr] quotidien-du-14-juin-2023-34621006: Downloading webpage
[tf1.fr] quotidien-du-14-juin-2023-34621006: Downloading JSON metadata
ERROR: Erreur dans le delivery

Unless someone else has an epiphany, the tf1IE/watIE can be updated simply for the sake of those "free-without-a-mandatory-account" short episode excerpts/clips...

Vangelis66 commented 1 year ago

5 days ago, @pingu1 wrote:

adding my bearer token with --add-header "Authorizartion: Bearer <JWT>" => ERROR: Permission insuffisante

... Funnily enough, I only spotted this now (prescription glasses need to be replaced, am afraid 😄 ), but if you did issue Authorizartion as the Request Header's name, no wonder you couldn't "get in"; should've been, as analysed above, authorization 😃 ...

rclsilver commented 1 year ago

Hi,

The issue seems to have been fixed for an other project with https://github.com/Catch-up-TV-and-More/plugin.video.catchuptvandmore/pull/1164

Any chance to backport the fix in youtube-dl?

octavioj commented 1 year ago

Hi,

The issue seems to have been fixed for an other project with Catch-up-TV-and-More/plugin.video.catchuptvandmore#1164

Any chance to backport the fix in youtube-dl?

I think it requires porting inputstream.adaptive from Kodi to youtube-dl and yt-dlp . It would be interesting but not sure how hard it would be.

Vangelis66 commented 1 year ago

@rclsilver , @octavioj

I don't want to burst your bubbles (😜 ), but to my knowledge (which isn't perfect, BTW), any Kodi-type plugin does not circumvent the actual DRM protection, which is exactly what the major issue is in this tracker and application (youtube-dl):

Il semble que TF1 ait ajouté une protection DRM: les videos sont encryptées et il faut obtenir une clé pour les décoder. Le problème est qu'ils peuvent restreindre l'accès aux apps qu'ils veulent pour obtenir cette clé.

donc il y a 2 problèmes: l'un de permission, l'autre de décryption.

il reste le problème du DRM, qui est à mon avis insoluble sans être partenaire de TF1 (c'est le but du DRM).

Un DRM, c'est simplement donner la clé à quelqu'un pour faire une action sans lui dire comment on lui a donné la clé. Si l'action est faisable via le site, elle est faisable via Kodi/CUTVM.

Sur mytf1, les autres browsers utilisent du mpeg-dash mpd et widevine drm C'est ce que nous utilisons dans CUTVM aussi. Inputstream adaptive, le plugin utilisé pour la gestion drm ne gère pas le fairplay

https://github.com/xbmc/inputstream.adaptive/wiki#supported-drm

La gestion du DRM de Widevine n'a pas changé. Ce qui manque à Catchup TV c'est la connexion au compte MyTF1 (qui est maintenant obligatoire).

IIANM, Kodi employs either a specialised plugin or a headless Google Chrome browser (e.g. on Raspberry Pis) with a working/whitelisted Widevine CDM, the decrypted media stream is then fed on-the-fly to the user's monitor (e.g. TV screen) ...

https://kodi.tv/article/dev-journal-kodi-and-drm/

From our perspective, supporting low-level DRM is a first step to changing that. Basically, what this means is providing some sort of interface to work with the DRM already present on your system. For example, Android ships with software that plays back DRMed content from Netflix. Kodi could hook into this already existing software in Android to playback the same content, so you never have to leave Kodi. As another example, in Windows, there isn't a very good 10-foot interface for Netflix, but if you have installed Chrome, you can watch Netflix from your desktop. Supporting low-level DRM means that Kodi could hook into the binary blob inside Chrome and use it purely for handling the DRM while video playback and control stays with Kodi.

Because the GPL and closed source binary blobs aren't compatible, we can't ship them along Kodi. What we can do is use them to play DRMed content if they are present on your system. Combine that with a specific provider add-on and you could access Netflix, HBO Go or your favorite legit content provider from inside Kodi.

octavioj wrote:

I think it requires porting inputstream.adaptive from Kodi to youtube-dl and yt-dlp

Kodi is first and foremost a rich media streaming application, while yt-dl/yt-dlp are primarily downloaders for local/offline playback in the player of one's choice and not in the player the media service chooses to use...

Even if these two apps borrowed code from CUTVM that deals with the new "must-log-in" MYTF1 policy, it'd be a moot point if avenues to clear (DRM-free) MYTF1 streams aren't discovered...