[TF1] No video formats found

pingu1 commented 2 years ago

Checklist

[ ] I'm reporting a new site support request I tried to find similar issues in issues reported in the past 7 days but I also noticed this error started to occur only today ...
[x] I've verified that I'm running youtube-dl version 2021.12.17
```
youtube-dl --version
2021.12.17
```
[x] I've checked that all provided URLs are alive and playable in a browser Indeed
(still) working URL: https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/quotidien-premiere-partie-du-16-mai-2022-52927145.html
broken URL: https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/quotidien-premiere-partie-du-17-mai-2022-71087892.html
[x] I've checked that none of provided URLs violate any copyrights
[x] I've searched the bugtracker for similar site support requests including closed ones

Example URLs

These 2 URLs are properly working in a web browser (requires authentication). The working URL also works with youtube-dl.

Description

Using youtube-dl -f bestvideo+bestaudio <URL> has been working like a charm up until the URL pasted above. Also, using the -F flag to see available formats fails.

Full output:

youtube-dl -f bestvideo+bestaudio --verbose https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/quotidien-deuxieme-partie-du-17-mai-2022-78671103.html
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-f', u'bestvideo+bestaudio', u'--verbose', u'https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/quotidien-deuxieme-partie-du-17-mai-2022-78671103.html']
[debug] Encodings: locale UTF-8, fs utf-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Python version 2.7.17 (CPython) - Darwin-19.6.0-x86_64-i386-64bit
[debug] exe versions: ffmpeg 5.0.1, ffprobe 5.0.1, rtmpdump 2.4
[debug] Proxy map: {}
[TF1] quotidien-deuxieme-partie-du-17-mai-2022-78671103: Downloading JSON metadata
[wat.tv] 13880912: Downloading JSON metadata
[wat.tv] 13880912: Downloading MPD manifest
WARNING: Failed to download MPD manifest: HTTP Error 404: Not Found
[wat.tv] 13880912: Downloading JSON metadata
WARNING: Unable to download JSON metadata: HTTP Error 403: Unauthorized access.
ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
Traceback (most recent call last):
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 815, in wrapper
    return func(self, *args, **kwargs)
  File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 836, in __extract_info
    ie_result = ie.extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 534, in extract
    ie_result = self._real_extract(url)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/wat.py", line 96, in _real_extract
    self._sort_formats(formats)
  File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 1374, in _sort_formats
    raise ExtractorError('No video formats found')
ExtractorError: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

dirkf commented 2 years ago

Reproduces for me.

dirkf commented 2 years ago

The TF1 extractor delegates to the WAT extractor.

Let's go to wat.tv. 410 Gone and this text:

La plateforme WAT telle que vous la connaissez n'existe plus, mais nombre de vos vidéos sont désormais disponibles sur MYTF1 / XTRA.

XTRA, c'est le nouveau Label de TF1, exclusivement dédié aux contenus que vous ne trouverez pas sur les chaines du Groupe TF1 : Créations originales, webséries, séries vintage !

Sur XTRA, les vidéos sont disponibles gratuitement et notre équipe éditoriale vous a sélectionné ce qui se fait de mieux actuellement sur le web !

Les contenus sont disponibles partout, sur le web (www.mytf1.fr/xtra), sur votre mobile/tablette via www.mytf1.fr/xtra ou l'APP MYTF1 (iOS et Android) et même sur votre télé (via le Replay MYTF1 sur la Freebox, Bbox et SFRbox).

Voilà. Someone needs to rework the TF1 extractor, ideally from its own region.

dirkf commented 2 years ago

Although the extractor can be re-worked, all that's needed is to add the pver parameter to the metadata API call (see streamlink), as in query={'context': 'MYTF1', 'pver': '4001000'}.

adrienverge commented 2 years ago

Thanks @dirkf, I confirm this works for me.

horsentiers commented 2 years ago

Although the extractor can be re-worked, all that's needed is to add the pver parameter to the metadata API call (see streamlink), as in query={'context': 'MYTF1', 'pver': '4001000'}.

Hello @dirkf , In what file can we find the parameyter to the metadata API ? Thanks

horsentiers commented 2 years ago

Although the extractor can be re-worked, all that's needed is to add the pver parameter to the metadata API call (see streamlink), as in query={'context': 'MYTF1', 'pver': '4001000'}.

Hello @dirkf , In what file can we find the parameyter to the metadata API ? Thanks

Ok I see it should be common.py in the extractor. Apologize to not have read the log.

horsentiers commented 2 years ago

Although the extractor can be re-worked, all that's needed is to add the pver parameter to the metadata API call (see streamlink), as in query={'context': 'MYTF1', 'pver': '4001000'}.

Hello @dirkf , In what file can we find the parameyter to the metadata API ? Thanks

Ok I see it should be common.py in the extractor. Apologize to not have read the log.

Finaly, I don't find the file........

dirkf commented 2 years ago

youtube_dl/extractor/wat.py l.60.

horsentiers commented 2 years ago

youtube_dl/extractor/wat.py l.60.

Thanks a lot.

horsentiers commented 2 years ago

youtube_dl/extractor/wat.py l.60.

I tried and the result is not very good. Here's the log:

youtube-dl --verbose https://www.tf1.fr/tf1/koh-lanta/videos/koh-lanta-le-totem-maudit-du-24-mai-2022-episode-13-partie-1-92531236.html [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: ['--verbose', 'https://www.tf1.fr/tf1/koh-lanta/videos/koh-lanta-le-totem-maudit-du-24-mai-2022-episode-13-partie-1-92531236.html'] [debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8 [debug] youtube-dl version 2021.12.17 [debug] Python version 3.10.4 (CPython) - Linux-5.17.0-2-amd64-x86_64-with-glibc2.33 [debug] exe versions: ffmpeg 4.4.2, ffprobe 4.4.2, rtmpdump 2.4 [debug] Proxy map: {} [TF1] koh-lanta-le-totem-maudit-du-24-mai-2022-episode-13-partie-1-92531236: Downloading JSON metadata [wat.tv] 13883258: Downloading JSON metadata [wat.tv] 13883258: Downloading MPD manifest WARNING: Failed to download MPD manifest: HTTP Error 404: Not Found [wat.tv] 13883258: Downloading JSON metadata WARNING: Unable to download JSON metadata: HTTP Error 403: Unauthorized access. ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. Traceback (most recent call last): File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 815, in wrapper return func(self, *args, **kwargs) File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 836, in __extract_info ie_result = ie.extract(url) File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 534, in extract ie_result = self._real_extract(url) File "/usr/local/bin/youtube-dl/youtube_dl/extractor/wat.py", line 96, in _real_extract self._sort_formats(formats) File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 1374, in _sort_formats raise ExtractorError('No video formats found') youtube_dl.utils.ExtractorError: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

dirkf commented 2 years ago

Please test with OP's problem URL https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/quotidien-premiere-partie-du-17-mai-2022-71087892.html (use options -v -F).

The koh-lanta URL is geo-blocked in UK:

$ python -m youtube_dl -v -F 'https://www.tf1.fr/tf1/koh-lanta/videos/koh-lanta-le-totem-maudit-du-24-mai-2022-episode-13-partie-1-92531236.html'
[debug] System config: [u'--prefer-ffmpeg']
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'-F', u'https://www.tf1.fr/tf1/koh-lanta/videos/koh-lanta-le-totem-maudit-du-24-mai-2022-episode-13-partie-1-92531236.html']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Git HEAD: 187a48aee
[debug] Python version 2.7.17 (CPython) - Linux-4.4.0-210-generic-i686-with-Ubuntu-16.04-xenial
[debug] exe versions: avconv 4.3, avprobe 4.3, ffmpeg 4.3, ffprobe 4.3
[debug] Proxy map: {}
[TF1] koh-lanta-le-totem-maudit-du-24-mai-2022-episode-13-partie-1-92531236: Downloading JSON metadata
[wat.tv] 13883258: Downloading JSON metadata
ERROR: Ce contenu n'est pas disponible dans votre zone géographique
This video is available in France, Guadeloupe, French Guiana, Réunion, Martinique, Mayotte, New Caledonia, French Southern Territories, Wallis and Futuna, Saint Pierre and Miquelon, Saint Barthélemy, Saint Martin (French part), French Polynesia, Monaco, Andorra, Belgium, Switzerland.
You might want to use a VPN or a proxy server (with --proxy) to workaround.
Traceback (most recent call last):
  File "youtube_dl/YoutubeDL.py", line 815, in wrapper
    return func(self, *args, **kwargs)
  File "youtube_dl/YoutubeDL.py", line 836, in __extract_info
    ie_result = ie.extract(url)
  File "youtube_dl/extractor/common.py", line 534, in extract
    ie_result = self._real_extract(url)
  File "youtube_dl/extractor/wat.py", line 66, in _real_extract
    self.raise_geo_restricted(error_desc, video_info.get('geoList'))
  File "youtube_dl/extractor/common.py", line 947, in raise_geo_restricted
    raise GeoRestrictedError(msg, countries=countries)
GeoRestrictedError: Ce contenu n'est pas disponible dans votre zone g\xe9ographique

$

This happens equally with my branch that removes the wat.tv extractor, because both use the same API mediainfo.tf1.fr/mediainfocombo/ to get the programme metadata.

horsentiers commented 2 years ago

Please test with OP's problem URL https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/quotidien-premiere-partie-du-17-mai-2022-71087892.html (use options -v -F).

The koh-lanta URL is geo-blocked in UK:

$ python -m youtube_dl -v -F 'https://www.tf1.fr/tf1/koh-lanta/videos/koh-lanta-le-totem-maudit-du-24-mai-2022-episode-13-partie-1-92531236.html'
[debug] System config: [u'--prefer-ffmpeg']
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'-v', u'-F', u'https://www.tf1.fr/tf1/koh-lanta/videos/koh-lanta-le-totem-maudit-du-24-mai-2022-episode-13-partie-1-92531236.html']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2021.12.17
[debug] Git HEAD: 187a48aee
[debug] Python version 2.7.17 (CPython) - Linux-4.4.0-210-generic-i686-with-Ubuntu-16.04-xenial
[debug] exe versions: avconv 4.3, avprobe 4.3, ffmpeg 4.3, ffprobe 4.3
[debug] Proxy map: {}
[TF1] koh-lanta-le-totem-maudit-du-24-mai-2022-episode-13-partie-1-92531236: Downloading JSON metadata
[wat.tv] 13883258: Downloading JSON metadata
ERROR: Ce contenu n'est pas disponible dans votre zone géographique
This video is available in France, Guadeloupe, French Guiana, Réunion, Martinique, Mayotte, New Caledonia, French Southern Territories, Wallis and Futuna, Saint Pierre and Miquelon, Saint Barthélemy, Saint Martin (French part), French Polynesia, Monaco, Andorra, Belgium, Switzerland.
You might want to use a VPN or a proxy server (with --proxy) to workaround.
Traceback (most recent call last):
  File "youtube_dl/YoutubeDL.py", line 815, in wrapper
    return func(self, *args, **kwargs)
  File "youtube_dl/YoutubeDL.py", line 836, in __extract_info
    ie_result = ie.extract(url)
  File "youtube_dl/extractor/common.py", line 534, in extract
    ie_result = self._real_extract(url)
  File "youtube_dl/extractor/wat.py", line 66, in _real_extract
    self.raise_geo_restricted(error_desc, video_info.get('geoList'))
  File "youtube_dl/extractor/common.py", line 947, in raise_geo_restricted
    raise GeoRestrictedError(msg, countries=countries)
GeoRestrictedError: Ce contenu n'est pas disponible dans votre zone g\xe9ographique

$

This happens equally with my branch that removes the wat.tv extractor, because both use the same API mediainfo.tf1.fr/mediainfocombo/ to get the programme metadata.

Here's what I have (I'm in France) : yt https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/quotidien-premiere-partie-du-17-mai-2022-71087892.html [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: ['--geo-bypass', '-v', '-F', 'https://www.tf1.fr/tmc/quotidien-avec-yann-barthes/videos/quotidien-premiere-partie-du-17-mai-2022-71087892.html'] [debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8 [debug] youtube-dl version 2021.12.17 [debug] Python version 3.10.4 (CPython) - Linux-5.17.0-3-amd64-x86_64-with-glibc2.33 [debug] exe versions: ffmpeg 4.4.2, ffprobe 4.4.2, rtmpdump 2.4 [debug] Proxy map: {} [TF1] quotidien-premiere-partie-du-17-mai-2022-71087892: Downloading JSON metadata [wat.tv] 13880876: Downloading JSON metadata [wat.tv] 13880876: Downloading MPD manifest WARNING: Failed to download MPD manifest: HTTP Error 404: Not Found [wat.tv] 13880876: Downloading JSON metadata WARNING: Unable to download JSON metadata: HTTP Error 403: Unauthorized access. ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. Traceback (most recent call last): File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 815, in wrapper return func(self, *args, **kwargs) File "/usr/local/bin/youtube-dl/youtube_dl/YoutubeDL.py", line 836, in __extract_info ie_result = ie.extract(url) File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 534, in extract ie_result = self._real_extract(url) File "/usr/local/bin/youtube-dl/youtube_dl/extractor/wat.py", line 96, in _real_extract self._sort_formats(formats) File "/usr/local/bin/youtube-dl/youtube_dl/extractor/common.py", line 1374, in _sort_formats raise ExtractorError('No video formats found') youtube_dl.utils.ExtractorError: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.

I don't understand....

dirkf commented 2 years ago

Me neither. I would say that you are not running the modified code, as your log looks like OP's original and mine without the patch, whereas I get 2 audio and 8 video formats with the patch.

The patch should have been in /usr/local/bin/youtube-dl/youtube_dl/extractor/wat.py. Has it definitely got the change from https://github.com/ytdl-org/youtube-dl/issues/30959#issuecomment-1131996260?

If you can get that to work, the corresponding log for the 'koh-lanta' page would be very helpful.

You shouldn't need --geo-bypass in FR.

dirkf commented 2 years ago

Also, try 'pver': '4003000' instead, as recently committed in yt-dlp.

horsentiers commented 2 years ago

Also, try 'pver': '4003000' instead, as recently committed in yt-dlp.

Indeed, it works ! What a surprise ! Thanks a lot for all these information !!

MartinDelille commented 2 years ago

Will the fix be included in the next release ?

ytdl-org / youtube-dl