search

ytdl-org / youtube-dl

Command-line program to download videos from YouTube.com and other video sites
http://ytdl-org.github.io/youtube-dl/
The Unlicense
132.3k stars 10.03k forks source link

【FranceTV】Download no longer possible #29956

Open NiesmialyGosc opened 3 years ago

NiesmialyGosc commented 3 years ago

Checklist

Verbose log

C:\Users\DELL\video>youtube-dl -v https://www.france.tv/france-3/plus-belle-la-vie/plus-belle-la-vie-saison-18/2739781-episode-du-vendredi-10-septembre-2021.html
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-v', 'https://www.france.tv/france-3/plus-belle-la-vie/plus-belle-la-vie-saison-18/2739781-episode-du-vendredi-10-septembre-2021.html']
[debug] Encodings: locale cp936, fs mbcs, out cp936, pref cp936
[debug] youtube-dl version 2021.03.14
[debug] Python version 3.4.4 (CPython) - Windows-10-10.0.18362
[debug] exe versions: ffmpeg N-89508-g1f1207145a, ffprobe N-89508-g1f1207145a
[debug] Proxy map: {'http': 'http://127.0.0.1:52528', 'socks': 'socks://127.0.0.1:52527', 'https': 'https://127.0.0.1:52528'}
[FranceTVSite] 2739781-episode-du-vendredi-10-septembre-2021: Downloading webpage
[FranceTV] b44f8a0a-e316-4735-9e1f-0fdf9839e2e2: Downloading video JSON
ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found (caused by HTTPError()); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type  youtube-dl -U  to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmp8z3w9mz6\build\youtube_dl\extractor\common.py", line 634, in _request_webpage
  File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\ytdl-org\tmp8z3w9mz6\build\youtube_dl\YoutubeDL.py", line 2279, in urlopen
  File "C:\Python\Python34\lib\urllib\request.py", line 470, in open
  File "C:\Python\Python34\lib\urllib\request.py", line 580, in http_response
  File "C:\Python\Python34\lib\urllib\request.py", line 508, in error
  File "C:\Python\Python34\lib\urllib\request.py", line 442, in _call_chain
  File "C:\Python\Python34\lib\urllib\request.py", line 588, in http_error_default

C:\Users\DELL\video>

Description

Hi team,

I can't seem to download any more video from FRANCE.TV, here are some examples:

https://www.france.tv/france-3/plus-belle-la-vie/plus-belle-la-vie-saison-18/2743611-episode-du-lundi-13-septembre-2021.html https://www.france.tv/france-2/amour-gloire-et-beaute/2743793-emission-du-lundi-13-septembre-2021.html https://www.france.tv/france-5/la-quotidienne/la-quotidienne-saison-9/2743883-emission-du-lundi-13-septembre-2021.html

I assure you it has nothing to do with any of my VPNs for I have verified these videos play like a charm in the browser with my VPNs turned on.

Could you please look into this FRANCE.TV issue on priority and help to publish the latest version ASAP?

Many Thanks!! NiesmialyGosc

ruleps commented 3 years ago

Même problème.

lcheylus commented 3 years ago

I confirm this issue with FranceTV site. Specific code for extractor is in youtube_dl/extractor/francetv.py

To download JSON metadata, extractor requests https://sivideo.webservices.francetelevisions.fr/tools/getInfosOeuvre/v2/ with video ID (line 94). But it seems that this API is deprecated :(

With datas used for tests in FranceTV extractor (line 51) : request for https://sivideo.webservices.francetelevisions.fr/tools/getInfosOeuvre/v2/?idDiffusion=162311093 => JSON message {"status":"NOK","code":404,"message":"This api is deprecated, contact France.tv video-factory"}

After a few debug, I didn't find the new URL/service in my HTTP logs to get metadatas via API.

LateLament commented 3 years ago

For this video : https://www.france.tv/documentaires/histoire/2743607-la-guillotine-une-invention-bien-francaise.html

the new API URL seems to be : https://player.webservices.francetelevisions.fr/v1/videos/ff7f5414-b1f7-4912-b80f-389094734b7b?country_code=FR&w=720&h=405&player_version=5.51.35&domain=www.france.tv&device_type=desktop&browser=chrome&browser_version=93&os=linux&gmt=+0200&video_product_id=2743607

sarnoud commented 3 years ago

Here is quick early patch I wrote that seems to repair quite a few links. youtube-dl-patch.txt

NiesmialyGosc commented 3 years ago

Here is quick early patch I wrote that seems to repair quite a few links. youtube-dl-patch.txt

Thank you @sarnoud ! So is there an on-the-ground solution for end users with little or no programming expertise to apply this patch? If yes could you provide some guidance on the steps? Thank you.

Warm regards, NiesmialyGosc

Philippe-Arnaud commented 3 years ago

Here is quick early patch I wrote that seems to repair quite a few links. youtube-dl-patch.txt

Thank you @sarnoud ! So is there an on-the-ground solution for end users with little or no programming expertise to apply this patch? If yes could you provide some guidance on the steps? Thank you.

Warm regards, NiesmialyGosc

In your Python directory (i.e. /usr/lib/python3/dist-packages for Linux) enter the following command:

patch youtube_dl/extractor/francetv.py < youtube-dl-patch.txt

Et voilà :-)

NiesmialyGosc commented 3 years ago

Here is quick early patch I wrote that seems to repair quite a few links. youtube-dl-patch.txt

Thank you @sarnoud ! So is there an on-the-ground solution for end users with little or no programming expertise to apply this patch? If yes could you provide some guidance on the steps? Thank you. Warm regards, NiesmialyGosc

In your Python directory (i.e. /usr/lib/python3/dist-packages for Linux) enter the following command:

patch youtube_dl/extractor/francetv.py < youtube-dl-patch.txt

Et voilà :-)

Thanks, so what if I'm using Windows 10 and not Linux?

(Plus I see 2 directories on my machine with Win10: C:\Users\DELL\AppData\Local\Programs\Python\Python39\Scripts and C:\Users\DELL\AppData\Roaming\Python\Python39\Scripts Not sure which one to use.. If that matters pls let me know. Hereinafter I've used the first directory)

C:\Users\DELL\AppData\Local\Programs\Python\Python39\Scripts>python
Python 3.9.6 (tags/v3.9.6:db3ff76, Jun 28 2021, 15:26:21) [MSC v.1929 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> patch youtube_dl/extractor/francetv.py < youtube-dl-patch.txt
  File "<stdin>", line 1
    patch youtube_dl/extractor/francetv.py < youtube-dl-patch.txt
          ^
SyntaxError: invalid syntax
>>>

Finally I've had to resort to busybox.exe to run Linux commands on my Win10 OS, but that wouldn't bring the physical directory /usr/lib/python3/dist-packages in place. Should I place the .txt patch file under the directory "C:\Users\DELL\AppData\Local\Programs\Python\Python39\lib\site-packages" as shown below? and what's next? Not sure what would be the correct way to proceed, sorry for my ignorance of Git.

~/video/TEST1 $ python -m site
sys.path = [
    'C:\\Users\\DELL\\video\\TEST1',
    'C:\\Users\\DELL\\AppData\\Local\\Programs\\Python\\Python39\\python39.zip',
    'C:\\Users\\DELL\\AppData\\Local\\Programs\\Python\\Python39\\DLLs',
    'C:\\Users\\DELL\\AppData\\Local\\Programs\\Python\\Python39\\lib',
    'C:\\Users\\DELL\\AppData\\Local\\Programs\\Python\\Python39',
    'C:\\Users\\DELL\\AppData\\Roaming\\Python\\Python39\\site-packages',
    'C:\\Users\\DELL\\AppData\\Local\\Programs\\Python\\Python39\\lib\\site-packages',
]
USER_BASE: 'C:/Users/DELL/AppData/Roaming\\Python' (exists)
USER_SITE: 'C:/Users/DELL/AppData/Roaming\\Python\\Python39\\site-packages' (exists)
ENABLE_USER_SITE: True
~/video/TEST1 $

Any guidance/instructions will be much appreciated.

Thanks again, NiesmialyGosc

guy-teube commented 3 years ago

Here is quick early patch I wrote that seems to repair quite a few links. youtube-dl-patch.txt

Thank you @sarnoud ! So is there an on-the-ground solution for end users with little or no programming expertise to apply this patch? If yes could you provide some guidance on the steps? Thank you. Warm regards, NiesmialyGosc

In your Python directory (i.e. /usr/lib/python3/dist-packages for Linux) enter the following command:

patch youtube_dl/extractor/francetv.py < youtube-dl-patch.txt

Et voilà :-)

Hello

I'm an Ubuntu 18.04 user. I have no youtube_dl directory in my /usr/lib/python3 directory (even in python3.6/3.7/3.8). a whereis youtube-dl says : /usr/local/bin/youtube-dl I have searched a file name francetv.py in my whole /usr directory, nothing found. Do you know how (and where) run your patch script? Thank you

lcheylus commented 3 years ago

I'm an Ubuntu 18.04 user. I have no youtube_dl directory in my /usr/lib/python3 directory (even in python3.6/3.7/3.8). a whereis youtube-dl says : /usr/local/bin/youtube-dl I have searched a file name francetv.py in my whole /usr directory, nothing found. Do you know how (and where) run your patch script?

Hello @guy-teube , your local youtube-dl installation should be in /usr/local/lib/python3.9/dist-packages/youtube_dl-...(modify python3.9 path if you have another version).

The file to patch is extractor/francetv.py

lcheylus commented 3 years ago

Hello @NiesmialyGosc, below I upload the complete francetv.py file patched with @sarnoud fix. Suppress .txt extension and replace extractor/francetv.py in your youtube-dl local installation by this modified version.

Hope that helps.

francetv.py.txt

NiesmialyGosc commented 3 years ago

Hello @NiesmialyGosc, below I upload the complete francetv.py file patched with @sarnoud fix. Suppress .txt extension and replace extractor/francetv.py in your youtube-dl local installation by this modified version.

Hope that helps.

francetv.py.txt

Sorry I don't understand. I'm not an professional IT guy and I can't understand some technical terms such as "patched with" & "youtube-dl local installation".

Is youtube-dl intended for users with IT development knowledge (e.g. python) only?

Could anyone please help to translate the above into an on-the-ground language without me as a normal user that wishes to watch online videos having to learn python grammar & vocabulary from scratch?

Thank you for your patience & time, NiesmialyGosc

D84web commented 3 years ago

Hello @NiesmialyGosc, below I upload the complete francetv.py file patched with @sarnoud fix. Suppress .txt extension and replace extractor/francetv.py in your youtube-dl local installation by this modified version. Hope that helps. francetv.py.txt

Sorry I don't understand. I'm not an professional IT guy and I can't understand some technical terms such as "patched with" & "youtube-dl local installation".

Is youtube-dl intended for users with IT development knowledge (e.g. python) only?

Could anyone please help to translate the above into an on-the-ground language without me as a normal user that wishes to watch online videos having to learn python grammar & vocabulary from scratch?

Thank you for your patience & time, NiesmialyGosc

@NiesmialyGosc, are you on a Windows or a linux Machine ? [EDIT] : sorry, I read again your first post. You are on a Windows machine... Maybe, you can search a file on Windows which name is "francetv.py". :-)

[En français, au cas où : recherche un fichier qui s'appelle "francetv.py" sur ton PC et remplace-le par le patch en supprimant au préalable l'extension *.txt du patch] :-)

On a linux terminal, I wrote this :

sudo thunar /usr/lib/python3/dist-packages/youtube_dl/extractor/

and I changed the file.

Unfortunately, this patch doesnot works (on my machine) : it downloads only a video file ; actually, there is neither sound nor subtitle.

I have the same problem with Youtube-DLG.

[En français, comme précisé ci-dessus en anglais, ce patch ne me récupère que le flux video - pas le flux audio ni le sous-titrage - je suis sous Ubuntu 20.04]

LateLament commented 3 years ago

Patch works. Thanks.

Need to download video and audio separately :

youtube-dl -i -f "bestvideo,bestaudio" https://www.france.tv/france-3/carnets-de-vol/2783631-emission-du-samedi-11-septembre-2021.html

Merging video and audio :

ffmpeg -i Carnets\ de\ vol-a386d839-8337-4975-bc98-767299aafb82.mp4 -i Carnets\ de\ vol-a386d839-8337-4975-bc98-767299aafb82.m4a -c copy -map 0:0 -map 1:0 Carnets\ de\ vol.mp4

D84web commented 3 years ago

Patch works. Thanks.

Need to download video and audio separately :

youtube-dl -i -f "bestvideo,bestaudio" https://www.france.tv/france-3/carnets-de-vol/2783631-emission-du-samedi-11-septembre-2021.html

Merging video and audio :

ffmpeg -i Carnets\ de\ vol-a386d839-8337-4975-bc98-767299aafb82.mp4 -i Carnets\ de\ vol-a386d839-8337-4975-bc98-767299aafb82.m4a -c copy -map 0:0 -map 1:0 Carnets\ de\ vol.mp4

Yes, it is ! I works. Unfortunately, no subtitle !

youtube-dl --list-subs https://www.france.tv/france-5/c-jamy/c-jamy-saison-2/2733221-emission-du-mercredi-8-septembre-2021.html [FranceTVSite] 2733221-emission-du-mercredi-8-septembre-2021: Downloading webpage [FranceTV] b349b7fc-c876-4dab-a013-5634ea146866: Downloading video JSON [FranceTV] b349b7fc-c876-4dab-a013-5634ea146866: Downloading signed dash manifest URL [FranceTV] b349b7fc-c876-4dab-a013-5634ea146866: Downloading MPD manifest WARNING: [FranceTV] Unknown MIME type application/mp4 in DASH manifest b349b7fc-c876-4dab-a013-5634ea146866 has no subtitles

lcheylus commented 3 years ago

Unfortunately, this patch doesnot works (on my machine) : it downloads only a video file ; actually, there is neither sound nor subtitle.

Yes, FranceTV have separate streams for audio and video. If you have ffmpeg installed, by default youtube-dl downloads bestvideo then bestaudio format separately, and merges them with ffmepg. No need to specify bestvideo+bestaudio format in parameters list.

man youtube-dl / FORMAT SELECTION
Since the end of April 2015 and version 2015.04.26, youtube-dl uses -f bestvideo+bestaudio/best as the default  format
selection  (see #5447 (https://github.com/ytdl-org/youtube-dl/issues/5447), #5456 (https://github.com/ytdl-org/youtube-dl/issues/5456)).
If ffmpeg or avconv are installed this results in downloading bestvideo and bestaudio separately  and  muxing them together
into a single file giving the best overall quality available.

I confirm that with @sarnoud patch, subtitles are not supported.

guy-teube commented 3 years ago

Hello @guy-teube , your local youtube-dl installation should be in /usr/local/lib/python3.9/dist-packages/youtube_dl-...(modify python3.9 path if you have another version).

The file to patch is extractor/francetv.py

Ok thank you. Sadly I have nothing like this in /usr/local/lib/python3.6/dist-packages. To install youtube-dl, I have followed the official doc: http://ytdl-org.github.io/youtube-dl/download.html, which says:

sudo curl -L https://yt-dl.org/downloads/latest/youtube-dl -o /usr/local/bin/youtube-dl
sudo chmod a+rx /usr/local/bin/youtube-dl
NiesmialyGosc commented 3 years ago

@D84web @LateLament

Thank you both team for the support! This is greatly appreciated & your tips really useful for future reference. THANK U. Can we have the subtitles downloaded as well?

I actually had to resort to "Lj video downloader" for the video download, and the only & key issue is actually with the subtitles for which I had to use the yt-dlp downloader:

https://www.france.tv/france-3/plus-belle-la-vie/plus-belle-la-vie-saison-18/2743347-episode-du-mardi-14-septembre-2021.html

yt-dlp --write-subs --skip-download --external-downloader aria2c --allow-u "https://cloudreplay.ftven.fr/6f1d1b301a316/1036062661_france-domtom_TA.ism/ZXhwPTE2MzE2MzcyMDJ+YWNsPSUyZjZmMWQxYjMwMWEzMTYlMmYxMDM2MDYyNjYxX2ZyYW5jZS1kb210b21fVEEuaXNtKn5obWFjPTg5OGFhNGFhODVkYWZjMmQ3N2MyOTg5M2M2MmMyOTYyOTRlNGE5MzM2NzE2ODhlNjMxOTA5MDc3MTgyMWUyZTA=/manifest.mpd" --output 4367.webvtt

However, the generated subtitle file is in a mess which looks like this:

            trun        p  '  € @  mdat<?xml version="1.0" encoding="utf-8"?><tt xmlns="http://www.w3.org/ns/ttml" xmlns:ttm="http://www.w3.org/ns/ttml#metadata" xmlns:ttp="http://www.w3.org/ns/ttml#parameter" xmlns:tts="http://www.w3.org/ns/ttml#styling" xml:lang="fr"><head><metadata><ttm:title></ttm:title><ttm:desc></ttm:desc><ttm:copyright></ttm:copyright></metadata><styling><style xml:id="captionStyle" tts:backgroundColor="transparent" tts:displayAlign="center" tts:extent="100% 20%" tts:fontFamily="proportionalSansSerif" tts:fontSize="30px" tts:origin="0% 75%" tts:textAlign="center" tts:textOutline="black 1px 0px" /></styling><layout><region style="captionStyle" xml:id="region2" /></layout></head><body><div region="region2"><p begin="00:00:00.000" end="00:00:02.233" region="region2" xml:id="caption1" ttm:role="caption">
        <span tts:textAlign="center" tts:color="white">-L&apos;animal n&apos;est pas commode.</span>
      </p><p begin="00:00:02.500" end="00:00:04.166" region="region2" xml:id="caption2" ttm:role="caption">
        <span tts:textAlign="center" tts:color="yellow">-Je vais le dresser.</span>

As a result, I have to remove those tags & recode the encoding which would take a lot of work.. Is there a better way to generate the subtitles than using the yt-dlp tool??? Or could any of you provide some tips on how to mass download, merge & reformat the webvtt files from France.TV, or a quick fix to remove the tags above? (If possible & if you could elaborate each step in detail, that'll be great)

Thanks & Regards NiesmialyGosc

LateLament commented 3 years ago

@guy-teube Try : locate "francetv.py" or find . -name "francetv.py"

For me (Arch Linux) :

/usr/lib/python3.7/site-packages/youtube_dl/extractor/francetv.py.0 /usr/lib/python3.7/site-packages/youtube_dl/extractor/francetv.py.1 /usr/lib/python3.9/site-packages/youtube_dl/extractor/francetv.py

NiesmialyGosc commented 3 years ago

and I have to remove those tags which would take a lot of work..

I can't seem to edit the srt file generated with the aforementioned method by using Subtitle Edit 3.6.2 for I see the start time & end time of each frame are distorted in Subtitle Edit.. Any thoughts?

Toto2u commented 3 years ago

@NiesmialyGosc

To get subtitles for:

https://www.france.tv/france-3/plus-belle-la-vie/plus-belle-la-vie-saison-18/2743347-episode-du-mardi-14-septembre-2021.html

The manifest.mpd is:

https://cloudreplay.ftven.fr/6f1d1b301a316/1036062661_france-domtom_TA.ism/ZXhwPTE2MzE2NTQ3MDh+YWNsPSUyZjZmMWQxYjMwMWEzMTYlMmYxMDM2MDYyNjYxX2ZyYW5jZS1kb210b21fVEEuaXNtKn5obWFjPTNlNjU1N2QxNTYwYWEwZjQ1NjcyYTIwMmRmYTgyYWRkYjg5ZDU5YThmYmMwODEwNmI5Nzc2NGViNDZjNTg1NzE=/manifest.mpd

In Firefox (any browser) replace manifest.mpd with textstream_fre=1000.webvtt and save file as vtt

https://cloudreplay.ftven.fr/6f1d1b301a316/1036062661_france-domtom_TA.ism/ZXhwPTE2MzE2NTQ3MDh+YWNsPSUyZjZmMWQxYjMwMWEzMTYlMmYxMDM2MDYyNjYxX2ZyYW5jZS1kb210b21fVEEuaXNtKn5obWFjPTNlNjU1N2QxNTYwYWEwZjQ1NjcyYTIwMmRmYTgyYWRkYjg5ZDU5YThmYmMwODEwNmI5Nzc2NGViNDZjNTg1NzE=/textstream_fre=1000.webvtt

NiesmialyGosc commented 3 years ago

@NiesmialyGosc

To get subtitles for:

https://www.france.tv/france-3/plus-belle-la-vie/plus-belle-la-vie-saison-18/2743347-episode-du-mardi-14-septembre-2021.html

The manifest.mpd is:

https://cloudreplay.ftven.fr/6f1d1b301a316/1036062661_france-domtom_TA.ism/ZXhwPTE2MzE2NTQ3MDh+YWNsPSUyZjZmMWQxYjMwMWEzMTYlMmYxMDM2MDYyNjYxX2ZyYW5jZS1kb210b21fVEEuaXNtKn5obWFjPTNlNjU1N2QxNTYwYWEwZjQ1NjcyYTIwMmRmYTgyYWRkYjg5ZDU5YThmYmMwODEwNmI5Nzc2NGViNDZjNTg1NzE=/manifest.mpd

In Firefox replace manifest.mpd with textstream_fre=1000.webvtt and save file as vtt

https://cloudreplay.ftven.fr/6f1d1b301a316/1036062661_france-domtom_TA.ism/ZXhwPTE2MzE2NTQ3MDh+YWNsPSUyZjZmMWQxYjMwMWEzMTYlMmYxMDM2MDYyNjYxX2ZyYW5jZS1kb210b21fVEEuaXNtKn5obWFjPTNlNjU1N2QxNTYwYWEwZjQ1NjcyYTIwMmRmYTgyYWRkYjg5ZDU5YThmYmMwODEwNmI5Nzc2NGViNDZjNTg1NzE=/textstream_fre=1000.webvtt

Works like a charm in my Chrome too, THANKS @Toto2u :) !! This is cool.

And THANKS to the whole team for your continuous support and time in getting this resolved (I'm really looking forward to the next release of youtube-dl.exe soon though, lol) !!

guy-teube commented 3 years ago

@guy-teube Try : locate "francetv.py" or find . -name "francetv.py"

For me (Arch Linux) :

/usr/lib/python3.7/site-packages/youtube_dl/extractor/francetv.py.0 /usr/lib/python3.7/site-packages/youtube_dl/extractor/francetv.py.1 /usr/lib/python3.9/site-packages/youtube_dl/extractor/francetv.py

I have searched in / and found it only in my backup directory, in /usr/lib/python3/dist-packages/youtube_dl/, in 2019-10-20. I haven't this directory since this date, si youtube-dl is working without it since this day.

D84web commented 3 years ago

@NiesmialyGosc

To get subtitles for:

https://www.france.tv/france-3/plus-belle-la-vie/plus-belle-la-vie-saison-18/2743347-episode-du-mardi-14-septembre-2021.html

The manifest.mpd is:

https://cloudreplay.ftven.fr/6f1d1b301a316/1036062661_france-domtom_TA.ism/ZXhwPTE2MzE2NTQ3MDh+YWNsPSUyZjZmMWQxYjMwMWEzMTYlMmYxMDM2MDYyNjYxX2ZyYW5jZS1kb210b21fVEEuaXNtKn5obWFjPTNlNjU1N2QxNTYwYWEwZjQ1NjcyYTIwMmRmYTgyYWRkYjg5ZDU5YThmYmMwODEwNmI5Nzc2NGViNDZjNTg1NzE=/manifest.mpd

In Firefox (any browser) replace manifest.mpd with textstream_fre=1000.webvtt and save file as vtt

https://cloudreplay.ftven.fr/6f1d1b301a316/1036062661_france-domtom_TA.ism/ZXhwPTE2MzE2NTQ3MDh+YWNsPSUyZjZmMWQxYjMwMWEzMTYlMmYxMDM2MDYyNjYxX2ZyYW5jZS1kb210b21fVEEuaXNtKn5obWFjPTNlNjU1N2QxNTYwYWEwZjQ1NjcyYTIwMmRmYTgyYWRkYjg5ZDU5YThmYmMwODEwNmI5Nzc2NGViNDZjNTg1NzE=/textstream_fre=1000.webvtt

@toto2u, how did you get the manifest.mpd, please ?

guy-teube commented 3 years ago

@D84web Open the console (F12) then Network tab. When the video is launched, make a search "manifest" (under this Network tab), and you will get this url.

Trit34 commented 3 years ago

Not totally fixed yet, because the patch only retrieves MPD manifests, but the videos were also available as M3U8 playlists (with audio and video combined in one MP4 file), and some of them (like Murdoch Mysteries/Les enquêtes de Murdoch) are only available as M3U8 playlists, and this early patch is not working with these videos (“Error: No video formats found”). Example link: https://www.france.tv/france-3/les-enquetes-de-murdoch/les-enquetes-de-murdoch-saison-14/2743953-rira-bien-qui-rira-le-dernier.html

Hopefully, a new version of the patch will be made and will fix that, too.

LateLament commented 3 years ago

Not totally fixed yet, because the patch only retrieves MPD manifests, but the videos were also available as M3U8 playlists (with audio and video combined in one MP4 file), and some of them (like Murdoch Mysteries/Les enquêtes de Murdoch) are only available as M3U8 playlists, and this early patch is not working with these videos (“Error: No video formats found”).

Hopefully, a new version of the patch will be made and will fix that, too.

Some protected videos ?

For this link : https://www.france.tv/france-3/les-enquetes-de-murdoch/saison-2/2743931-l-eventreur-de-toronto.html

https://player.webservices.francetelevisions.fr/v1/videos/671287b1-794f-4e8e-b95a-3ab3dc823466?country_code=FR&w=720&h=405&player_version=5.51.35&domain=www.france.tv&device_type=desktop&browser=chrome&browser_version=93&os=linux&diffusion_mode=tunnel_first&gmt=%2B0200&video_product_id=2743931

contains : "drm":true,"drm_type":"widevine"

Trit34 commented 3 years ago

Some protected videos ?

For this link : https://www.france.tv/france-3/les-enquetes-de-murdoch/saison-2/2743931-l-eventreur-de-toronto.html

https://player.webservices.francetelevisions.fr/v1/videos/671287b1-794f-4e8e-b95a-3ab3dc823466?country_code=FR&w=720&h=405&player_version=5.51.35&domain=www.france.tv&device_type=desktop&browser=chrome&browser_version=93&os=linux&diffusion_mode=tunnel_first&gmt=%2B0200&video_product_id=2743931

contains : "drm":true,"drm_type":"widevine"

AFAIK, these videos were dowloadable with yt-dl until yesterday, DRM or not. I could download S14E1 and S14E2, last Sunday. But they were available only as M3U8 streams, as I said before.

D84web commented 3 years ago

@D84web Open the console (F12) then Network tab. When the video is launched, make a search "manifest" (under this Network tab), and you will get this url.

@guy_teube : Chapeau ! It works. :-)

That's can help me for some documentaries which will end tomorrow... I hope the Team will find a solution.

Whatever, thanks all for your help

PS : Sorry for English... I get rusty.

Trit34 commented 3 years ago

@D84web Open the console (F12) then Network tab. When the video is launched, make a search "manifest" (under this Network tab), and you will get this url.

@guy_teube : Chapeau ! It works. :-)

That's can help me for some documentaries which will end tomorrow... I hope the Team will find a solution.

Whatever, thanks all for your help

PS : Sorry for English... I get rusty.

En français pour toi : je recommande de passer d’abord par youtube-dl -F [URL] pour récupérer les identifiants des bons flux vidéo et audio, car j’ai déjà constaté chez France TV que laisser récupérer automatiquement bestvideo et bestaudio pouvait récupérer le flux audio d’audiodescription pour les malvoyants ou la version originale d’une série étrangère. C’est pas forcément ce qu’on veut.

Par exemple : pour Slam d’aujourd’hui, tu fais ensuite youtube-dl -f dash-video=2000000+dash-audio_fre=96000 https://www.france.tv/france-3/slam/2743369-emission-du-mardi-14-septembre-2021.html, et si tu as ffmpeg installé, ce dernier sera automatiquement sollicité pour réunir les deux flux en un fichier MP4 unique. Tu utiliseras ensuite la méthode donnée plus haut pour les sous-titres.

guy-teube commented 3 years ago

J'ai essayé avec l'url du manifest, j'ai toujours une erreur. Exemple avec cet épisode de C Jamy : $ youtube-dl https://cloudreplay.ftven.fr/08b42500c0416/1036025568_france-domtom_TA.ism/ZXhwPTE2MzE2NjcwNjN+YWNsPSUyZjA4YjQyNTAwYzA0MTYlMmYxMDM2MDI1NTY4X2ZyYW5jZS1kb210b21fVEEuaXNtKn5obWFjPTExMGQ3YWMwNGM4MDBhZDNmOGIyMGU3MGEwOTU1NGE0ZmUwZTllNjZiNjQ5OGMxZTdkNDBiZWI5NzgxNjMwYzQ=/manifest.mpd [generic] manifest: Requesting header WARNING: Falling back on generic information extractor. [generic] manifest: Downloading webpage [generic] manifest: Extracting informationWARNING: [generic] Unknown MIME type application/mp4 in DASH manifest [dashsegments] Total fragments: 756 [download] Destination: manifest-manifest.fvideo=2000000.mp4 ERROR: unable to download video

kapouer commented 3 years ago

Patch works against latest version. First do -F <url> then -f <video>+<audio> <url> and voilà. Thanks all.

boulderob commented 3 years ago

This patch is definitely working for the following selection of typical but unrelated videos so that's a good sign (probably still live for around 7 more days).

https://www.france.tv/series-et-fictions/telefilms/2743743-les-heritiers.html https://www.france.tv/series-et-fictions/telefilms/1087997-etats-d-urgence.html https://www.france.tv/france-3/crimes-parfaits/crimes-parfaits-saison-3/2743343-une-etoile-est-morte.html https://www.france.tv/france-3/un-cas-pour-deux/un-cas-pour-deux-saison-26/994953-erreur-parfaite.html

I could swear that up until this change / patch, that my experience was the opposite : that -F would report m3u8 and no dash. With this patch you get no m3u8 at all. Perhaps it's not a problem and the site has just decided to switch to only mpd now?

But as mentioned by others all is NOT perfect. Since the recent change plus using the new patch, I can also confirm that all of the prior videos listed above that actually DO have subtitles online are reporting NO subs in --list-subs now. So the patch still needs to incorporate parsing the mpd manifests for the subtitles to get it back to 100% working as before.

Thanks for the work on the patch up to this point though!

lcheylus commented 3 years ago

I could swear that up until this change / patch, that my experience was the opposite : that -F would report m3u8 and no dash. With this patch you get no m3u8 at all. Perhaps it's not a problem and the site has just decided to switch to only mpd now?

No, you can get m3u8 and Dash formats for FranceTV videos. I have a personal fix for this issue that gives me the both formats.

$ youtube-dl -F https://www.france.tv/series-et-fictions/telefilms/2743743-les-heritiers.html
[FranceTVSite] 2743743-les-heritiers: Downloading webpage
[FranceTV] 6a219d1a-8f6b-4f5e-8639-4093538e1a84: Downloading fallback desktop video JSON
[FranceTV] 6a219d1a-8f6b-4f5e-8639-4093538e1a84: Downloading fallback mobile video JSON
[FranceTV] 6a219d1a-8f6b-4f5e-8639-4093538e1a84: Downloading signed dash manifest URL
[FranceTV] 6a219d1a-8f6b-4f5e-8639-4093538e1a84: Downloading MPD manifest
WARNING: [FranceTV] Unknown MIME type application/mp4 in DASH manifest
[FranceTV] 6a219d1a-8f6b-4f5e-8639-4093538e1a84: Downloading signed hls manifest URL
[FranceTV] 6a219d1a-8f6b-4f5e-8639-4093538e1a84: Downloading m3u8 information
[info] Available formats for 6a219d1a-8f6b-4f5e-8639-4093538e1a84:
format code                          extension  resolution note
hls-audio-aacl-96-Audio_Description  mp4        audio only [qtz] 
hls-audio-aacl-96-Audio_Français     mp4        audio only [fr] 
dash-audio_fre=96000                 m4a        audio only [fr] DASH audio   96k , m4a_dash container, mp4a.40.2 (48000Hz)
dash-audio_qtz=96000                 m4a        audio only [qtz] DASH audio   96k , m4a_dash container, mp4a.40.2 (48000Hz)
dash-video=400000                    mp4        384x216    DASH video  400k , mp4_dash container, avc1.42C01E, 25fps, video only
hls-522                              mp4        384x216     522k , avc1.42C01E, 25.0fps, video only
dash-video=950000                    mp4        640x360    DASH video  950k , mp4_dash container, avc1.4D401F, 25fps, video only
hls-1105                             mp4        640x360    1105k , avc1.4D401F, 25.0fps, video only
dash-video=1400000                   mp4        960x540    DASH video 1400k , mp4_dash container, avc1.4D401F, 25fps, video only
hls-1582                             mp4        960x540    1582k , avc1.4D401F, 25.0fps, video only
dash-video=2000000                   mp4        1280x720   DASH video 2000k , mp4_dash container, avc1.64001F, 25fps, video only
hls-2218                             mp4        1280x720   2218k , avc1.64001F, 25.0fps, video only (best)

@sarnoud patch is only a quick fix.

It needs a full rework to fix this issue completely :

boulderob commented 3 years ago

Ok great @Icheylus ! So your patch parses both formats but I think you're saying it still does not parse subs right? Can you post your personal fix please since it does get both formats?

Uhm. Upon reflection, I'm thinking if you're able to still get the old m3u8 formats then you should be able to get the old subs from that manifest because that was never broken to begin with!

Thanks

guy-teube commented 3 years ago

Je n'ai pas de solution, n'ayant pas le fichier à patcher sur mon poste :( Ayant respecté l'installation définie par la doc, je ne sais pas pourquoi je ne l'ai pas. Jusqu'à lundi, je pouvais télécharger sans poblème sur France.tv.

lcheylus commented 3 years ago

Je n'ai pas de solution, n'ayant pas le fichier à patcher sur mon poste :( Ayant respecté l'installation définie par la doc, je ne sais pas pourquoi je ne l'ai pas. Jusqu'à lundi, je pouvais télécharger sans poblème sur France.tv.

@guy-teube With your installation (/usr/local/bin/youtube-dl), you have a bundle with only one Python binary, no full sources in /usr/lib/python3/dist-packages/youtube_dl/ (and no extractor/francetv.py file to patch).

First, reinstall YT-DL with full sources sudo pip install --upgrade youtube_dl then find extractor/francetv.py file in /usr/local/lib/, and replaced it with patched version.

Trit34 commented 3 years ago

@guy-teube Note that on some systems, the sources will be located in /usr/lib/python3.9/site-packages/youtube_dl, actually. It is that way on Arch Linux.

But if you’re on Windows, like @NiesmialyGosc, you only have one executable file, and everything packed inside. You can’t change any individual extractor if you don’t recompile the whole software yourself.

NiesmialyGosc commented 3 years ago

But if you’re on Windows, like @NiesmialyGosc, you only have one executable file, and everything packed inside. You can’t change any individual extractor if you don’t recompile the whole software yourself.

Correct. I should've realised folks here were talking about Arch Linux whereas I've been actually on Win10 (as was highlighted in my 2nd reply) & only have an all-in-one executable file "youtube-dl.exe" which was why I was kind of confused above when asked to "replace extractor/francetv.py in my youtube-dl local installation" lol

Just wondering if anyone here could share some expertise on how to recompile youtube-dl.exe on Win10 with the patch written in Python integrated. Maybe "Git for Windows" or Cygwin would work? (ref: How do I apply a diff patch on Windows?) But what exact command lines should I type? I'll give it a try too & come back in case of need for help. Thanks!

How to convert .py to .exe? Step by step guide.

Hardelot commented 3 years ago

I have the same problem with France TV on LINUX FEDORA and Video Downloader.

kdliss commented 3 years ago

thanks for the good response to this issue. on debian linux, I could download the full source version: wget https://yt-dl.org/downloads/2021.06.06/youtube-dl-2021.06.06.tar.gz and eventually apply the patch, which works. it gave me a little deeper insight into youtube-dl structure!

Hardelot commented 3 years ago

Votre méthode est périmée je crains.

Trit34 commented 3 years ago

@Hardelot Précise.

Hardelot commented 3 years ago

06/06/2021 alors que nous sommes le 15/09/2021...!!!???

moviuro commented 3 years ago

@Hardelot last release as of 2021-09-09 is https://github.com/ytdl-org/youtube-dl/releases/tag/2021.06.06

guy-teube commented 3 years ago

Je n'ai pas de solution, n'ayant pas le fichier à patcher sur mon poste :( Ayant respecté l'installation définie par la doc, je ne sais pas pourquoi je ne l'ai pas. Jusqu'à lundi, je pouvais télécharger sans poblème sur France.tv.

@guy-teube With your installation (/usr/local/bin/youtube-dl), you have a bundle with only one Python binary, no full sources in /usr/lib/python3/dist-packages/youtube_dl/ (and no extractor/francetv.py file to patch).

First, reinstall YT-DL with full sources sudo pip install --upgrade youtube_dl then find extractor/francetv.py file in /usr/local/lib/, and replaced it with patched version.

I've tried to reinstall with pip: I didn't find either the francetv.py file.

I've tried to download the tar.gz file. Here, I've found the francetv.py! I've patch it (now it's 20 686 bytes, and 20 534 bytes for the non-fixed version, someone can confirm?), but even with it, running from : ./bin/youtube-dl myurl, I still have "ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found".

Seems it will stay unresolved on my Ubuntu...

alexisfrjp commented 3 years ago

Je n'ai pas de solution, n'ayant pas le fichier à patcher sur mon poste :( Ayant respecté l'installation définie par la doc, je ne sais pas pourquoi je ne l'ai pas. Jusqu'à lundi, je pouvais télécharger sans poblème sur France.tv.

@guy-teube With your installation (/usr/local/bin/youtube-dl), you have a bundle with only one Python binary, no full sources in /usr/lib/python3/dist-packages/youtube_dl/ (and no extractor/francetv.py file to patch). First, reinstall YT-DL with full sources sudo pip install --upgrade youtube_dl then find extractor/francetv.py file in /usr/local/lib/, and replaced it with patched version.

I've tried to reinstall with pip: I didn't find either the francetv.py file.

I've tried to download the tar.gz file. Here, I've found the francetv.py! I've patch it (now it's 20 686 bytes, and 20 534 bytes for the non-fixed version, someone can confirm?), but even with it, running from : ./bin/youtube-dl myurl, I still have "ERROR: Unable to download JSON metadata: HTTP Error 404: Not Found".

Seems it will stay unresolved on my Ubuntu...

Try ~/.local/lib/python3.8/site-packages/youtube_dl/extractor if installed with pip(3) --user.

alexisfrjp commented 3 years ago

Here is quick early patch I wrote that seems to repair quite a few links. youtube-dl-patch.txt

I don't the see related merge request. Are you gonna do it?

lcheylus commented 3 years ago

Here is quick early patch I wrote that seems to repair quite a few links. youtube-dl-patch.txt

I don't the see related merge request. Are you gonna do it?

I repeat : @sarnoud patch is a quick fix, not complete.

This fix needs a rework before creating an Pull Request / Ask for Merge :

boulderob commented 3 years ago

Now that I'm using the patch / downloading videos as DASH/mpd (vs m3u8 the old way), my downloads are really slow and more often then not timeout. i have to constantly restart the downloads repeatedly to ever reach 100% download. It's almost unusable. I didn't have this problem with m3u8. Anybody else seeing this out there.

@lcheylus i know you're still working on your patch and that it's not well tested or complete yet but you seem to have been able to at least get the old m3u8 format ids working. Could you send it to me? I'll take full responsibility on my own system if it doesn't work perfectly :) It's easy enough to put back if I run into trouble. Plus maybe i can work on it a bit though I can't promise.

Thanks

Hardelot commented 3 years ago

Que dois-je mettre dans .local/lib/python3.9/site-packages/youtube_dl/extractor/ et comment faire en lignes de commande ?