Open baffled-mart opened 6 years ago
I can repeat the problem here, of the 5 episodes available 3 download, 2 fail.
Thank you for your near immediate response. I wish I could help but wouldn't have a clue where to start. If it helps, I found the same occurs with (including the 2nd Protectors); https://www.itv.com/hub/grantchester/2a2958a0002 https://www.itv.com/hub/the-protectors/ENT1315a0006 https://www.itv.com/hub/the-widower/2a1391a0003 https://www.itv.com/hub/white-right-meeting-the-enemy-exposure/2a5133a0001
Ah, I see that similar has been commented on "youtube-dl". "... That hasn't been for all the ITV videos, it just seems to be a tiny but annoying fraction, heh. I believe they're not always using secure-mercury now. However, those videos all still play in the browser, so something must be working right. Looking at the network requests, it seems maybe there's a shift to a new playlist service? ..." https://github.com/rg3/youtube-dl/issues/14944
If that is any help.
@baffled-mart wrote:
I found the same occurs with (including the 2nd Protectors); https://www.itv.com/hub/grantchester/2a2958a0002 https://www.itv.com/hub/the-protectors/ENT1315a0006 https://www.itv.com/hub/the-widower/2a1391a0003 https://www.itv.com/hub/white-right-meeting-the-enemy-exposure/2a5133a0001
However, those videos all still play in the browser, so something must be working right.
What you get in a compatible desktop browser is encrypted (DRM'ed) MPEG-DASH streams (see also the discussion in https://github.com/monsieurvideo/get-flash-videos/issues/211), whereas the itv.pm plugin of GFV relies upon unencrypted RTMPE (aka Flash) streams (downloadable by RTMPDump).
rg3/youtube-dl#14944
If you look closely at https://github.com/rg3/youtube-dl/issues/14944#issuecomment-351080759, you'll be able to identify the (encrypted) MPEG-DASH manifest (mpd, i.e. Media Presentation Description):
"Base" : "https://itvpnpdotcom-a.akamaihd.net/",
"MediaFiles" : [
{
"Href" : "2-2271-0033-001/18/1/VAR011/2-2271-0033-001_18_1_VAR011.ism/.mpd?hdnea=st=1513091350~exp=1513112950~acl=/*~hmac=13c038ae3549a09a864295735957c6627be0e40a1856b31adfd09803f268c512",
"KeyServiceUrl" : "https://itvpnp.live.ott.irdeto.com/ClearKey/getkeys?CrmId=itvpnp&AccountId=itvpnp&ContentId=2-2271-0033-001_18"
i.e. the full mpd URI is
https://itvpnpdotcom-a.akamaihd.net/2-2271-0033-001/18/1/VAR011/2-2271-0033-001_18_1_VAR011.ism/.mpd?hdnea=st=1513091350~exp=1513112950~acl=/*~hmac=13c038ae3549a09a864295735957c6627be0e40a1856b31adfd09803f268c512
which is tokenised (expiry+authentication) and uses the ClearKey DRM encryption...
GFV has still no support for MPEG-DASH streams, and, AFAIK, the ClearKey encryption hasn't been cracked (at least publicly...).
@njtaylor
The yt-dl dev has tackled this issue via https://github.com/rg3/youtube-dl/commit/3fae11ac00 ; that is Python, while GFV is written in Perl; I hope you can "translate"; however, the fixed yt-dl version DOES NOT RETURN ANY FLASH streams for the ITVHub listings affected by this issue, only AppleHLS streams are identified:
youtube-dl --version
2017.12.14
youtube-dl -F "https://www.itv.com/hub/the-widower/2a1391a0003"
[ITV] 2a1391a0003: Downloading webpage
[ITV] 2a1391a0003: Downloading XML
[ITV] 2a1391a0003: Downloading JSON metadata
[ITV] 2a1391a0003: Downloading m3u8 information
[info] Available formats for 2a1391a0003:
format code extension resolution note
hls-136 mp4 audio only 136k , mp4a.40.2@128k
hls-569 mp4 512x288 569k , avc1.64001F@ 408k, 25.0fps, mp4a.40.2@128k
hls-754 mp4 512x288 754k , avc1.64001F@ 582k, 25.0fps, mp4a.40.2@128k
hls-1001 mp4 896x504 1001k , avc1.64001F@ 816k, 25.0fps, mp4a.40.2@128k
hls-1286 mp4 896x504 1286k , avc1.64001F@1084k, 25.0fps, mp4a.40.2@128k
hls-1562 mp4 896x504 1562k , avc1.64001F@1345k, 25.0fps, mp4a.40.2@128k (best)
FWIW, the audio stream is mistakenly reported as being AAC LC@128kbps, while in reality is just @96kbps, as in the rest of the downloadable programmes in FLV container... If, indeed, Flash streams are AWOL for these programmes, then am afraid it's time you implemented AppleHLS support for ITV, as I hinted back in the summer here...
ITV issues aside, I wish Merry Christmas to all those that observe it :smile:
@Vangelis66 Thanks, as always, for your learned comments (g_i_p too). Just a bit beyond my understanding. :-) MC to you too.
Initial support for HLS AES-128 added to GFV still being tested. Worked for couple of downloads from ITV, needs more work - restarts, command line options to force use.
ITV is not consistent in supporting some downloads are HLS only, many new programmes have RTMP downloads and HLS. The basic HLS support is now there for ITV.
Thanks for the update and all your hard work. Much appreciated.
Initial support for HLS AES-128 added to GFV still being tested. Worked for couple of downloads from ITV, needs more work - restarts, command line options to force use.
ITV is not consistent in supporting some downloads are HLS only, many new programmes have RTMP downloads and HLS. The basic HLS support is now there for ITV.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.
Able to view episode on-line in browser but unable to download for off-line viewing. An example is episode 10 of The Protectors (other examples can be listed). Any help appreciated. Platform; Windows Vista 32 SP2 Perl version ActivePerl-5.20.2.2001
Debug output; Using method 'itv' plugin version 0.09 for https://www.itv.com/hub/the-protectors/ENT1315a0022 Downloading https://www.itv.com/hub/the-protectors/ENT1315a0022 -> GET https://www.itv.com/hub/the-protectors/ENT1315a0022 <- 200 text/html; charset=UTF-8 15443 (82551): UTF8 on, ASCII, 82551 characters 82551 bytes ~0022
Couldn't extract Flash movie URL. This site may need specific support adding, or fixing.
Please confirm that the URL is valid and that you are running the latest version of get-flash-videos.
If the latest version does not support this site please create a bug report at https://github.com/monsieurvideo/get-flash-videos making sure you include the output with --debug enabled. Alternatively, fix it yourself and send us a pull request. Couldn't download any videos.