search

Ascoware / get-iplayer-automator

The goal of Get iPlayer Automator is to allow iTunes and your Mac to become the hub for your British Television experience regardless of where in the world you are. Currently, Get iPlayer Automator allows you to download and watch BBC and ITV shows on your Mac. Series-Link/PVR functionality ensures you will never miss your favourite shows. Programmes are fully tagged and added to iTunes automatically upon completion. It is simple and easy to use, and runs on any machine running Mac OS X 10.7 or later. And since the shows are in iTunes, it is extremely easy to transfer them to your iPod, iPhone, or Apple TV allowing you to enjoy your shows on the go or on your television.
GNU General Public License v3.0
144 stars 27 forks source link

BBC News programme being downloaded with yt-dlp! And fails to complete. #445

Closed jefro108 closed 1 year ago

jefro108 commented 1 year ago

Trying to download a BBC News programme Grenada: Confronting the Past and it gets to 100% Downloaded but not complete, when I noticed in the log that it's not being downloaded with get_iplayer but yt-dlp!

Get iPlayer Automator 1.25.1 Initialized.
Updating Programme Index Feeds...

INFO: Indexing tv programmes (concurrent)
INFO: Added 361 tv programmes to cache
INFO: Indexing radio programmes (concurrent)
INFO: Added 620 radio programmes to cache
INFO: 22362 matching programmes
BBC Index Updated
No proxy to load
Proxy load complete.
Starting Downloads

Downloading Show 1/1:

Downloading Grenada: Confronting the Past
ITV: Requesting Metadata
DEBUG: Metadata response status code: 200
INFO: Metadata processed.
DEBUG: youtube-dl args:["https://www.bbc.co.uk/programmes/m0017n9r", "--user-agent", "Mozilla/5.0", "-o", "/Volumes/media/Downloads/TV Shows/Grenada - Confronting the Past/Grenada - Confronting the Past.s01e01.%(ext)s", "-f", "best[height<=720]", "--write-sub", "--sub-format", "vtt", "--convert-subtitles", "srt", "--embed-subs", "--verbose", "--embed-thumbnail"]
DEBUG: youtube-dl environment: ["SSL_CERT_FILE": "/Applications/Get iPlayer Automator.app/Contents/Resources/cacert.pem", "PATH": "/Applications/Get iPlayer Automator.app/Contents/Resources/yt-dlp_macos:/Applications/Get iPlayer Automator.app/Contents/Resources/get_iplayer/utils/bin"]
[debug] Command-line config: ['https://www.bbc.co.uk/programmes/m0017n9r', '--user-agent', 'Mozilla/5.0', '-o', '/Volumes/media/Downloads/TV Shows/Grenada - Confronting the Past/Grenada - Confronting the Past.s01e01.%(ext)s', '-f', 'best[height<=720]', '--write-sub', '--sub-format', 'vtt', '--convert-subtitles', 'srt', '--embed-subs', '--verbose', '--embed-thumbnail']

[debug] Encodings: locale US-ASCII, fs utf-8, out ascii (No ANSI), err ascii (No ANSI), pref US-ASCII
[debug] yt-dlp version 2021.12.27 [6223f67a8] (mac_dir)

[debug] Python version 3.9.9 (CPython 64bit) - macOS-12.6.1-x86_64-64bit

[debug] exe versions: ffmpeg 4.3.2-tessus

[debug] Optional libraries: Cryptodome, sqlite

[debug] Proxy map: {}

[debug] [bbc.co.uk] Extracting URL: https://www.bbc.co.uk/programmes/m0017n9r

[bbc.co.uk] m0017n9r: Downloading video page

[bbc.co.uk] m0017n9r: Downloading playlist JSON

[bbc.co.uk] m0017n9q: Downloading media selection JSON

[bbc.co.uk] m0017n9q: Downloading MPD manifest

[bbc.co.uk] m0017n9q: Downloading MPD manifest

[bbc.co.uk] m0017n9q: Downloading MPD manifest

[bbc.co.uk] m0017n9q: Downloading MPD manifest

[bbc.co.uk] m0017n9q: Downloading MPD manifest

[bbc.co.uk] m0017n9q: Downloading MPD manifest

[bbc.co.uk] m0017n9q: Downloading m3u8 information

[bbc.co.uk] m0017n9q: Downloading m3u8 information

[bbc.co.uk] m0017n9q: Downloading m3u8 information

[bbc.co.uk] m0017n9q: Downloading m3u8 information

[bbc.co.uk] m0017n9q: Downloading m3u8 information

[bbc.co.uk] m0017n9q: Downloading m3u8 information

[bbc.co.uk] m0017n9q: Downloading captions

[debug] Formats sorted by: hasvid, ie_pref, lang, quality, res, fps, hdr:12(7), vcodec:vp9.2(10), acodec, filesize, fs_approx, tbr, vbr, abr, asr, proto, vext, aext, hasaud, source, id

[debug] Downloading subtitles: en
WARNING: No subtitle format found matching "vtt" for language en, using ttml

[info] m0017n9q: Downloading 1 format(s): mf_cloudfront-5510-1

[info] Writing video subtitles to: /Volumes/media/Downloads/TV Shows/Grenada - Confronting the Past/Grenada - Confronting the Past.s01e01.en.ttml
Subtitle path = /Volumes/media/Downloads/TV Shows/Grenada - Confronting the Past/Grenada - Confronting the Past.s01e01.en.ttml

[debug] Invoking downloader on "http://vod-sub-uk-live.akamaized.net/iplayer/subtitles/ng/modav/p0c61zzb_m0017n9q_cUnknown_1652635566229.xml?__gda__=1675795442_6ab6a09c86e9fb18a5a6120704bc9bb3"

[download] Destination: /Volumes/media/Downloads/TV Shows/Grenada - Confronting the Past/Grenada - Confronting the Past.s01e01.en.ttml
Downloading to /Volumes/media/Downloads/TV Shows/Grenada - Confronting the Past/Grenada - Confronting the Past.s01e01.en.ttml

[download]   1.5% of 64.59KiB at  7.02KiB/s ETA 00:09

[download]   4.6% of 64.59KiB at 20.88KiB/s ETA 00:02
[download]  10.8% of 64.59KiB at 48.34KiB/s ETA 00:01
[download]  23.2% of 64.59KiB at 103.12KiB/s ETA 00:00
[download]  48.0% of 64.59KiB at 212.21KiB/s ETA 00:00
[download]  97.5% of 64.59KiB at 429.04KiB/s ETA 00:00
[download] 100% of 64.59KiB at 437.05KiB/s ETA 00:00  

[download] 100% of 64.59KiB in 00:00                

[info] Downloading video thumbnail 0 ...

[info] Writing video thumbnail 0 to: /Volumes/media/Downloads/TV Shows/Grenada - Confronting the Past/Grenada - Confronting the Past.s01e01.jpg

[SubtitlesConvertor] Converting subtitles

WARNING: You have requested to convert dfxp (TTML) subtitles into another format, which results in style information loss

Deleting original file /Volumes/media/Downloads/TV Shows/Grenada - Confronting the Past/Grenada - Confronting the Past.s01e01.en.ttml (pass -k to keep)

[debug] Invoking downloader on "https://vod-hls-uk.live.cf.md.bbci.co.uk/usp/auth/vod/piff_abr_full_hd/028a9b-m0017n9q/vf_m0017n9q_b0a56dbb-a77e-481d-a5f8-fc8f7e97415c.ism.hlsv2.ism/vf_m0017n9q_b0a56dbb-a77e-481d-a5f8-fc8f7e97415c.ism.hlsv2-audio_eng_1=128000-video=5070000.m3u8"

[download] /Volumes/media/Downloads/TV Shows/Grenada - Confronting the Past/Grenada - Confronting the Past.s01e01.mp4 has already been downloaded

[download] 100% of 862.19MiB

WARNING: m0017n9q: Possible MPEG-TS in MP4 container or malformed AAC timestamps

[EmbedSubtitle] Embedding subtitles in "/Volumes/media/Downloads/TV Shows/Grenada - Confronting the Past/Grenada - Confronting the Past.s01e01.mp4"

[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i 'file:/Volumes/media/Downloads/TV Shows/Grenada - Confronting the Past/Grenada - Confronting the Past.s01e01.mp4' -i 'file:/Volumes/media/Downloads/TV Shows/Grenada - Confronting the Past/Grenada - Confronting the Past.s01e01.en.srt' -c copy -map 0 -dn -map -0:s -map -0:d -c:s mov_text -map 1:0 -metadata:s:s:0 language=eng -movflags +faststart 'file:/Volumes/media/Downloads/TV Shows/Grenada - Confronting the Past/Grenada - Confronting the Past.s01e01.temp.mp4'

[debug] ffmpeg version 4.3.2-tessus  https://evermeet.cx/ffmpeg/  Copyright (c) 2000-2021 the FFmpeg developers
  built with Apple clang version 11.0.0 (clang-1100.0.33.17)
  configuration: --cc=/usr/bin/clang --prefix=/opt/ffmpeg --extra-version=tessus --enable-avisynth --enable-fontconfig --enable-gpl --enable-libaom --enable-libass --enable-libbluray --enable-libdav1d --enable-libfreetype --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libmysofa --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenh264 --enable-libopenjpeg --enable-libopus --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvmaf --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-version3 --pkg-config-flags=--static --disable-ffplay
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'file:/Volumes/media/Downloads/TV Shows/Grenada - Confronting the Past/Grenada - Confronting the Past.s01e01.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.45.100
  Duration: 00:23:13.09, start: 0.000000, bitrate: 5191 kb/s
    Stream #0:0(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
    Stream #0:1(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x720 [SAR 1:1 DAR 16:9], 5053 kb/s, 50 fps, 50 tbr, 90k tbn, 100 tbc (default)
    Metadata:
      handler_name    : VideoHandler
Input #1, srt, from 'file:/Volumes/media/Downloads/TV Shows/Grenada - Confronting the Past/Grenada - Confronting the Past.s01e01.en.srt':
  Duration: N/A, bitrate: N/A
    Stream #1:0: Subtitle: subrip
Multiple -c, -codec, -acodec, -vcodec, -scodec or -dcodec options specified for stream 2, only the last option '-c:s mov_text' will be used.
Output #0, mp4, to 'file:/Volumes/media/Downloads/TV Shows/Grenada - Confronting the Past/Grenada - Confronting the Past.s01e01.temp.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.45.100
    Stream #0:0(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 128 kb/s (default)
    Metadata:
      handler_name    : SoundHandler
    Stream #0:1(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv, bt709), 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 5053 kb/s, 50 fps, 50 tbr, 90k tbn, 90k tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:2(eng): Subtitle: mov_text (tx3g / 0x67337874)
    Metadata:
      encoder         : Lavc58.91.100 mov_text
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
  Stream #1:0 -> #0:2 (subrip (srt) -> mov_text (native))
Press [q] to stop, [?] for help

ERROR: Postprocessing: Press [q] to stop, [?] for help

Traceback (most recent call last):
  File "yt_dlp/YoutubeDL.py", line 3064, in process_info
  File "yt_dlp/YoutubeDL.py", line 3235, in post_process
  File "yt_dlp/YoutubeDL.py", line 3178, in run_pp
  File "yt_dlp/postprocessor/common.py", line 22, in run
  File "yt_dlp/postprocessor/common.py", line 117, in wrapper
  File "yt_dlp/postprocessor/ffmpeg.py", line 656, in run
  File "yt_dlp/postprocessor/ffmpeg.py", line 280, in run_ffmpeg_multiple_files
  File "yt_dlp/postprocessor/ffmpeg.py", line 319, in real_run_ffmpeg
yt_dlp.postprocessor.ffmpeg.FFmpegPostProcessorError: Press [q] to stop, [?] for help

yt-dlp finished downloading

I noticed also that the programme Status says "Not in cache"

Using the latest version 1.25.1 on macOS 12.6.1

skovatch commented 1 year ago

Yikes! Did you add it via 'Get Current Webpage'? Given the age of the show it's not surprising that it isn't in the cache, but I suspect the metadata scraper didn't detect it as BBC so it routed the download through the ITV downloader.

Thanks for the report in any event.

skovatch commented 1 year ago

Never mind, I found it. The logic for scraping BBC URLs with 'programmes' in the URL forgot to set the tvNetwork field.

jefro108 commented 1 year ago

It was using the Get Current Webpage. Although yt-dlp nearly did it. Just fell over at the end. I ended up with a folder with jpg, srt and mp4 files - just not muxed!

skovatch commented 1 year ago

Fixed in v1.25.4