search

Ascoware / get-iplayer-automator

The goal of Get iPlayer Automator is to allow iTunes and your Mac to become the hub for your British Television experience regardless of where in the world you are. Currently, Get iPlayer Automator allows you to download and watch BBC and ITV shows on your Mac. Series-Link/PVR functionality ensures you will never miss your favourite shows. Programmes are fully tagged and added to iTunes automatically upon completion. It is simple and easy to use, and runs on any machine running Mac OS X 10.7 or later. And since the shows are in iTunes, it is extremely easy to transfer them to your iPod, iPhone, or Apple TV allowing you to enjoy your shows on the go or on your television.
GNU General Public License v3.0
144 stars 27 forks source link

ITV Download Fail GIA 1.20.1 #304

Open sxmpete opened 3 years ago

sxmpete commented 3 years ago

Failure similar to issue 303. I am raising separate issue since 303 discussion seems to have strayed from my original concern of a python problem, into on a cache reset issue which I am not experiencing. Running mac OS Mojave, ExpressVPN, log follows: Get iPlayer Automator 1.20.1 Initialized. GetITVShows: ITV Cache Update Starting <AppController: 0x7fc28c716060>: Updating Programme Index Feeds...

/Applications/Get iPlayer Automator v1.20.1.app/Contents/Resources/get_iplayer/perl/bin/get_iplayer -e14400 --type=tv --refresh --nopurge --profile-dir=/Users/peterdonlevy2/Library/Application Support/Get iPlayer Automator .* INFO: Indexing tv programmes (concurrent) GetITVShows (Info): Episodes: 2556 Today Programmes: 388 GetITVShows: Update Finished INFO: Added 649 tv programmes to cache <AppController: 0x7fc28c716060>: Index Updated.

INFO: Loading proxy settings... INFO: No proxy to load INFO: Proxy load complete. INFO: No proxy will be used AppController: Starting Downloads

Downloading Show 1/1:

ITV Download (ID=1a7665a0025): Downloading Doc Martin - Mon Dec 07 INFO: Preparing Request for Auth Info INFO: Requesting Metadata. DEBUG: Metadata response status code: 200 INFO: Metadata processed. DEBUG: youtube-dl args:["https://www.itv.com/hub/doc-martin/1a7665a0025", "-f", "mp4/best", "-o", "/Users/peterdonlevy2/Movies/TV Shows/Doc Martin/Doc Martin.s07e01.Rescue Me.%(ext)s", "--verbose"] DEBUG: youtube-dl environment: ["PYTHONPATH": "/Applications/Get iPlayer Automator v1.20.1.app/Contents/Resources", "PATH": "/Applications/Get iPlayer Automator v1.20.1.app/Contents/MacOS:/Applications/Get iPlayer Automator v1.20.1.app/Contents/Resources/get_iplayer/utils/bin:/usr/bin"] [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: [u'https://www.itv.com/hub/doc-martin/1a7665a0025', u'-f', u'mp4/best', u'-o', u'/Users/peterdonlevy2/Movies/TV Shows/Doc Martin/Doc Martin.s07e01.Rescue Me.%(ext)s', u'--verbose'] [debug] Encodings: locale US-ASCII, fs utf-8, out None, pref US-ASCII [debug] youtube-dl version 2020.12.12 [debug] Python version 2.7.10 (CPython) - Darwin-18.7.0-x86_64-i386-64bit [debug] exe versions: ffmpeg 4.3, rtmpdump 2.4-102-ga3a600d-get_iplayer [debug] Proxy map: {} [debug] Using fake IP 25.86.6.4 (GB) as X-Forwarded-For. [ITV] 1a7665a0025: Downloading webpage [ITV] 1a7665a0025: Downloading JSON metadata [ITV] 1a7665a0025: Downloading m3u8 information [debug] Invoking downloader on u'https://itvpnpdotcom.content.itv.com/1-7665-0025-003/18/3/VAR012/1-7665-0025-003_18_3_VAR012.ism/1-7665-0025-003_18_3_VAR012-audio=128000-video=1541348.m3u8' [hlsnative] Downloading m3u8 manifest ERROR: unable to download video data: '' Traceback (most recent call last): File "/Applications/Get iPlayer Automator v1.20.1.app/Contents/Resources/youtube-dl/youtube_dl/YoutubeDL.py", line 1936, in process_info success = dl(filename, info_dict) File "/Applications/Get iPlayer Automator v1.20.1.app/Contents/Resources/youtube-dl/youtube_dl/YoutubeDL.py", line 1875, in dl return fd.download(name, info) File "/Applications/Get iPlayer Automator v1.20.1.app/Contents/Resources/youtube-dl/youtube_dl/downloader/common.py", line 366, in download return self.real_download(filename, info_dict) File "/Applications/Get iPlayer Automator v1.20.1.app/Contents/Resources/youtube-dl/youtube_dl/downloader/hls.py", line 62, in real_download urlh = self.ydl.urlopen(self._prepare_url(info_dict, man_url)) File "/Applications/Get iPlayer Automator v1.20.1.app/Contents/Resources/youtube-dl/youtube_dl/YoutubeDL.py", line 2248, in urlopen return self._opener.open(req, timeout=self._socket_timeout) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 431, in open response = self._open(req, data) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 449, in _open '_open', req) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain result = func(*args) File "/Applications/Get iPlayer Automator v1.20.1.app/Contents/Resources/youtube-dl/youtube_dl/utils.py", line 2736, in https_open req, **kwargs) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1200, in do_open r = h.getresponse(buffering=True) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1132, in getresponse response.begin() File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 453, in begin version, status, reason = self._read_status() File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 417, in _read_status raise BadStatusLine(line) BadStatusLine: '' ITV Download (ID=1a7665a0025): youtube-dl finished downloading AppController: Downloads Finished

raythw commented 3 years ago

I’ve no idea whether this is related or not. Attempting to download 'Dickinson's Real Deal.s01e944.Tue Jan 05' GiA crashed entirely. Restarting got this:

Get iPlayer Automator 1.20.1 Initialized. GetITVShows: ITV Cache Update Starting <AppController: 0x7ffbb7c0a880>: Updating Programme Index Feeds...

INFO: Indexing tv programmes (concurrent) GetITVShows (Info): Episodes: 2548 Today Programmes: 382 GetITVShows: Update Finished INFO: Added 0 tv programmes to cache <AppController: 0x7ffbb7c0a880>: Index Updated.

INFO: Loading proxy settings... INFO: No proxy to load INFO: Proxy load complete. INFO: No proxy will be used AppController: Starting Downloads

Downloading Show 1/1:

ITV Download (ID=1a8665a0944): Downloading Dickinson's Real Deal - Tue Jan 05 INFO: Preparing Request for Auth Info INFO: Metadata processed. ITV Download (ID=1a8665a0944): youtube-dl finished downloading INFO: Downloading thumbnail INFO: Thumbnail download completed to file:///var/folders/r6/53pzyjd55x7b40m97cw3zpv80000gn/T/CFNetworkDownload_fEhXMj.tmp INFO: Beginning AtomicParsley Tagging. INFO: Tagging failed. INFO: Converting Subtitles... INFO: Converting to SubRip: /Users/raywhite/Downloads/Dickinson's Real Deal/Dickinson's Real Deal.s01e944.Tue Jan 05.en.vtt INFO: Conversion to SubRip complete: Unknown.srt AppController: Downloads Finished Adding Dickinson's Real Deal - Tue Jan 05 to TV Can't add file to TV -- incompatible format.

Odd. I've also had a repetition of the lock-up problem with another episode of the same programme,, where attempting to quit the application (and then cancelling that action) caused GiA to continue and add the programme to TV.

I'm running 1.20.1 on Big Sur 11.1.

Thanks to all for the work done on this very useful application.

hugofalkman commented 3 years ago

Here is a log for both an ITV download that fails (3/8) and one that completes (4/8)

Screenshot 2021-01-27 at 13 05 10

Downloading Show 3/8:

ITV Download (ID=2a4269a0020): Downloading Marcella - Tue Jan 26 INFO: Preparing Request for Auth Info INFO: Requesting Metadata. DEBUG: Metadata response status code: 200 INFO: Metadata processed. DEBUG: youtube-dl args:["https://www.itv.com/hub/marcella/2a4269a0020", "-f", "mp4/best", "-o", "/Users/zilla/Movies/TV Shows/Marcella/Marcella.s03e03.Episode 3.%(ext)s", "--write-sub", "--embed-subs", "--verbose"] DEBUG: youtube-dl environment: ["PYTHONPATH": "/Applications/Get iPlayer Automator.app/Contents/Resources", "PATH": "/Applications/Get iPlayer Automator.app/Contents/MacOS:/Applications/Get iPlayer Automator.app/Contents/Resources/get_iplayer/utils/bin:/usr/bin"] [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: [u'https://www.itv.com/hub/marcella/2a4269a0020', u'-f', u'mp4/best', u'-o', u'/Users/zilla/Movies/TV Shows/Marcella/Marcella.s03e03.Episode 3.%(ext)s', u'--write-sub', u'--embed-subs', u'--verbose'] [debug] Encodings: locale US-ASCII, fs utf-8, out None, pref US-ASCII [debug] youtube-dl version 2020.12.12 [debug] Python version 2.7.16 (CPython) - Darwin-20.2.0-x86_64-i386-64bit [debug] exe versions: ffmpeg 4.3, rtmpdump 2.4-102-ga3a600d-get_iplayer [debug] Proxy map: {} [debug] Using fake IP 25.91.167.17 (GB) as X-Forwarded-For. [ITV] 2a4269a0020: Downloading webpage [ITV] 2a4269a0020: Downloading JSON metadata [ITV] 2a4269a0020: Downloading m3u8 information WARNING: Failed to download m3u8 information: HTTP Error 403: Forbidden ERROR: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. Traceback (most recent call last): File "/Applications/Get iPlayer Automator.app/Contents/Resources/youtube-dl/youtube_dl/YoutubeDL.py", line 803, in wrapper return func(self, *args, **kwargs) File "/Applications/Get iPlayer Automator.app/Contents/Resources/youtube-dl/youtube_dl/YoutubeDL.py", line 824, in __extract_info ie_result = ie.extract(url) File "/Applications/Get iPlayer Automator.app/Contents/Resources/youtube-dl/youtube_dl/extractor/common.py", line 532, in extract ie_result = self._real_extract(url) File "/Applications/Get iPlayer Automator.app/Contents/Resources/youtube-dl/youtube_dl/extractor/itv.py", line 115, in _real_extract self._sort_formats(formats) File "/Applications/Get iPlayer Automator.app/Contents/Resources/youtube-dl/youtube_dl/extractor/common.py", line 1359, in _sort_formats raise ExtractorError('No video formats found') ExtractorError: No video formats found; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. ITV Download (ID=2a4269a0020): youtube-dl finished downloading

Downloading Show 4/8:

ITV Download (ID=2a4269a0021): Downloading Marcella - Tue Jan 26 INFO: Preparing Request for Auth Info INFO: Requesting Metadata. DEBUG: Metadata response status code: 200 INFO: Metadata processed. DEBUG: youtube-dl args:["https://www.itv.com/hub/marcella/2a4269a0021", "-f", "mp4/best", "-o", "/Users/zilla/Movies/TV Shows/Marcella/Marcella.s03e04.Episode 4.%(ext)s", "--write-sub", "--embed-subs", "--verbose"] DEBUG: youtube-dl environment: ["PYTHONPATH": "/Applications/Get iPlayer Automator.app/Contents/Resources", "PATH": "/Applications/Get iPlayer Automator.app/Contents/MacOS:/Applications/Get iPlayer Automator.app/Contents/Resources/get_iplayer/utils/bin:/usr/bin"] [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: [u'https://www.itv.com/hub/marcella/2a4269a0021', u'-f', u'mp4/best', u'-o', u'/Users/zilla/Movies/TV Shows/Marcella/Marcella.s03e04.Episode 4.%(ext)s', u'--write-sub', u'--embed-subs', u'--verbose'] [debug] Encodings: locale US-ASCII, fs utf-8, out None, pref US-ASCII [debug] youtube-dl version 2020.12.12 [debug] Python version 2.7.16 (CPython) - Darwin-20.2.0-x86_64-i386-64bit [debug] exe versions: ffmpeg 4.3, rtmpdump 2.4-102-ga3a600d-get_iplayer [debug] Proxy map: {} [ITV] 2a4269a0021: Downloading webpage [ITV] 2a4269a0021: Downloading JSON metadata [ITV] 2a4269a0021: Downloading m3u8 information [info] Writing video subtitles to: /Users/zilla/Movies/TV Shows/Marcella/Marcella.s03e04.Episode 4.en.vtt ITV Download (ID=2a4269a0021): Subtitle path = /Users/zilla/Movies/TV Shows/Marcella/Marcella.s03e04.Episode 4.en.vtt [debug] Invoking downloader on u'https://itvpnpdotcom.content.itv.com/2-4269-0021-001/18/1/VAR012/2-4269-0021-001_18_1_VAR012.ism/2-4269-0021-001_18_1_VAR012-audio=96000-video=1703877.m3u8' [hlsnative] Downloading m3u8 manifest [hlsnative] Total fragments: 463

[download] 0.0% of ~636.37MiB at Unknown speed ETA 33:36:05

[download] 0.0% of ~636.37MiB at 91.49KiB/s ETA 13:09:55

[download] 0.0% of ~636.37MiB at 889.45KiB/s ETA 01:17:09

[download] 0.0% of ~636.37MiB at 936.99KiB/s ETA 43:33

[download] 0.0% of ~636.37MiB at 1.23MiB/s ETA 24:30

[download] 0.0% of ~636.37MiB at 1.85MiB/s ETA 13:37

[download] 0.1% of ~636.37MiB at 2.86MiB/s ETA 07:38

skovatch commented 3 years ago

OK, we have multiple issues here, and they aren't related.

@hugofalkman These are youtube-dl errors trying to connect to ITV, not GiA. Notice the 403 error -- that could be blocking based on location, load or any number of things.

@raythw yours is a problem I've seen occasionally where GiA isn't able to figure out the download location, so a number of post-processing steps fail. I have been looking at that.

@sxmpete you are on macOS 10.14 which has python 2.7, but it should have the necessary changes needed to make HTTPS work correctly. My first advice is to update your OS, at least to 10.15. Otherwise you'll need a newer version of python.

So please, folks, check other bugs first. I will try to cross-reference them later.

sxmpete commented 3 years ago

Upgraded to Python 3.9 and DL still failing, although different error. Log attached ITV DL Fail 29Jan2021.docx