No progress with downloading The Bay (Series 3, Episode 1) from ITV

[adbeckett]

Using the latest version of Get iPlayer Automator Version 1.21.12 (20210729001) running on Big Sur 11.6.2, trying to download any episode of The Bay Series 3 seems to get stuck. The directory for the series is created, but it remains empty.

I'm in the UK, no VPN involved. The verbose log is below:

INFO: Loading proxy settings... INFO: No proxy to load INFO: Proxy load complete. INFO: No proxy will be used AppController: Starting Downloads

Downloading Show 1/1:

Downloading The Bay: Series 3 INFO: Preparing Request for Auth Info INFO: Requesting Metadata. DEBUG: Metadata response status code: 200 INFO: Metadata processed. DEBUG: youtube-dl args:["https://www.itv.com/hub/the-bay/2a5041a0013", "-f", "mp4/best", "-o", "/Users/andrew/Movies/TV Shows/The Bay - Series 3/The Bay - Series 3.s03e01.Episode 1.%(ext)s", "--verbose"] DEBUG: youtube-dl environment: ["PYTHONPATH": "/Applications/Get iPlayer Automator.app/Contents/Resources", "PATH": "/Applications/Get iPlayer Automator.app/Contents/MacOS:/Applications/Get iPlayer Automator.app/Contents/Resources/get_iplayer/utils/bin:/usr/bin"] [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: [u'https://www.itv.com/hub/the-bay/2a5041a0013', u'-f', u'mp4/best', u'-o', u'/Users/andrew/Movies/TV Shows/The Bay - Series 3/The Bay - Series 3.s03e01.Episode 1.%(ext)s', u'--verbose'] [debug] Encodings: locale US-ASCII, fs utf-8, out None, pref US-ASCII [debug] youtube-dl version 2021.04.26-gia_01 [debug] Python version 2.7.16 (CPython) - Darwin-20.6.0-x86_64-i386-64bit [debug] exe versions: ffmpeg 4.3.2-tessus [debug] Proxy map: {} [debug] Using fake IP 25.9.71.8 (GB) as X-Forwarded-For. [ITV] 2a5041a0013: Downloading webpage

[skovatch]

Sorry, but I'm unable to reproduce this so far. I'm downloading the show right now. Is this a restart of a download? If so try deleting the partial download and restarting it.

[g0vgs]

Mine has been working fine until I tried to download the latest Vera, when I got stuck at Downloading. The Vera Series 11 directory was created but nothing else. It is now working with a proxy, so I wonder if it is IP address related regarding a quota at ITV or something?

[adbeckett]

I left it when I went out this morning and it eventually downloaded. Unfortunately whilst I scrolled back to see if I could see how it started, it appeared OK - but it had probably just failed and re-tried. So I added the rest of the series (episodes 2 and 3 don't show up if you search for "The Bay" so I have to use the "Use Current Webpage" approach for those, with 4,5,6 from the search. Anyway, Episodes 2 and 3 downloaded with no issues, then Episode 4 hung at the same point before eventually failing with:

Downloading Show 3/5:

Downloading The Bay: Series 3 INFO: Preparing Request for Auth Info INFO: Requesting Metadata. DEBUG: Metadata response status code: 200 INFO: Metadata processed. DEBUG: youtube-dl args:["https://www.itv.com/hub/the-bay/2a5041a0016", "-f", "mp4/best", "-o", "/Volumes/Media Library/TV Shows/The Bay - Series 3/The Bay - Series 3.s03e04.Episode 4.%(ext)s", "--verbose"] DEBUG: youtube-dl environment: ["PYTHONPATH": "/Applications/Get iPlayer Automator.app/Contents/Resources", "PATH": "/Applications/Get iPlayer Automator.app/Contents/MacOS:/Applications/Get iPlayer Automator.app/Contents/Resources/get_iplayer/utils/bin:/usr/bin"] [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: [u'https://www.itv.com/hub/the-bay/2a5041a0016', u'-f', u'mp4/best', u'-o', u'/Volumes/Media Library/TV Shows/The Bay - Series 3/The Bay - Series 3.s03e04.Episode 4.%(ext)s', u'--verbose'] [debug] Encodings: locale US-ASCII, fs utf-8, out None, pref US-ASCII [debug] youtube-dl version 2021.04.26-gia_01 [debug] Python version 2.7.16 (CPython) - Darwin-19.6.0-x86_64-i386-64bit [debug] exe versions: ffmpeg 4.3.2-tessus [debug] Proxy map: {} [debug] Using fake IP 25.166.152.16 (GB) as X-Forwarded-For. [ITV] 2a5041a0016: Downloading webpage ERROR: Unable to download webpage: ('The read operation timed out',) (caused by SSLError('The read operation timed out',)); please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; type youtube-dl -U to update. Be sure to call youtube-dl with the --verbose flag and include its complete output. File "/Applications/Get iPlayer Automator.app/Contents/Resources/youtube-dl/youtube_dl/extractor/common.py", line 634, in _request_webpage return self._downloader.urlopen(url_or_request) File "/Applications/Get iPlayer Automator.app/Contents/Resources/youtube-dl/youtube_dl/YoutubeDL.py", line 2279, in urlopen return self._opener.open(req, timeout=self._socket_timeout) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 429, in open response = self._open(req, data) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 447, in _open '_open', req) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 407, in _call_chain result = func(*args) File "/Applications/Get iPlayer Automator.app/Contents/Resources/youtube-dl/youtube_dl/utils.py", line 2737, in https_open req, **kwargs) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1201, in do_open r = h.getresponse(buffering=True) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 1121, in getresponse response.begin() File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 438, in begin version, status, reason = self._read_status() File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 394, in _read_status line = self.fp.readline(_MAXLINE + 1) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 480, in readline data = self._sock.recv(self._rbufsize) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 754, in recv return self.read(buflen) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 641, in read v = self._sslobj.read(len) youtube-dl finished downloading

Episodes 5 and 6 downloaded OK.

So that left just episode 3 with a problem. After it failed and was in the timer waiting to try again, I hit stop, and then start again, and this time it succeeded.

It appears there is an intermittent time out on the "Downloading webpage" stage, which probably explains why it worked OK for @skovatch and maybe the same for @g0vgs .

Andrew

[robincm]

I have tried ALL episodes of the Bay and GiA keeps stopping just after the line "using Fake IP ........" with the comment "Downloading webpage" and then no further activity. Stopping and restarting gives no further progres

[skovatch]

youtube-dl from the command line isn't able to fetch anything from ITV either, so this isn't specifically GiA-related. It gets stuck at the "Downloading Webpage" step, which is odd considering that's just HTML.

[gadgetchnnel]

I saw this comment in the youtube-dl repo which mentioned using --user-agent "Mozilla/5.0".

I just tested that from the command line with the version of youtube-dl bundled with GIA and was able to successfully download an ITV programme - without the option it hangs.

Edited: Since youtube-dl supports using a config file for default settings, as a workaround, you can create a file in ~/.config/youtube-dl/config containing the line --user-agent "Mozilla/5.0" to allow GIA to work.

[robincm]

At the risk of seeming totally useless would you be so kind as to explain in simple terms just how to set up the config file? Thanks

[gadgetchnnel]

@robincm To quickly do this from Terminal run these commands:

sudo mkdir -p  ~/.config/youtube-dl
echo '--user-agent "Mozilla/5.0"' | sudo tee -a ~/.config/youtube-dl/config

The first command will create the youtube-dl directory if it does not already exist, and the second will append the user agent option to the config file (creating the file if it doesn't exist).

[jeanpaulz85]

It does not work

On Jan 23, 2022, at 16:41, Steven Rollason @.***> wrote:

mkdir -p ~/.config/youtube-dl echo '--user-agent "Mozilla/5.0"' | sudo tee -a ~/.config/youtube-dl/config

[gadgetchnnel]

Do you mean that those commands didn't work, or downloading still didn't work after running them? I've successfully downloaded with that applied to the config file, so try:

• Running cat ~/.config/youtube-dl/config to check that the config file exists and contains that option
• Restarting GIA, as it may not pick up the config setting until it is restarted

[jeanpaulz85]

I did exactly what you suggested still no downloading from ITV

On Jan 23, 2022, at 16:41, Steven Rollason @.***> wrote:

@robincm https://github.com/robincm To quickly do this from Terminal run these commands:

mkdir -p ~/.config/youtube-dl echo '--user-agent "Mozilla/5.0"' | sudo tee -a ~/.config/youtube-dl/config The first command will create the youtube-dl directory if it does not already exist, and the second will append the user agent option to the config file (creating the file if it doesn't exist).

— Reply to this email directly, view it on GitHub https://github.com/Ascoware/get-iplayer-automator/issues/370#issuecomment-1019510290, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGWP7IVECMQMPQWG6TKP3ELUXQOS3ANCNFSM5MHY5ZJQ. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you are subscribed to this thread.

[gadgetchnnel]

Are you on the latest version of GIA? I believe there were other ITV related fixes previously, so you may also need them for it to work.

[jeanpaulz85]

GIA Version 1.21.12 (20210729001)

On Jan 23, 2022, at 17:25, Steven Rollason @.***> wrote:

Are you on the latest version of GIA? I believe there were other ITV related fixes previously, so you may also need them for it to work.

— Reply to this email directly, view it on GitHub https://github.com/Ascoware/get-iplayer-automator/issues/370#issuecomment-1019518960, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGWP7ISPMV4EIXPQREP474TUXQTWBANCNFSM5MHY5ZJQ. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you commented.

[gadgetchnnel]

If you've tried any of the previous fixes, such as creating a symbolic link pointing to Python within the GIA application bundle, you may need to undo those (or even delete GIA and re-download the latest version) as they may be interfering.

[robincm]

sorry! it's me again! Each time I enter one of the commands & press return all I get is another copy of the line I have just typed!! Think I am doing something wrong!! - (I have never used Terminal before)

[stoney27]

@robincm Can you attach a screenshot or cut and past the output you are getting in your terminal?

[robincm]

@stoney27 Thanks for helping. Here are 2 screenshots - first was last night's outputs while 2nd is what I got this morning!

[zugernaut]

I successfully followed the instructions for creating the config file (thanks for this!), but it has sadly not resolved the issue of hanging on the web-page download and ultimately timing out. Have tried restarting GIA and a full reboot, to no avail. Sigh!

[stoney27]

@robincm Not sure what is the problem is on the first screenshot, what was the command just before the terminal ask for your password? But the second is a missing ' at the end of the echo command before the pipe which is this | character. You might have better luck cutting and pasting the commands from gadgetchnnel's post it might be easier. I use the Terminal every day and still cut and past commands :)

[robincm]

Victory!! Thankyou so much @stoney27 The Bay is now downloading!

[stoney27]

Glad I could help.

-S

On Jan 24, 2022, at 7:16 AM, robincm @.***> wrote:

Victory!! Thankyou so much @stoney27 https://github.com/stoney27 The Bay is now downloading!

— Reply to this email directly, view it on GitHub https://github.com/Ascoware/get-iplayer-automator/issues/370#issuecomment-1020037848, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANSRN6MKT7XNBDN2VUP5QDUXU7IPANCNFSM5MHY5ZJQ. You are receiving this because you were mentioned.

[g0vgs]

Not tried to download anything since my OP. Just tried to get Trigger Point off ITV now and it just sits there until it times out. Turned on my VPN to London and it’s downloading right now!

On 24 Jan 2022, at 12:21, Scott Stonefield @.***> wrote:

Glad I could help.

-S

On Jan 24, 2022, at 7:16 AM, robincm @.***> wrote:

Victory!! Thankyou so much @stoney27 https://github.com/stoney27 The Bay is now downloading!

— Reply to this email directly, view it on GitHub https://github.com/Ascoware/get-iplayer-automator/issues/370#issuecomment-1020037848, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANSRN6MKT7XNBDN2VUP5QDUXU7IPANCNFSM5MHY5ZJQ. You are receiving this because you were mentioned.

— Reply to this email directly, view it on GitHub https://github.com/Ascoware/get-iplayer-automator/issues/370#issuecomment-1020041640, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUAPFYYFWS7IDWHKIV23B33UXU74BANCNFSM5MHY5ZJQ. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you were mentioned.

[zugernaut]

The new release of GIA appears to have done the trick! Downloads seem to working again. Thanks, Scott.

[g0vgs]

Definitely fixed. Many thanks :)

On 25 Jan 2022, at 10:58, zugernaut @.***> wrote:

The new release of GIA appears to have done the trick! Downloads seem to working again. Thanks, Scott.

— Reply to this email directly, view it on GitHub https://github.com/Ascoware/get-iplayer-automator/issues/370#issuecomment-1021061460, or unsubscribe https://github.com/notifications/unsubscribe-auth/AUAPFYYX7FBWJ6OXZWKIXX3UXZ66BANCNFSM5MHY5ZJQ. Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub. You are receiving this because you were mentioned.

[skovatch]

Fixed in v1.21.13