GetiPlayerAutomator / get-iplayer-automator

Moved to https://github.com/Ascoware/get-iplayer-automator! The goal of Get iPlayer Automator is to allow iTunes and your Mac to become the hub for your British Television experience regardless of where in the world you are. Currently, Get iPlayer Automator allows you to download and watch BBC and ITV shows on your Mac. Series-Link/PVR functionality ensures you will never miss your favourite shows. Programmes are fully tagged and added to iTunes automatically upon completion. It is simple and easy to use, and runs on any machine running Mac OS X 10.7 or later. And since the shows are in iTunes, it is extremely easy to transfer them to your iPod, iPhone, or Apple TV allowing you to enjoy your shows on the go or on your television.

https://github.com/Ascoware/get-iplayer-automator

GNU General Public License v3.0

127 stars 54 forks source link

Custom Proxy not working in GIA #312

Closed willson556 closed 10 years ago

willson556 commented 10 years ago

From kontrabe...@gmail.com on December 21, 2013 17:17:05

I'm in way over my level of tech knowledge and would appreciate any help you can give. I have searched through existing issues for help, but haven't found anything that will solve my problem.

I was using GIA very successfully until about a week ago when the provided proxy failed. I read through the info here and decided to get access to a private proxy in the UK. The proxy requires a user id and password.

I have set up a network location in preferences for my computer (using locations under the apple icon) with the necessary information for the custom proxy. When I select that custom location, using either chrome or safari , I can successfully (1) watch programmes online on BBC or ITV iPlayer and (2) download to the BBC iPlayer Downloads. I cannot, however, download using GIA. What steps will reproduce the problem? I have tried several scenarios in GIA.

I select my private proxy location in my network preferences. I select “provided” proxy in GIA preferences. I get a message asking me if GIA can use the confidential info for my private proxy’s IP address. I allow this, the proxy is tested and I get the following message: “Failed to load http://www.google.com within 30 seconds Using proxy: http://62.253.249.22:8080 Error: The request timed out.”
I select my private proxy location in my network preferences. I select “ custom” proxy in GIA preferences and enter as xxx.xxx.xxx.xx. I never get asked for a password and I get the following message: “The proxy you specified could not be contacted. […] If you are outside the UK, please either specify a new proxy of your own or use the one that is provided.”
I get the same result as 2, using either Safari or Chrome.
I get the same result as 1, entering the IP address for my custom proxy as: xxx.xxx.xxx.xx xxx.xxx.xxx.xx:808 xxx.xxx.xxx.xx:1080 xxx.xxx.xxx.xx:8080
I don’t select my private proxy location in my network preferences, I select “ custom” proxy in GIA preferences and enter as xxx.xxx.xxx.xx. I get the same result as 2 above, whether using Chrome or Safari. If I specifiy a port (.808, for example) after the IP address, I get prompted to use the password in my keychain, and get the result in 1 above.
I select my private proxy location in my network preferences. I select “none” as my proxy in GIA preferences. I get the following message: “The show could not be located on Iplayer/ITV Player. If outside the UK, please use a proxy.” What is the expected output? What do you see instead? Expect to be connected to custom proxy, but can’t seem to connect and download via GIA. Selecting custom location in network preferences does allow me to watch programmes on BBC and ITV, so believe private proxy is correctly set up in network preferences. What version of the product are you using? On what operating system? I am outside the UK and am using the following: MAC: OS X 10.7.5 GIA 1.5.8 Chrome 31.0.1650.63 Safari 6.1.1 Please let me know if any other information would be helpful.

What am I doing wrong??? I’m very grateful for your time and patience!

Original issue: http://code.google.com/p/get-iplayer-automator/issues/detail?id=320

willson556 commented 10 years ago

From dinkypumpkin on December 21, 2013 10:09:50

Correction: In all scenarios, custom proxy URL (incl. credentials) must be set in GiA preferences for BBC downloads to work since get_iplayer doesn't have access system proxy settings. I just saw "ITV Player" in your report and didn't think outside that box. Sorry about that.

To test: Delete proxy from network preferences. I think will also remove the associated credentials from keychain. Set GiA proxy to Custom and enter the full proxy URL (incl, credentials). Then try to download a single BBC programme using the instructions in wiki: VerboseLog . Quit and restart GiA, then try to download a single ITV programme using the same instructions. Attach both logs to your reply.

willson556 commented 10 years ago

From kontrabe...@gmail.com on December 21, 2013 10:22:00

Thank you for all suggestions! Am testing and saving logs and will post shortly.

I will try this new test, but am not sure what you mean by enter the "full proxy URL (incl, credentials)". I only have an IP address as xxx.xxx.xxx.xx. Do I somehow need to format as a url and include the user id and password?? Sorry for basic question and appreciate your patience.

willson556 commented 10 years ago

From dinkypumpkin on December 21, 2013 10:27:57

Also: By full proxy URL, I mean the port number must be included as well.

willson556 commented 10 years ago

From kontrabe...@gmail.com on December 21, 2013 10:37:43

I understand now - thank you. Also, should I be clicking "Use Proxy for All Activities"?

Many thanks!

willson556 commented 10 years ago

From dinkypumpkin on December 21, 2013 10:55:02

If your proxy provider didn't give you a port number with the server IP address, then omit it. It will default to using port 80 (for HTTP). If you have a secure proxy connection, you use https: in the URL and your provider should have given you the port number (typically 443). And yes, you have to embed the username and password in the URL. Google for the format.

"Use Proxy for All Activities" is only required in rare circumstances, so don't use it unless I see that your tests indicate you need to do so.

One other note: You also have the option of setting HTTP proxy preferences only in your web browser rather making it a system-wide setting. It might be better to add the proxy to your browser (rather than network preferences) and only flip it on when you need it. If you only use the proxy for GiA, then you don't need to set it anywhere except GiA preferences. I wanted to be clear about that since my first response was incomplete.

willson556 commented 10 years ago

From kontrabe...@gmail.com on December 21, 2013 10:55:09

When I try a BBC Programme, I get the following:

Failed to load http://www.google.com within 30 seconds Using proxy: http://xxx.xxx.xxx.xx:xxx Error: Authentication needed

willson556 commented 10 years ago

From kontrabe...@gmail.com on December 21, 2013 11:10:15

Getting closer, but not there yet. Since I have a feeling user error is involved, let me clarify what I am doing. I am entering my custom proxy as:

https://user:password@xxx.xxx.xxx.xx:###

I was provided a different port number for http, so I have been using that. If that is what I should be entering for the custom proxy, then I will post the log.

willson556 commented 10 years ago

From kontrabe...@gmail.com on December 21, 2013 11:22:07

Ok,

If I enter, "https://user:password@xxx.xxx.xxx.xx I don't get any errors, but nothing downloads.

If I enter, "https://user:password@xxx.xxx.xxx.xx:### with the http port number I was given, I get "Failed to load, unknown error".

Which log(s) would you like me to send?

willson556 commented 10 years ago

From kontrabe...@gmail.com on December 21, 2013 11:32:24

OOOO - ITV success. I got an error message using https://user:password@xxx.xxx.xxx.xx:### but it then went on to download anyway.

willson556 commented 10 years ago

From dinkypumpkin on December 21, 2013 11:34:48

The log in #8 indicates that your credentials still weren't included in the proxy URL, unless you deleted them from the log. If not, quit and restart GiA and then recheck the setting to make sure it sticks. The log also shows you trying 3 different port numbers. You only need to test the one given to you by the proxy provider. The log also shows you using http:, but #9 and #10 indicate you're using https: in the URL. Which is correct?

Feel free to attach both logs. I suspect only the latter is relevant, but it can't hurt to look at both.

CAUTION: Be sure to change all instances of your proxy username and password in the log file before posting. The prosy URL may appear a number of times, so do a search-and-replace on the entire file.

willson556 commented 10 years ago

From kontrabe...@gmail.com on December 21, 2013 11:47:42

And then I tried doing the same BBC. Got error, told it to proceed anyway. Nothing downloaded. Log attached.

willson556 commented 10 years ago

From dinkypumpkin on December 21, 2013 12:07:48

It's clearly working with ITV, even though the proxy test fails. That's a bit curious, but it may just be that Google blocks your proxy service. You can use a hidden preference to set a different proxy test target URL. See wiki: HiddenPrefs .

As for BBC, the log indicates that get_iplayer seems to hang waiting for a response from the BBC server, which is very unusual. There should be more in the log, even if your proxy is denied access to BBC content. It's possible that the BBC blocks your proxy service. One more test may useful: Try a BBC download with "Use Proxy for All Activities" set and post the verbose log. That would help to diagnose whether or not the proxy is being blocked.

willson556 commented 10 years ago

From kontrabe...@gmail.com on December 21, 2013 12:29:56

I tried to figure out the hidden preferences using information you linked. I just couldn't figure what enter into the terminal to find the defaults. Sorry - I think I've hit the limits of my tech capabilities. If you tell me what to enter into the terminal, I will try it and see what I get.

I tried #13 using proxy for all activities. I got the following error: Failed to load http://www.google.com within 30 seconds Using proxy: https://[deleted] Error: Unknown errorLog attached.

I told it to continue anyway. It does the 0.0% - initialising at the bottom, but never retrieves the metadata and nothing downloads. I cancelled after a couple minutes and attached the log.

willson556 commented 10 years ago

From kontrabe...@gmail.com on December 21, 2013 12:32:41

I just tried specifying Safari in GIA preferences and got the same result.

willson556 commented 10 years ago

From kontrabe...@gmail.com on December 21, 2013 12:44:04

I also went into Chrome and restored all default settings and relaunched. Same deal.

willson556 commented 10 years ago

From dinkypumpkin on December 21, 2013 12:49:30

15: That indicates to me that the BBC is blocking your proxy service. Turn off "Use Proxy for All Activities" and then try a few more ITV downloads and a few more BBC downloads. If all the ITV downloads work and none of the BBC downloads work, that would tend to confirm that you're being blocked by the BBC.

If you're not comfortable working at the command prompt, don't worry about setting a different browser test URL via preferences. You can just disable the browser test in GiA preferences (Advanced).

16: The default browser setting in GiA preferences has nothing to do with proxies, so reset to whatever browser you normally use.

willson556 commented 10 years ago

From dinkypumpkin on December 21, 2013 12:55:31

An alternative explanation is that your proxy service is oversubscribed and can't handle your connection attempts. Again, testing a few more ITV and BBC downloads should help. If ITV downloads hang as well, then your successful attempt above could have been a fluke and your proxy service is the problem. Very little data is returned from proxied requests, so it's not likely to be a bandwidth issue.

willson556 commented 10 years ago

From dinkypumpkin on December 21, 2013 13:10:00

You may also want to test your proxy completely outside of GiA and get_iplayer. Open a command prompt in Terminal and run the command below, inserting your proxy URL where indicated:

curl -L -v -x https://user:pass@xxx.xxx.xxx.xxx:### http://www.bbc.co.uk/mediaselector/4/mtis/stream/b03mnjbq That command attempts to access the resource that hung your BBC download in #13. If it doesn't return a big pile of XML text, then the proxy can't connect or is blocked. Feel free to attach the output if you're not sure (copy and paste from Terminal)

willson556 commented 10 years ago

From kontrabe...@gmail.com on December 21, 2013 14:01:38

Re Comment #19

All 3 ITV programmes downloaded, but not the BBC programme. Just hung like before and never downloads. IF it is the proxy, then why would I have been able to download directly from Chrome to iPlayer Downloads when I originally set my network location to the proxy? Completely crazy making. Thanks again so much for all your help.

willson556 commented 10 years ago

From dinkypumpkin on December 21, 2013 14:11:15

21: Attach the entire output so I can make sure. There's gibberish and then there's the correct gibberish.

22: iPlayer access from your browser works differently than get_iplayer (and thus GiA). The download facility is also different. Those services use resources unavailable to get_iplayer.

willson556 commented 10 years ago

From dinkypumpkin on December 21, 2013 14:44:42

One more thing: delete comment #21. I won't specify why precisely, but trust me on this one.

willson556 commented 10 years ago

From dinkypumpkin on December 21, 2013 14:49:24

And when you post the curl command output, only post the XML payload (the gibberish)

willson556 commented 10 years ago

From kontrabe...@gmail.com on December 21, 2013 15:19:36

24 Thank you.

25 Payload attached.

willson556 commented 10 years ago

From dinkypumpkin on December 21, 2013 16:11:55

I'm afraid I've run out of ideas. You have the correct proxy URL format, and it is configured correctly in GiA. We know this because ITV downloads work. Your proxy doesn't seem to be blocked by the BBC. We know this because the curl command retrieved the expected XML file. I don't know why get_iplayer appears to hang when using the proxy. It works with many, many different proxy services. I wasn't sure from earlier comments, but if you have a choice between http: and https: (non-secure and secure) proxy URLs, try them both.

You may also want to test get_iplayer outside of GiA to see if GiA itself may be the problem. Copy the command below to a prompt in Terminal and run it, inserting your proxy URL where indicated.

/usr/bin/perl "/Applications/Get iPlayer Automator.app/Contents/Resources/get_iplayer.pl" --profile-dir "$HOME/Library/Application Support/Get iPlayer Automator/" --pid b03mnjbz --test --verbose --partial-proxy --proxy https://user:pass@xxx.xxx.xxx.xxx:###

There is no need to post the results. If the command completes, the last few lines of output should look like:

INFO: Trying flashvhigh1 mode to record tv: Tudor Monastery Farm: Series 1 - Episode 6 INFO: Attempt number: 1 / 50 INFO: File name prefix = Tudor_Monastery_Farm_Series1-_Episode_6_b03mnjbz_default
DEBUG: Record using flashvhigh1 mode return code: 'skip' INFO: skipping this programme

willson556 commented 10 years ago

From kontrabe...@gmail.com on December 21, 2013 16:23:37

Thank you so much for all your time and persistence!

I copied the prompt and filled in the proxy info. I got this as a result:

Can't open perl script "/Applications/Get iPlayer Automator.app/Contents/Resources/get_iplayer.pl": No such file or directory

???

willson556 commented 10 years ago

From dinkypumpkin on December 21, 2013 16:52:50

You must not have GiA installed in the usual spot (/Applications). Adjust the get_iplayer.pl path as necessary, replacing "/Applications" with the correct value.

willson556 commented 10 years ago

From kontrabe...@gmail.com on December 21, 2013 17:19:22

Aha. Fixed that. Got back a bunch of info, but not the info you mentioned in #27.

Attachment: File 1.txt

willson556 commented 10 years ago

From dinkypumpkin on December 21, 2013 17:27:45

Looks like you put a colon after --proxy. Should only be a space between --proxy and the URL.

willson556 commented 10 years ago

From fly...@onetel.com on December 21, 2013 17:36:37

Hi Sorry to butt in but I am also having problems. I have read through all this super info & have tried your suggested remedy but the last few lines of output I get are:

Matches: 1182: Tudor Monastery Farm: Series 1 - Episode 6, BBC Two, Audio Described,Factual,History,Religion & Ethics,TV, default,audiodescribed

INFO: 1 Matching Programmes INFO: Programme not in history INFO: iPlayer metadata URL = http://www.bbc.co.uk/iplayer/playlist/b03mnjbz INFO: Disabled proxy: http://62.253.249.22:8080 INFO: Getting page http://www.bbc.co.uk/iplayer/playlist/b03mnjbz INFO: Restored proxy to http://62.253.249.22:8080 INFO: Using Not Live standard TV and Radio: b03mnjbq INFO: Version: default, VersionPid: b03mnjbq, Duration: 3540 INFO: Programme not in history INFO: Will search for versions: default WARNING: Not using flash modes since rtmpdump is not found INFO: Mode list: INFO: Checking existence of default version INFO: Version = default INFO: Getting page http://www.bbc.co.uk/mediaselector/4/mtis/stream/b03mnjbq?cb=03513 WARNING: Not using flash modes since rtmpdump is not found INFO: No specified modes () available for this programme with version 'default' (try using --modes=)

But I'm not sure what it all means!! Thanks for all your assistance, Robin

willson556 commented 10 years ago

From dinkypumpkin on December 21, 2013 17:58:44

32: Ack. You reminded me that my instructions in #27 are wrong. My machine has a standalone get_iplayer installation, but your machines won't. If you have working proxy, the end of the output from that command in #27 will look roughly like:

INFO: Getting page http://www.bbc.co.uk/mediaselector/4/mtis/stream/b03mnjbq?cb=88528 WARNING: Not using flash modes since rtmpdump is not found ...above line repeated several times... INFO: Found mode flashhd1: (pc_streaming_hd) rtmp h264 1280x720 2800kbps stream (CDN: akamai/10) ...many more "Found mode" messages... INFO: No specified modes () available for this programme with version 'default' (try using --modes=flashhd,flashhigh,flashlow,flashstd,flashvhigh,rtsphd,rtsphigh,rtsplow,rtspstd,rtspvhigh,subtitles)

The output in #32 comes from using the broken GiAprovided proxy. When it hits:

INFO: Getting page http://www.bbc.co.uk/mediaselector/4/mtis/stream/b03mnjbq?cb=03513 it eventually times out and the last 2 lines are printed. nicole's proxy doesn't seem to time out within a reasonable period and never gets past that point. The thing I'm trying to determine is whether that is intrinsic to get_iplayer or unique to get_iplayer running within GiA. If it's the latter, then the command line might provide a workaround.

willson556 commented 10 years ago

From dinkypumpkin on December 31, 2013 09:05:41

Status: Done