GetiPlayerAutomator / get-iplayer-automator

Moved to https://github.com/Ascoware/get-iplayer-automator! The goal of Get iPlayer Automator is to allow iTunes and your Mac to become the hub for your British Television experience regardless of where in the world you are. Currently, Get iPlayer Automator allows you to download and watch BBC and ITV shows on your Mac. Series-Link/PVR functionality ensures you will never miss your favourite shows. Programmes are fully tagged and added to iTunes automatically upon completion. It is simple and easy to use, and runs on any machine running Mac OS X 10.7 or later. And since the shows are in iTunes, it is extremely easy to transfer them to your iPod, iPhone, or Apple TV allowing you to enjoy your shows on the go or on your television.
https://github.com/Ascoware/get-iplayer-automator
GNU General Public License v3.0
127 stars 54 forks source link

Cannot download BBC programmes from outside UK: Proxies do not work, VPNs blocked #483

Closed pstibbons closed 7 years ago

pstibbons commented 9 years ago

What steps will reproduce the problem? The usual procedure for downloading.

What is the expected output? What do you see instead? A successful download. It doesn't download successfully.

What version of GiA are you using? On what version of OS X? 1.8.5. 10

What programme are you attempting to download (full title, series and episode)? Mostly BBC content.

If you cannot download a programme, provide a verbose log of the attempt (see Verbose Log in wiki). See screenshot

Please provide any additional information below. I made an attempt to follow the cleaninstall procedure, but the results were the same. image image

dinkypumpkin commented 9 years ago

Provided proxy does not work. Clean install will not help. You must use your own VPN. You cannot rely on the provided proxy.

dinkypumpkin commented 9 years ago

To anyone thinking of posting here that you are receiving the same error - don't do it. "Me too" posts do not help. This issue described above is a known problem that may never be fixed. Save yourself some headache: Find your own VPN and set Proxy = None in GiA preferences. You cannot rely on the provided proxy.

pstibbons commented 9 years ago

That's good to know. Thanks.

On Sunday, May 24, 2015, dinkypumpkin notifications@github.com wrote:

To anyone thinking of posting here that you are receiving the same error - don't do it. "Me too" posts do not help. This issue described above is a known problem that may never be fixed. Save yourself some headache: Find your own VPN and set Proxy = None in GiA preferences. You cannot rely on the provided proxy.

— Reply to this email directly or view it on GitHub https://github.com/GetiPlayerAutomator/get-iplayer-automator/issues/483#issuecomment-105063005 .

pstibbons commented 9 years ago

By the way, can you recommend VPN sources. Thanks.

On Sunday, May 24, 2015, Ponder Stibbons pstibbons8@gmail.com wrote:

That's good to know. Thanks.

On Sunday, May 24, 2015, dinkypumpkin <notifications@github.com javascript:_e(%7B%7D,'cvml','notifications@github.com');> wrote:

To anyone thinking of posting here that you are receiving the same error

  • don't do it. "Me too" posts do not help. This issue described above is a known problem that may never be fixed. Save yourself some headache: Find your own VPN and set Proxy = None in GiA preferences. You cannot rely on the provided proxy.

— Reply to this email directly or view it on GitHub https://github.com/GetiPlayerAutomator/get-iplayer-automator/issues/483#issuecomment-105063005 .

chewitt commented 9 years ago

http://lmgtfy.com/?q=uk+vpn+and+proxy+services

pstibbons commented 9 years ago

I thought that. Cheers.

On Sunday, May 24, 2015, Christian Hewitt notifications@github.com wrote:

http://lmgtfy.com/?q=uk+vpn+and+proxy+services

— Reply to this email directly or view it on GitHub https://github.com/GetiPlayerAutomator/get-iplayer-automator/issues/483#issuecomment-105111998 .

JTKGlasgow commented 9 years ago

Ok, let me ask the same question a different way. Can anyone recommend a VPN source that works well with GiA? I have tried a couple VPN providers in the past, but was not able to get either of them to work. If anyone could clarify which VPN service(s) they have used and how the set them up to work with GiA, I'd be grateful.

pstibbons commented 9 years ago

That would very helpful, to be sure. I tried one today. It didn't work out. It was called Private Internet Access or PIA.

On Monday, May 25, 2015, JTKGlasgow <notifications@github.com javascript:_e(%7B%7D,'cvml','notifications@github.com');> wrote:

Ok, let me ask the same question a different way. Can anyone recommend a VPN source that works well with GiA? I have tried a couple VPN providers in the past, but was not able to get either of them to work. If anyone could clarify which VPN service(s) they have used and how the set them up to work with GiA, I'd be grateful.

— Reply to this email directly or view it on GitHub https://github.com/GetiPlayerAutomator/get-iplayer-automator/issues/483#issuecomment-105325365 .

pstibbons commented 9 years ago

By the way, each time I try the CleanInstall procedure, it fails at line

  1. This the bit that is supposed to leave you with a reset that never materializes. There seems to be a missing step. Any thoughts.

On Monday, May 25, 2015, Ponder Stibbons pstibbons8@gmail.com wrote:

That would very helpful, to be sure. I tried one today. It didn't work out. It was called Private Internet Access or PIA.

larryy commented 9 years ago

I highly recommend BolehVPN.net. First, they're extremely reliable and the connection basically never drops. Second, while they offer normal all-traffic-goes-through-the-tunnel mode, they also offer a unique (as far as I know) local proxy mode that you can use to route just GiA (or just GiA and one particular browser, or...) through the VPN tunnel and leave the rest of your network traffic alone. Basically, their software creates a local IP address, 10.10.10.1:808, and you tell GiA it's using a proxy at that address, but it's really going over the VPN tunnel. All other browsing and network services work normally (which is important to me since the same machine that runs GiA serves my personal web domain and cannot be on a VPN). All downloads on BBC and ITV work well.

(I have nothing to do with BolehVPN.net, except being a happy customer. When I first asked for and received a free trial they had this special proxy-mode working with a number of countries, but not the UK. I mentioned I needed it in the UK for the service to do me any good, and three days later they had it up and running for me. So their customer service is great too. Though since I got it up and running I haven't had any need for their customer service, which is as it should be.)

pstibbons commented 9 years ago

Interesting comments below this article.

http://www.macupdate.com/app/mac/39019/get-iplayer-automator

On Tuesday, May 26, 2015, larryy notifications@github.com wrote:

I highly recommend BolehVPN.net. First, they're extremely reliable and the connection basically never drops. Second, while they offer normal all-traffic-goes-through-the-tunnel mode, they also offer a unique (as far as I know) local proxy mode that you can use to route just GiA (or just GiA and one particular browser, or...) through the VPN tunnel and leave the rest of your network traffic alone. Basically, their software creates a local IP address, 10.10.10.1:808, and you tell GiA it's using a proxy at that address, but it's really going over the VPN tunnel. All other browsing and network services work normally (which is important to me since the same machine that runs GiA serves my personal web domain and cannot be on a VPN). All downloads on BBC and ITV work well.

(I have nothing to do with BolehVPN.net, except being a happy customer. When I first asked for and received a free trial they had this special proxy-mode working with a number of countries, but not the UK. I mentioned I needed it in the UK for the service to do me any good, and three days later they had it up and running for me. So their customer service is great too. Though since I got it up and running I haven't had any need for their customer service, which is as it should be.)

— Reply to this email directly or view it on GitHub https://github.com/GetiPlayerAutomator/get-iplayer-automator/issues/483#issuecomment-105440817 .

Gedoe commented 9 years ago

I may have found a workaround for using the program outside the UK. At least, the automator is downloading as we speak. I used one of the free proxies from this page: http://spys.ru/free-proxy-list/GB/ I had to try a few proxies from this list before I got a working one. It seems that only the https proxies still work, not the http ones. At the moment this one works: 62.254.185.165:8080

I have also tested the Hola plugin for Firefox. Interestingly I could only watch HD content, not SD.

Vangelis66 commented 9 years ago

I used one of the free proxies from this page:

Revealing your source for "free" proxies on a public place like this is a bad idea for two reasons:

  1. Those so called "free" or "open" proxies that can be found on the net are the result of a process called "port scanning" and in fact are misconfigured / compromised systems; these by default have usually a finite available bandwidth and once people from all over the world start (ab)using them, they soon die down... Even before that happens, the increased traffic may alert their owners, who are quick to patch the vulnerabilities / misconfiguration....
  2. The BBC people have recently strengthened their geo-filtering schemes... In the last month or so I have found that fresh proxies from "trusted" sources are being quickly blacklisted for iPlayer purposes - one new, working, UK proxy may work the first day it appears on some proxy list, the next day (while it continues to function for other UK-only sites) may cease to work on iPlayer... This tells me that popular proxy lists (like the ones a Google search might return) are being monitored by BBC personnel, the contents of those lists are manually blacklisted...

It seems that only the https proxies still work, not the http ones.

HTTPS is not used for BBC iPlayer (and I doubt get_iplayer has support for https proxies), however the ITV programme URLs do use https... Another result from the recently enforced stricter geo-filtering is that simple "transparent" (or non-anonymous) UK proxies do not work anymore... You have better chance going for an anonymous or "high anonymous" (also called elite) UK proxy... Finally, some of the proxies on lists like the one you mentioned do not work simply because they belong to school or work environments, which block iPlayer (and other streaming services) through in-house filters...

mostlyotter commented 9 years ago

As far as I can tell, GiA will no longer download BBC SD television with a UK proxy. GiA will only download BBC SD TV with a UK VPN.

GiA will still download BBC HD television and radio as before, as well as ITV TV, with a working UK proxy. The provided proxy still seems not to be a working one.

I tried BolehVPN and set up GiA to use their proxied UK server. For me, this worked like any other proxy and GiA failed to download BBC SD television.

I've even wondered about putting Snow Leopard Server (licensed to run in a virtual machine) on a Linux VPS in the UK to run GiA, but I've no idea if these are crazy thoughts.

No one at the BBC should ever think that a Mac OS X user would rather wonder about words like Debian or CentOS than be allowed to pay a nonresident license fee, subscribe to a global iPlayer or do anything else that the BBC likes to get content that's considered "too British" to be sold to US residents.

Any UK residents who wonder what I mean by "too British" should go look at the BBC America schedule, which has had a minimum of ten hours a week of Star Trek TNG since 2010.

larryy commented 9 years ago

Thanks for the info, mostlyotter, and apologies for the misdirection on BolehVPN. I always download HD content, which, as you note, is working well currently. I didn't realize it was failing with SD, but I just confirmed your observation (to be certain it was the same with my setup).

If VPN ceases to work for HD content, then I'll be with you doing a VPS or collocate or something. I watch as much British programming as I do US, and can't imagine doing without. I wish QI, Have I Got (a Bit More) News for You, Charlie Brooker's Weekly Wipe, Never Mind the Buzzcocks (may it RIP), and Doctor Who were on every week, and am annoyed they didn't renew David Mitchell's Something I Said. There's lots of other brilliant comedy and occasional science fiction. I would happily pay the BBC a licensing fee, if they had a mechanism for authorizing users outside the UK. But with too many lawyers in the loop, I doubt that'll happen anytime soon.

pstibbons commented 9 years ago

I've run in to a novelty: I have BBC Radio and ITV, but no BBC television/movies. It could be the VPN I subscribed to. But there it is. No Telly.

Vangelis66 commented 9 years ago

@pstibbons wrote:

but no BBC television/movies

Nothing to do with your VPN, am afraid... get_iplayer (GiP), the perl script inside GiA which is responsible for BBC files (TV + Radio) WAS BROKEN TODAY (02/06/2015), after the BBC iPlayer team removed the (old) TV listing feeds that GiP relied upon to fetch TV... See posts within the GiP support forum:

https://squarepenguin.co.uk/all-forums/

and the GiP mailing list:

http://lists.infradead.org/pipermail/get_iplayer/2015-June/date.html

for more explanation... Wait until the code maintainer (dinkypumpkin) tries his best to bring back as much of the lost functionality as can be salvaged after today's mayhem...

Regards.

pstibbons commented 9 years ago

Thanks for the update.

dinkypumpkin commented 9 years ago

Reopening to avoid duplicates

pstibbons commented 9 years ago

That was accidental, sorry...

hoosiersunited commented 9 years ago

A couple of quick questions/comments on this:

  1. Will GiA be updated to address the changes from the BBC (dinkypumpkin, you do an awesome job so this is by no means a demand I am just wondering if there will be a fix)? Also, will this fix then allow GiA to work with proxies again, or is that simply the BBC's increased enforcement?
  2. Does GiA work with VPNs still? I would look to subscribe to one but want to make sure it works before spending some money on it.

Thanks - while I am conversational in techie I am not fluent and want to make sure I understand what we are facing.

dinkypumpkin commented 9 years ago

Edited 2015-06-07: Made clear that proxy issues apply to BBC TV.

Edited 2015-06-07: Added references to #486 detailing potential problem with Quick Caching and PVR.

Vangelis66 commented 9 years ago

@dinkypumpkin wrote:

None of the recent changes to get_iplayer has anything to do with proxies or non-UK access.

From the available evidence, GiA with proxies is mostly dead Judge from what people have posted here and #482

Hello DP When a working and not blacklisted (by the BBC) HTTP (UK) proxy is employed with GiP (hence GiA), then:

  1. BBC radio at the UK-only highest bitrates can be downloaded.
  2. BBC TV (VOD) can be downloaded in flashhd & flashlow tvmodes. (flashvhigh,flashhigh,flashstd,flashnormal tvmodes stopped being downloadable via proxy due to CDN changes by the BBC on May 18th!). This is after your recent fixes (GiP tag 2.94). And ITV downloads do still work in GiA, so I wouldn't say myself that "GiA with proxies is mostly dead"

(Princess Bride, anyone?). I'm not going to spend any more time on it.

In fact I do have a tested workaround that would re-enable SD BBC TV downloads via HTTP proxy, but that would involve modifying the get_iplayer perl script;

  1. Are you willing to implement it? I can't write code myself...
  2. If affirmative to question 1, please reach me via e-mail for further details - I help in the GiP mailing list as "Vangelis forthnet" you know who I am...
  3. if "2" is much of a chore, will you accept mail (and hopefully reply back) to the address you use(d) in the list? Believe me, the fix @hoosiersunited and others are asking for here is but a few lines of perl code away... It'd be a shame otherwise for people to fork out good money to subscribe to dubious VPN operators... I'll be waiting for a straightforward answer from you...

Kind regards

dinkypumpkin commented 9 years ago

OK, that should have read "GiA with proxies is mostly dead for BBC TV". Original comment edited. You can still get "Flash - HD" streams with proxy because one CDN is not yet geo-blocking, or isn't doing it very well. Who knows how long that will last. As some of you remember from a few months ago, when that CDN goes on the blink, there is no HD. You can still get "Flash - Low" TV streams with proxy for reasons unknown.

Vangelis66 commented 9 years ago

Thanks very much for your reply...

@dinkypumpkin commented:

You can still get "Flash - Low" TV streams with proxy for reasons unknown.

Well, the reason is not unknown at all... at least to me! If you inspect YOUR code more closely: http://git.infradead.org/get_iplayer.git/blobdiff/79b5a5efe98453077d4312e81b33a6122f30b6c9..098807de5aebc38fe4e80dabdaf0028af9b74de5:/get_iplayer you'll figure it out yourself, too! [hint: read again my previous comment at: https://github.com/GetiPlayerAutomator/get-iplayer-automator/issues/482#issuecomment-104807832]. Any answer on points 1./2./3. of my previous comment? I'm not self-motivated though, I fetch 95% radio and can compose my own rtmpdump commands for TV, if need be (once a month); I am a sharing person who wants to help people back (as you did help me over the years in the list...)

All the best

larryy commented 9 years ago

Vangelis66 - If you send me your tested workaround for SD downloading, with enough context to understand what changes you have made to get things working, I will see if I can integrate those changes into get_iplayer. I have only a passing familiarity with Perl, but have been programming in various languages for longer than I care to admit, and have at least stared at the get_iplayer code a tiny bit. I have temporarily enabled display of my email address on my profile, specifically so you can send me this information. If you're willing to share the information, please do; if not, please just let me know. I want to stop displaying my email address as soon as possible. Thanks in advance.

Vangelis66 commented 9 years ago

@larryy :

A long and very detailed e-mail has just been sent - please acknowledge reception here; if affirmative, you can proceed to hide e-mail address in profile at your convenience...

larryy commented 9 years ago

@Vangelis66 : Got it. Thank you! I've only skimmed it at this point, but it looks nice and thorough, and the logic around when to apply the different mediaset URLs seems perfectly sound. If I don't get to it sooner, I'll put some serious time into it this weekend.

larryy commented 9 years ago

@mostlyotter : An update and clarification on the VPN issue... Because we were both using BolehVPN in that proxied-VPN mode I described previously in this thread, GiA (and GiP) will use the proxied VPN like any other proxy to find the stream, and that works as advertised. But as @Vangelis66 kindly explained to me offline, rtmpdump, that GiP (and GiA) uses, does the actual data streaming via SOCKS4, not HTTP, and rtmpdump's SOCKS4 traffic will not be routed through GiA's proxy. That's why SD downloads fail with BolehVPN in proxied-UK mode, like they do with any other proxy. If you use the whole-machine VPN mode (I did some brief experimenting with the "xcloaked" mode) SD downloads work correctly. I just can't leave my download machine in whole-machine VPN mode all the time. But I put it in whole-machine VPN mode for a brief time and downloaded several SD programs I'd been trying to get successfully. So true VPN (from Boleh or probably any reliable service vendor) will indeed allow SD downloads.

I wonder if rtmpdump can be invoked with the right flags to use a proxy for its SOCKS4 streaming (without routing all network traffic through the VPN tunnel). I'm guessing not or the owners of GiP and GiA would have already configured them to work that way.

dinkypumpkin commented 9 years ago

For future visitors: rtmpdump can use SOCKS, but it does not by default. That wouldn't make sense.

Users would need to configure a SOCKS proxy (should GiA ever provide that capability), so it doesn't make sense to suggest that GiA could ever "work that way" by default. It also seems that Vangelis didn't tell you that get_iplayer has supported rtmpdump + SOCKS for years. Until somebody is motivated to surface the capability in GiA, you can configure its embedded get_iplayer to use a SOCKS proxy with rtmpdump if you wish.

larryy commented 9 years ago

Thanks for the clarification, @dinkypumpkin. I suspect @Vangelis66 was commenting on how to use SOCKS streaming with rtmpdump and I just misunderstood. The point remains that GiA's http proxy use is independent of rtmpdump's streaming (whether RTMP or SOCKS), which is why the proxied-VPN mode fails for SD and the whole-machine VPN mode succeeds. (And coupled with different behaviors from the various CDNs is why proxied-VPN mode is sufficient for HD content currently, though that could change any day.)

Vangelis66 commented 9 years ago

@dinkypumpkin wrote:

It also seems that Vangelis didn't tell you that get_iplayer has supported rtmpdump + SOCKS for years.

Hello... What I did in fact tell him was (copy/pasted verbatim from my e-mail:)

GiA (& GiP) uses the provided HTTP UK proxy to bypass the geo-filter imposed on this "mediaselector" URL... (snip) You have to realise that the download utility rtmpdump does not have support for HTTP proxy (only for Socks4), so it can't utilise the proxy in GiP/GiA - any attempts to connect to CDN servers are done via your original non-UK IP; the response of the FlashMediaServer (FMS) depends upon its configuration... (snip) When a UK VPN is used, all your box is virtually transported to the UK; rtmpdump can then connect to CDNs exposing the VPN's UK IP address, so the servers allow the connections to be established (this is how they are configured), so the download starts... Unfortunately, all rtmp traffic must pass through the VPN server, which means reduced speeds...

I am not a native speaker, but I feel that was crystal clear English, with little room (if any) left for misinterpretation... The above was sufficient for @larryy to realise why his SD BBC TV downloads were failing via his VPN-induced HTTP proxy and in order to get the files he had to switch to a "full" VPN mode... FWIW, many years ago a patch was posted on the rtmpdump mailing list (read this thread: http://lists.mplayerhq.hu/pipermail/rtmpdump/2010-August/001130.html ) that enables HTTP proxy support for rtmpdump (via the http_proxy environment variable), but sadly never made it on to the main official repo...

As far as Socks proxy support in GiP is concerned..., don't get me started! NOWHERE in GiP's --longhelp, Man Page, wiki, documentation etc. exists even a mere mention of "Socks" :-( Only the --proxy option is briefly covered... As you know very well, the talk of Proxy (be it HTTP or SOCKS) or VPN is strictly off-limits on the mailing list, SquarePenguin's Support Forum and the GitHub issue tracker you yourself maintain! One must be familiar enough with rtmpdump itself and the advanced GiP options to figure out that one can instruct rtmpdump (in GiP) to use a SOCKS4 proxy by adding in a CLI command something along the following line: --rtmp-tv-opts="-S proxyhost:proxyport" Furthermore, GiP lacks global support for SOCKS4 (that I am aware); --proxy (-p) is reserved only for HTTP proxies - this isn't the case with, say, get-flash-videos, where an incantation like: --proxy socks://proxyhost:proxyport redirects both HTTP & RTMP traffic through the SOCKS4 proxy... Coupled with the fact there are very few open (=free) UK SOCKS4 proxies on line, I'd dare say very few GiP users are aware "that get_iplayer has supported rtmpdump + SOCKS for years."

@ all buying VPN subscriptions: Be sure the default RTMP port (1935) is left open in the UK VPN server you connect to - while HTTP traffic goes through port 80 (and HTTPS through 443), RTMP traffic goes via port 1935; in GiP there's an option for rtmpdump to use port 443/80 ( --rtmpport 443 or --rtmpport 80), I however do not know how this could be set in GiA...

@willson556

As requested, an e-mail explaining everything in the finest detail has been sent...

@larryy

I know you have already bought a VPN subscription, but in your case, where your box is also your server and you don't want to use the "whole-machine VPN mode" on it, perhaps a much better solution (to the VPN) would be to buy an SSH connection (tunnel) to an SSH Server based in the UK. Through the SSH tunnel you can enable a SOCKS4/5 proxy connection to the remote server in the UK (https://www.mikeash.com/ssh_socks.html); usually the proxy is of the form: 127.0.0.1:1080 Only those apps on your machine that need to be proxied to the UK (browsers, downloaders etc.) and are compatible with SOCKS would have to be configured individually to use the proxy - the rest of your system can use its original internet connection... You can use proxy chaining to proxy apps that do not support SOCKS (like GiP). On Windows, I used Privoxy (Mac OS X version available), a local HTTP proxy with default port=8118, and edited its main configuration file by adding the line: forward-socks5 / 127.0.0.1:1080 . This actually forwards the HTTP traffic Privoxy receives to the SOCKS proxy, itself forwarding it to the SSH server in the UK. So, you can use GiP to download a "flashvhigh" tvmode like this:

get_iplayer -p "http://localhost:8118" --partial-proxy --pid=xxxxxxxx --modes=flashvhigh --rtmp-tv-opts="-S localhost:1080" --force

And while you stay connected in the SSH tunnel, you can still use Privoxy, i.e. http://localhost:8118, as a custom proxy in GiA for your ITV/BBC TV HD/ BBC Radio (in the UK bitrates), since all the related RTMP CDN servers are not geo-blocked (subject to change any time...). Just a tip you should consider for the future...

Regards

mostlyotter commented 9 years ago

The BBC may be geo-blocking some radio now.

I've needed to use a VPN for the last few episodes of The Kitchen Cabinet, and last week, for The Infinite Monkey Cage, The Robert Peston Interview Show and The Archers Omnibus.

To be sure this is geo-blocking, here are logs for The Archers Omnibus which failed with a custom proxy for the second week in a row.

Custom proxy using GiA 1.3.7.1 updated manually to GiP 2.94 https://gist.github.com/mostlyotter/30ec7d55bd53c4e73db4

Custom proxy using GiA 1.8.6 on OS X 10.10.4 https://gist.github.com/mostlyotter/d99d299b8b0b07963bfd

VPN using GiA 1.3.7.1 updated manually https://gist.github.com/mostlyotter/cdf420e1956c2ef8040c

dinkypumpkin commented 9 years ago

You may get a more definitive picture if you run get_iplayer without GiA. In this case, GiA eats the most valuable part of the log.

Vangelis66 commented 9 years ago

@mostlyotter commented:

The BBC may be geo-blocking some radio now.

(APOLOGIES FOR A LONG POST)

Hi - for radio what they do is geo-filtering, i.e. stream higher bitrates in the UK and lower ones in the rest of the world - FWIW, I can stream "The Archers" from my non-UK IP at 48kbps HE-AACv1: http://www.bbc.co.uk/programmes/b061pgrs which should be fine for spoken content, plus, if you download the "Download MP3" link on that page from your overseas IP you'll end up with an MP3 file @128kbps, which is more than adequate for your listening pleasure...

Now, I'll deal with your reported issue:

You'll find that with GiP 2.94, if you DO NOT SPECIFY A PROXY (the provided one/ a custom one) to GiA, you'll still be able to download an MP4 file at 128kbps AAC-LC. So for those audio programmes that do not come down at high bitrates with a UK proxy, try without one! It may sound weird, but explanation follows...

I have inspected all 3 of your logs thoroughly, which do in fact speak volumes to me already... I do not know to what extent you are familiar with the inner workings of GiP, but dinky surely is...

"The Archers Omnibus - 12/07/2015" has a pid=b061pgrs http://www.bbc.co.uk/programmes/b061pgrs.xml reveals 2 versions: Original version, vpid=b061pgf2 Podcast version, vpid=p02w4fy2 RTMP streams for Original are found at: http://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/pc/vpid/b061pgf2/proto/rtmp while RTMP stream for Podcast is found at: http://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/pc/vpid/p02w4fy2/proto/rtmp If you load those URLs from a UK IP (i.e. when GiA/GiP uses a UK HTTP proxy), you get the following flashaacstd radiomodes:

Original version: flashaacstd1: (stream-uk-audio_streaming_aac_med) rtmp aac 128kbps stream (CDN: limelight/20) FMS hostname="bbcodspdns.fcod.llnwd.net" => NON GEO-BLOCKED flashaacstd2: (stream-uk-audio_streaming_aac_med) rtmp aac 128kbps stream (CDN: akamai/10) FMS hostname="cp107663.edgefcs.net" => NON GEO-BLOCKED So those two should be downloadable with a UK HTTP proxy

Podcast version: flashaacstd1: (ibroadcast_audio_pc) rtmp aac 128kbps stream (CDN: akamai/10) FMS hostname="cp41752.edgefcs.net" => GEO-BLOCKED !!! This one is blocked at server level, so rtmpdump (which isn't proxied) is refused connection, hence download fails...

Your reported issue is caused by the fact, which I could call a bug of GiP 2.94 (at least in the way it's used in GiA), that BY DEFAULT GiP tries the flashaacstd1 mode that corresponds to the "Podcast version" of the show, fails for the reason I've just explained and then DOES NOT TRY/FALL BACK to the non-blocked flashaacstd modes that correspond to the "Original version"...

If you do not employ a UK HTTP proxy, then http://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/pc/vpid/p02w4fy2/proto/rtmp reveals the international (not always available for every radio show) "podcast" RTMP stream: flashaacstd1: (stream-nonuk-ibroadcast_audio_pc ) rtmp aac 128kbps stream (CDN: akamai/10) FMS hostname="cp44293.edgefcs.net" => NON GEO-BLOCKED which is merrily downloadable from your non-UK IP :-) , which explains my initial statement...

@dinkypumpkin said:

You may get a more definitive picture if you run get_iplayer without GiA

Running the standalone GiP 2.94 (on Win32) pretty much confirms what I've already said: From my non-UK IP: perl get_iplayer-294.pl --type=radio --pid=b061pgrs --modes=flashaac --force https://gist.github.com/Vangelis66/c24c7b1c1a2ae3b11532 The 128kbps podcast M4A file is fetched... Using a custom UK HTTP proxy (details of which suppressed): perl get_iplayer-294.pl -p "http://proxyhost:port" --partial-proxy --type=radio --pid=b061pgrs --modes=flashaac --force https://gist.github.com/Vangelis66/a4e0d65527393d1be84d The 128kbps podcast M4A file fails to come down, because the FMS geo-blocks (ERROR: rtmp server requested close), BUT THE STANDALONE SCRIPT falls back to the radiomodes relating to the "Original" version and proceeds to download the "second" flashaacstd1 mode..., something which is not seen in the case of the GiA logs posted by @mostlyotter...

EDIT: It appears that by specifying "--versions=original", GiP 2.94 skips the "podcast" version - unsure how this can be set in GiA; also, one has to perform "--info" prior to recording perl get_iplayer-294.pl -i --type=radio --pid=b061pgrs to make use of that:

title: The Archers Omnibus: 12/07/2015 type: radio verpids: default: p02w4fy2 verpids: original: b061pgf2 verpids: podcast: p02w4fy2 version: default versions: default,original,podcast

IMHO, GiP should've picked the "original" version as default, NOT the "podcast" one...

Vangelis66 commented 9 years ago

On Thu, Jul 16th 2015, I commented:

EDIT: It appears that by specifying "--versions=original", GiP 2.94 skips the "podcast" version - unsure how this can be set in GiA; (sniip) IMHO, GiP should've picked the "original" version as default, NOT the "podcast" one...

Not sure if @mostlyotter is keeping track of this, but this "bug" of GiP (preferring the Podcast version as default over the Original) has been hopefully fixed in the "develop" branch of GiP, see:

https://github.com/get-iplayer/get_iplayer/issues/182

https://github.com/get-iplayer/get_iplayer/commit/4a4796ac8b082d81e93905b5d915ea489e835294

Regards

robingram commented 8 years ago

I use FoxyProxy and haven't been able to download any BBC TV programmes from New Zealand for about a month now. If I use their VPN instead of the proxy then programmes do download but incredibly slowly.

FoxyProxy claim that due to changes made by the BBC proxies will no longer work for downloads. I'd be interested in knowing if this is true. Is anyone (everyone?) else able to download using GiA from outside the UK using a proxy? If so, I'd be interested in recommendations for a proxy service. :)

hugofalkman commented 8 years ago

I use UnoTelly. Works great with GiA for both BBC and iTV

kelDommage commented 8 years ago

To touch on this, I had been having the exact same issue with another VPN and what solved it was turning off any tracker-blocker (as the VPN shows me originating in the UK, it's not an issue). Turning off the masking seems to have resolved it.

macplayer commented 8 years ago

This is working (today 2/15/16) on a Mac with OS 10.11.2 and v1.8.6 of GIA. Go into GIA prefs and disable "Quick Caching" and "Items to Cache: BBC TV." Quit and relaunch. Make sure you're connected to a UK VPN in the US. Go to a show to download on the BBC iPlayer web site and use "Use Current Webpage" to add a show to GIA's "Download Queue." Then "Start" download. Your mileage will vary depending on the VPN provider you're using. Some VPN services, because of BBC detection, only allow lower quality videos, others allow HD. If your downloads are hit and miss, consider setting BBC Formats, in Prefs, to a default lower quality (ie. rather than HD, Very High or High, try Standard or Low).

BBCADDICT commented 8 years ago

i have the same set up and made the adjustments in preferences but am still getting the same issue the process just gets stuck at the connecting stage then fails.

ownloading Show 1/1:

BBC Download (ID=b073gz3n): Downloading Stag - Episode 2 INFO: pid found in cache 1710: Stag - Episode 2, BBC Two England, , default INFO: 1 Matching Programmes INFO: Checking existence of default version INFO: flashhd1,flashvhigh2,flashvhigh1,flashhigh2,flashhigh1 modes will be tried for version default INFO: Trying flashhd1 mode to record tv: Stag - 2. Episode 2 INFO: File name prefix = Stag - 2. Episode 2 ((flashhd))
RTMPDump v2.4-77-gdc76f0a (c) 2010 Andrej Stepanchuk, Howard Chu, The Flvstreamer Team; license: GPL Connecting ...

macplayer commented 8 years ago

It's hit and miss for me. One of my VPN providers doesn't work at all, the other one does sort of. I can download one or two shows, but the other one will fail. I then turn off/on the VPN, and I'm able to download a few more.

BBCADDICT commented 8 years ago

There are no issues with GIA its definatley a problem at the VPN providers end thanks to all