BBC Radio Programs not downloading "Unknown Problem" - change in web interface?

[whitedb]

I am in the UK and not using any proxy.

What steps will reproduce the problem? Be specific, and provide as much detail as possible. Trying to download an number of different BBC radio programs, e.g. I'm sorry I'll read that again" PID b012lm9q

What is the expected output? What do you see instead? I expect a download but get an error message in GiA "Problem Unknown. Please Submit a bug report"

What version of GiA are you using? On what version of OS X? Mac OSX, 10.12.6, GiA 1.13.13 (latest) but the same problem occurred with 1.13.10

Are you using a web proxy? DNS proxy? VPN? VPS? No

What program are you attempting to download (full title, series and episode)? Provide only ONE program as an example. I'm Sorry, I'll Read That Again: Series 8: Episode 2, pid b007jqyp

Please provide any additional information below. I notice that the BBC has changed its interface and when you go to play the program it switches from the familiar URL and interface; https://www.bbc.co.uk/programmes/b012lm9q to https://www.bbc.co.uk/radio/play/b012lm9q which has a broad banner across the top. The download fails. All programs that default to this new interface seem to fail (not extensively tested). I tried entering the PID directly into GiA but the problem persisted. The log shows;

BBC Download (ID=b012lm9q): Downloading I'm Sorry INFO: 1 total programmes BBC Download (ID=b012lm9q): I'm Sorry Failed AppController: Downloads Finished

Another download failure shows:

BBC Download (ID=p00m9jd5): Downloading 6 Music Classic Concert: Todd Rundgren – Archive concert (1994) INFO: 1 total programmes WARNING: No media streams found for requested programme versions and recording modes. WARNING: The programme may no longer be available - check the iPlayer site. WARNING: If you are outside the UK, check if your VPN/VPS/Smart DNS/web proxy has been blocked. BBC Download (ID=p00m9jd5): 6 Music Classic Concert: Todd Rundgren – Archive concert (1994) Failed

The programs do play from the webpage.

If I try "Use Current webpage" on the newly formatted page I get the error:

Invalid Page: https://www.bbc.co.uk/radio/play/b012lm9q

I suspect this change of ULR to actually play from is causing the problem.

[whitedb]

Another log entry for the failed download showed:

BBC Download (ID=b007jqyp): Downloading I'm Sorry, I'll Read That Again: Series 8: Episode 2 INFO: 1 total programmes WARNING: No media streams found for requested programme versions and recording modes. WARNING: The programme may no longer be available - check the iPlayer site. WARNING: If you are outside the UK, check if your VPN/VPS/Smart DNS/web proxy has been blocked. BBC Download (ID=b007jqyp): I'm Sorry, I'll Read That Again: Series 8: Episode 2 Faile

[BruceGoatly]

I've just looked this up on iPlayer and it seems that the episode isn't available - which is odd, because the episodes before and after (episodes 1 and 3) are showing as available. However, those episodes are also giving 'Invalid page' errors even though get_iplayer on my PC can snag them from the PID. So, yes, this seems to be a URL parsing problem, probably the one that Scott is already working on.

[Vangelis66]

I've just looked this up on iPlayer and it seems that the episode isn't available - which is odd, because the episodes before and after (episodes 1 and 3) are showing as available.

According to https://www.bbc.co.uk/programmes/b0129s8d/broadcasts/2018/08 Series 8 Episode 2 (b007jqyp) was not recently repeated/re-broadcast; from Episode 1, on Aug 17th 2018, they jumped straight ahead to Episode 3 the week after (Aug 24th 2018); in fact, as per https://www.bbc.co.uk/programmes/b007jqyp/broadcasts Episode 2 last aired on July 11th 2011; so not odd at all it isn't available on iPlayer Radio currently :wink: ...

[BruceGoatly]

not odd at all it isn't available on iPlayer Radio currently

This is of course quite true... I misread the air dates of the surrounding episodes and thought a week was missing. 😉

But I think my point about URLs still holds.

[Vangelis66]

If I try "Use Current webpage" on the newly formatted page I get the error: Invalid Page: https://www.bbc.co.uk/radio/play/b012lm9q I suspect this change of ULR to actually play from is causing the problem.

If GiA uses web scraping code to harvest the PID from "Current Webpage", then it is expected that this feature be broken if the powers at the beeb change the page template/format/source... 😞 Recently, youtube-dl also had to adapt to this change in iPlayer Radio...

@whitedb Have you tried the "Use Current webpage" feature with just the "old" page template https://www.bbc.co.uk/programmes/b012lm9q , i.e. before hitting the "Listen Now" button (and the redirection it triggers)?

I tried entering the PID directly into GiA but the problem persisted.

This is indeed not expected and needs to be troubleshot... As reported already, latest standalone GiP 3.17 has no issues fetching via PID... get_iplayer --type=radio --pid=b012lm9q

Episodes:
I'm Sorry, I'll Read That Again: Series 8 - Episode 3, BBC Radio 4 Extra, b012lm
9q
INFO: 1 total programmes

INFO: Downloading radio: 'I'm Sorry, I'll Read That Again: Series 8 - 3. Episode
 3 (b012lm9q) [original]'
 11.8% of ~77.18 MB @   6.3 Mb/s ETA: 00:01:26 (hafhigh1/ll) [audio]

So, if you're in a hurry, try standalone get_iplayer or youtube-dl (both work on a Mac!).

Another download failure shows:

BBC Download (ID=https://www.bbc.co.uk/programmes/p00m9jd5): Downloading 6 Music Classic Concert: Todd Rundgren – Archive concert (1994) INFO: 1 total programmes WARNING: No media streams found for requested programme versions and recording modes. WARNING: The programme may no longer be available - check the iPlayer site. WARNING: If you are outside the UK, check if your VPN/VPS/Smart DNS/web proxy has been blocked. BBC Download (ID=p00m9jd5): 6 Music Classic Concert: Todd Rundgren – Archive concert (1994) Failed

@skovatch This failure has nothing to do with the BBC changing programme page URLs etc... This specific audio show, https://www.bbc.co.uk/programmes/p00m9jd5 is an archived one; for starters, it's geo-fenced to the UK only (not a problem for OP, but possibly one for you...); secondly, there exist only geo-fenced RTMPE streams for it, that GiP has stopped supporting since v3.00.... Please have a look (with a UK IP) at the mediaselector API URLs results:

pid=p00m9jd5 => vpid=p00m9jd7
https://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/pc/vpid/p00m9jd7
https://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/iptv-all/vpid/p00m9jd7
https://open.live.bbc.co.uk/mediaselector/5/select/version/2.0/mediaset/apple-ipad-hls/vpid/p00m9jd7

No HLS/DASH stream in sight, hence no ability to dump it... One has to resort to using a really old (2.99 and older) version of get_iplayer (with rtmpdump installed and working) to be able to fetch this one...

Regards

[skovatch]

@Vangelis66 thanks for troubleshooting this. Yes, if BBC changed their URLs around I'll have to fix up the "Get Current Webpage" to adapt to the new format. Might take a day or two, as I won't have free evenings for a day or so.

This is indeed not expected and needs to be troubleshot... As reported already, latest standalone GiP 3.17 has no issues fetching via PID... get_iplayer --type=radio --pid=b012lm9q

This isn't the same thing as typing the PID into the search field. That invokes get_iplayer with --search, and indeed, get_iplayer --type=radio --search=b012lm9q won't find anything.

[Vangelis66]

This isn't the same thing as typing the PID into the search field. That invokes get_iplayer with --search

Thanks for this 😸 ; the

I tried entering the PID directly into GiA

bit by OP had me confused with get_iplayer's Web PVR Manager GUI (and its Quick URL input field...) 🤦 Apologies...

[skovatch]

Fixed the Get Current Webpage problem in v1.13.14, but not the the ability to download archived shows. Which is a shame, because I'd listen to that Todd Rundgren concert.

[Vangelis66]

@skovatch wrote:

not the the ability to download archived shows. Which is a shame, because I'd listen to that Todd Rundgren concert.

This probably bends many rules, but here you go:

[redacted by the admin]

NB: Please make sure to delete this link after you grab the file (I know you can edit other people's comments in your repo 😜 ) ...

A well-deserved present for all the hard effort you direct to this project 👍

[skovatch]

@Vangelis66 Thanks for that! I have been rediscovering Todd Rundgren on Apple Music and also on "Daryl's House", Daryl Hall's show.

[Vangelis66]

[redacted by the admin]

I just realized GitHub's (new) feature that allows for viewing a comment's edit history could be used for retrieving the original link, so for good measure the file itself has been deleted from hosting server... Enjoy listening to it... 😜

[jefro108]

This failure has nothing to do with the BBC changing programme page URLs etc... This specific audio show, https://www.bbc.co.uk/programmes/p00m9jd5 is an archived one

This programme downloads fine with MacYTDL https://forum.videohelp.com/threads/384566-MacYTDL-youtube-dl-GUI-for-Apple-Macs using the URL

So just wondering @skovatch whether you could get youtube-dl to download any archive shows with a programme page URL