BBC: new tagging scheme? + connected HD download issues.

[willson556]

From jim.sinf...@gmail.com on July 10, 2012 04:20:31

re. Specifically about BBC downloads. (1) Tagging scheme changes? (2) HD (720) downloads failing more.

Under app prefs "Download Format", I set HD and Very High. That used to give me a file name appended with the tag "{{flashvhigh}}" (for Very High, 468) or "{{flashhd}}" (for HD, 720).

Recently, strangely both the 468 and 720 have been left tagged with "{{flashvhigh}}" on them? And this is happening for all HD (720) downloads from the BBC.

More importantly, additionally I have found that I have to often re-try several times to get the HD 720 version of the programme to download. This seems to be getting worse, because as of today, a couple of programmes available in HD just refused to download at all in that format, so I was left with only the Very High versions instead.

(Strangely the HD versions do work when download using the BBC's Desktop app, and would also playback directly on BBC's website and my Sony TV which has iPlayer built-in. But even after trying 50+ times on FOUR separate Macs to get the HD 720 version of said programmes, they refused to work, regardless of settings in GIA.)

My thoughts... Perhaps BBC maybe soon thinking of offering 1080i/p downloads, so this may have a bearing on both these issues. • Tagging: perhaps the BBC are altering their scheme – eg. so "{{flashhd}}" would then work on new HD 1080 instead? • HD 720 download failures: perhaps BBC are tightening-up background system security again, before offering 1080?

Speculation obviously, so guessing. But causing problems so could be checked-out anyway. What version of the product are you using? On what operating system? • many Lion (10.7.4) AND Snow Leopard (10.6.8) machines. • GIA 1.3.7.1 (399).

Original issue: http://code.google.com/p/get-iplayer-automator/issues/detail?id=83

[willson556]

From dinkypumpkin on July 10, 2012 04:37:19

I think you have run afoul of 2, possibly 3, different issues. If you'll give me examples of each (720 tagged as flashvhigh, HD programmes that take several attempts, HD programmes not available at all), I should hopefully be able to diagnose. Also, for HD programmes that take several attempts, do you get get partial downloads on failure, or do the downloads not start? And are you using GIA outside the UK?

[willson556]

From jim.sinf...@gmail.com on July 10, 2012 07:41:54

OK, I'm in the UK BTW (before I start, all the examples above should have the normal () brackets not these {} ones! I was tired when I posted the initial post!!)

• To be clear, when I said the HD version fails, what I meant is that it ignores the HD version on the BBC servers, and just downloads the Very High version. Even though I have the "Flash - HD" set in the GiA prefs.

As soon as I realise the download is the lower SD version (by the filesize being too small as shown on the main GiA window: eg. ".../~954 MB" when it should be ~1.7 GB for the HD version): => I press Stop => open the Download History window to make sure it's blank (which it is of course, as only a few percent of the file actually downloaded before I stopped it) => delete that partially downloaded little-bit-of-an-SD-file from my download location (obviously, otherwise the app fails immediately on pressing Start again) => then hit Start again. Close the app, re-open it, refresh cache... nothing.

Repeatedly does the same thing, ignores the HD version instead starts downloading the SD version (again, not all the time, but yesterday, around half the time this was happening.) [typical of course... I just tried a few HD ones just now, and they do work, so can't give you an example of this!]

• The tagging issue happens on EVERYTHING I currently download in HD (well, the ones that work that is). They are all getting the "((flashvhigh))" tag, exactly the same as the programmes that are only available in Very High.

I'm a longterm heavy GiA user: use it every other day to download 4-8 programmes, and this has been happening for the last few weeks. eg. just did "Blackout - Episode 2", HD downloaded first time, but the file in my download location is titled "Blackout - Episode 2 ((flashvhigh)).mp4" instead of "Blackout - Episode 2 ((flashhd)).mp4" it should be doing.

Are you telling me no one else has this?? I have several Macs and they are all doing the same on both these issues, surely I can't be the only one? :-(

---

While you're here, any opinion on the "My thoughts..." points I raised above? (Though I suppose if this is all just me and no one else then I guess they're moot!) Don't get technical though in response, as I am not a coder, so simple suggestions would be best. ;-)

PS. Out of interest, noticed you answering a lot of Q's here? (is Thomas W. still working on GiA? Lots of unanswered/filed issue#'s, hence I wondered.)

[willson556]

From dinkypumpkin on July 10, 2012 08:15:57

No, you're not the only user encountering this issue. I'm fairly sure your problems with "missing" HD versions and file names with "flashvhigh" are down to GIA using an outdated version of get_iplayer. The BBC started pushing HD content with lower bit rates and get_iplayer was amended to accommodate that change, but the latest release of get_iplayer isn't included in GIA. However, you can update it yourself:

1. If you have "BBC Podcasts" selected in "Items to cache" in Preferences, you must unselect "Quick Caching" in Preferences. Quck caching will not work for podcasts with the updated get_iplayer
2. Quit GIA
3. Update get_iplayer

Run the following command at a terminal prompt:

/usr/bin/perl "/Applications/Get iPlayer Automator.app/Contents/Resources/get_iplayer.pl" --profile-dir="/Applications/Get iPlayer Automator.app/Contents/Resources" --update

1. Remove all existing caches

Run the following command at a terminal prompt:

rm "$HOME/Library/Application Support/Get iPlayer Automator/"*.cache

1. Open GIA and refresh caches

Let me know how that works. I'll take up your other questions in a separate post.

[willson556]

From dinkypumpkin on July 10, 2012 09:00:26

To follow up - It is also possible that some of the missing HD downloads are in fact missing at the BBC end (it happens once in a while). It's impossible to know without specific programmes to check. It also sometimes take days for the HD version of a particular episode to appear on the BBC media servers. For example, the last episode of LIne of Duty (broadcast Tue) didn't appear until after the Sun repeat, but the previous episode showed up fairly quickly. There is no pattern that I can discern.

Re: your "my thoughts" items - I don't see any of that going on. I think an obsolete get_iplayer was the root of your troubles. One of the BBC content distribution networks has implemented geo-locking, though, which is why I asked if you were in the UK. If 1080 streaming does show up at some point, I think it will come alongside some other changes that would necessitate additional amendments to get_iplayer. I would think any related GIA-specific changes would be pretty minor, though.

Re: support issues - I'm involved with get_iplayer itself, and I'm a long-time GIA user, so I monitor this site to help out when the former causes problems with the latter. No idea if Thomas has abandoned GIA or not. He has only been offline about 5 months, so not a terribly long time. Plus, the majority of the open issues could be closed without further action. Some non-UK users have been bitten by his proxy falling over, but they can find alternatives. I would guess more users are going to hit the same problem you did. Ideally, GIA would get a refresh with any major updates/changes to get_iplayer, but hopefully the back-door update procedure I gave you will suffice for the time being. I don't expect it to be the case, but if GIA is abandoned I or someone will probably keep it going in some reduced form.

[willson556]

From jim.sinf...@gmail.com on July 10, 2012 10:35:04

Missing HD's — I know all that (remember, 3 times a week I'm on the iPlayer site making sure I download what's on each channel that I want by checking-through the iPlayer BBC/ITV daily schedule of availability [ITV get 30 days usefully, but easier to do on same schedule as BBC channels). One can usually test HD availability/working in various ways: eg. playing directly on website/TV/DVD player, downloading using the official Desktop app and playing item, if HD link missing then manually adding "/hd/" to check the iPlayer HD page myself. And yes, they've started being cheeky by sometimes holding back the HD until second broadcast (for reasons unclear!) making it all more of a pain in the neck than needed to watch their damn HD versions of content.

1080i/p — been broadcasting for around a year now on the Freeview HD channels. While I'm no expert, I do know the ~3GB file size limit was one issue for the BBC's iPlayer users on WinXP, but XP usage is minimal (XP's really legacy software nowadays) given Win7's uptake (and soon Win8). Along with iOS devices all doing 1080p on their retina screens (though the devices are so small they clearly don't really need 1080p iPlayer directly!) AirPlay-ing content to Apple's new ATV3 supporting 1080p streaming (batteries permitting!). With a likely opening-up of iOS ATV apps with an ATV App Store in Autumn. (I thought I read iDevices had massive consumption stats of iPlayer content.) Although the bottleneck now is high-speed broadband availability being not widespread enough for this increase in volume. But still, I thought given all this, they may have decided finally to offer them, jumping sooner rather then later to at least offer a higher "Full HD" iPlayer service for at least some content for those who can handle it, as a slow build-up/open beta test type thing. Who knows. :-(

GIA/g_ip updating — I know very little of the technical background. Is anyone maintaining either/both of these or not then? Really, I read the other open issues (many of which you'd helpfully given advise on to others) but rather than manually dealing with this one by one, surely the underlying g_ip code needs updating then incorporating in GIA for all users? Why's it not being updated in the GIA app here? Sorry this goes over my proverbial (technical) head. The Project Home page (and links within it) here seem completely out-of-date too... ?

Commands — did all the above (removed podcasts to keep quick caching), entered those commands in Terminal, pressing return after each one: After (3) I got the below, so I presume the update failed:

Can't open perl script "/Applications/Get iPlayer Automator.app/Contents/Resources/get_iplayer.pl": No such file or directory

After (4) I got no response, so closed Terminal.

On reopening/recaching, the tagging issue remains wrong. Most HD's I tried to download, worked before and after doing this. So didn't seem to fix anything — I think it's the tagging not working that's the real issue here (he says with no real idea whatsoever, lol). :-$

[willson556]

From dinkypumpkin on July 10, 2012 12:28:38

Re: 1080p - I only meant the content delivery networks that back the iPlayer service. I wouldn't be surprised if they offered 1080p at some point, but it will obviously be a lot more expensive and resource-intensive. I saw a news item that the underlying streaming architecture for iPlayer may be changed to accommodate, but I don't know anything beyond that. I just hope we get some warning so that we can adapt get_iplayer.

Re: get_iplayer and GIA - get_iplayer is maintained and up-to-date, and will be for the foreseeable future. The latest release has been available for a month, and the source repository is always available. It's up to Thomas to incorporate it into GIA. AFAIK, he is the only one who can update the GIA source repository and cut releases.

Re: update failure - Do you have GIA installed in /Applications? If so, try "sudo" before "/usr/bin/perl"in step 3 and enter your password when prompted (I assume you are an administrator on your machine). I would expect a different error if access permissions were the problem, but it will only take a moment to test. If you installed elsewhere, replace "/Applications" with wherever you installed GIA.

If that doesn't bear fruit, then you'll have to search your machine to find the location of that file. I assure you it exists somewhere on your machine. GIA won't work without it, and several people (including me) successfully used this hack. When you find it, replace the values in step 3 with the correct location. But before looking elswhere, double-check the default locations from a terminal prompt:

ls -l "/Applications/Get iPlayer Automator.app/Contents/Resources/get_iplayer.pl"

Also check the directory:

ls -ld "/Applications/Get iPlayer Automator.app/Contents/Resources"

You wouldn't expect any response from step 4 unless no cache files exist. If you want feedback during deletion, change "rm" to "rm -v".

[willson556]

From jim.sinf...@gmail.com on July 12, 2012 23:16:26

Back again. Oh I see, the delay in offering 1080 is likely a backend related issue; interesting to know, thanks.

Anyway... Thanks for getting me to check the path where the app was; your step 3 command wouldn't work before because I had GIA in a folder within my apps folder. (I think I'm gonna give-up using folders in the sodding /Applications folder, it always causes so many issues it seems! — is it best not to 'folderise' apps in /Applications anyway then in your experience??). It then worked. As did the step 4 command too.

Testing confirmation...

• tagging — now shows correct tags for each one: ie. "((flashvhigh))" (for Very High, 468) and "((flashhd))" (for HD, 720).
• HD working — more importantly, AFAICT so far the HD version now works properly by downloading HD version first time rather then repeated attempts before it would do or not at all.

Many thanks dinkypumpkin for your efforts, much appreciated. I hope this issue# can help others who may need this too. Or even better if Thomas could update the GIA app. ;-)

---

2 (hopefully final) quick Q's:

a) Am I right that if a BBC podcast is wanted, then in GIA prefs one has to turn Quick Caching off, re-tick BBC Podcast box, then refresh cache? (if so, is an update coming to get_iplayer so we can turn on Quick Caching with BBC Podcast caching?)

b) Are there any other apps that can download iPlayer/ITV (or even C4!) content, on Mac OR Win — whether they use get_iplayer or some other code? (should say, in the past I used to use these apps: "iPlayer Downloader", "iPlayer Grabber", "iPlayer+", and "FreeRapid" — but AFAICT they're all dead.) ...Always good to have backup options isn't it, and a friend uses a Win 7 machine, hence why I ask.

[willson556]

From dinkypumpkin on July 16, 2012 04:30:49

Good to know that it worked.

Re: subfolder in /Applications - This is an unusual situation. It's rare you would ever need to go spelunking inside an application bundle. FWIW, for anything like GIA that is delivered as a single application bundle (which is already a folder), I just drag it into /Applications.

Re: podcasts and quick caching - Those are the steps required. And it's not get_iplayer that needs an update for quick caching to work with podcasts. Thomas will need to update the version of get_iplayer that he uses to produce the cache files on his server. The BBC podcast archive to grew too large for get_iplayer's hard-coded limits, so it had to be updated. The latest release of get_iplayer produces a different range of index numbers in the podcast cache than the old version.

Re: other iPlayer downloaders - I don't think iPlayer+ is quite dead, but it is basically just an old version of get_iplayer, so wouldn't be of much help here. I'm not aware of anything like GIA for Windows. There is Radio Downloader, but that of course is only for radio programmes. Sniffer-based downloaders like HiDownload or Jaksta are limited to downloading the lower-quality streams used in the iPlayer web site (I'm fairly sure they don't download HD streams even if available). A lot of Windows users get very good mileage from the Web PVR Manager that is installed with get_iplayer for Windows. It's roughly equivalent to GIA in functionality, though it works rather differently.

[willson556]

From jim.sinf...@gmail.com on July 18, 2012 15:47:23

Re: subfolder in /Applications – yeah I actually meant not an app bundle (which all apps pretty much are) but rather a group of app bundles inside a homemade folder I did in /Apps — eg. folder called "Downloaders" (path "/Applications/Downloaders/foo") "foo" being Get iPlayer Automator & a few other (dead or otherwise!) related downloading apps.

Re: other iPlayer downloaders – yeah I use Radio Downloader in a Win 7 virtual machine for radio shows. Reason —being a bit OCD about it, lol!— GiA has a small annoyance in that the thumbnail images it grabs for all BBC TV/radio content (ITV's get the larger ones, strangely) are still sized at only 150x84, whereas RD gets me larger 260x146. Thanks for Win suggestion of iPlayer PVR Manager you mention, passed it onto friend, and I may perhaps at least see how it works in my Win 7 VM, as a back up, given GIA is the only real Mac solution we have.

The basic updates required to GIA then (in my limited understanding, so there may be others): A) Updating the version of get_iplayer used to produce the cache files on GIA server. [==> surely this is quite a quick/easy update?] B) Making the thumbnails the bigger ones that are available (given RD can grab them). C) Update the proxy, for non-UK folks.

Dunno how to contact a dev via Google Code — and is it worth doing so for Thomas I wonder?

As a great many of us longterm GIA users on the Mac platform, and given this GIA app seems to effectively be the only easy, and seemingly best, way to get iPlayer (and ITV content - though I do wish ITV would improve their shite 704x396 resolution programmes, which is very small in this day and age on HDTV's!) — we all really hope GIA continues to at least be maintained (and improved on maybe, but even just maintained would be great, to be honest). I respect A LOT Thomas's hard work so far, but no signs of this in the last 4 months or so makes me a little nervous. :-|

Thanks again.

[willson556]

From dinkypumpkin on July 19, 2012 09:33:27

RE: small thumbnails - I uploaded a patch for that in #65. Should be a simple fix. In the meantime, you can try this workaround:

Create a plain text file named:

$HOME/Library/Application Support/Get iPlayer Automator/options

Make sure the file name has no extension, and check that the extension isn't hidden (via Get Info in Finder)

That file contains settings for get_iplayer. Add the following line to that file:

thumbsize 6

This setting will force the thumbsize option through to get_iplayer since GIA doesn't pass it properly. The value 6 corresponds to the largest thumbnail size (640x360).

RE: Web PVR Manager - it works on OSX as well, though there isn't much reason to use it with GIA available. It's available via Homebrew: https://github.com/dinkypumpkin/homebrew-get_iplayer/wiki RE: dev contact - You can click the link under Members on the project home page to go to his profile and then expand his email address via captcha.

RE: maintenance - Somebody changed the provided proxy recently, so Thomas must be out there somewhere. Right now, GIA only needs a very minor update to address the issues you listed. Maybe he'll get around to that as well.

[willson556]

From jim.sinf...@gmail.com on July 19, 2012 11:56:14

Cannot do that... In TextEdit, I have to save first as RTF (won't allow to save as .txt), then manually change in Finder to end in .txt (which turns the Kind to "Plain Text"), but then when I remove the .txt extension in Finder, the Kind changes to "Document". Is that right? What am I meant to do with the file then? (closed/re-opened/cleared cache and tried GIA: still small 150x84 though?)

I also tried just grabbing that download from #65 — but again what do I do with it exactly?

Yours confused.

[willson556]

From dinkypumpkin on July 19, 2012 12:17:14

The file probably still has RTF junk in it and probably still has a hidden extension, so it's unusable. Use Format -> Make Plain Text to change the format of the document to plain text, then save it. You can make plain text the default format in Preferences -> New Document. To be certain about the file name, don't remove the .txt extension in the main Finder window - it just hides it. As I mentioned, open the Finder Get Info dialog for the file. Uncheck the Hide Extension box and then remove the extension under Name & Extension.

Sorry - didn't mean to imply that you could do anything with the patch from #65. It's only useful if you want to build GIA yourself, which I assume you don't. I just mentioned it to point out that a trivial patch for the thumbnail sizing issue was already available.

[willson556]

From dinkypumpkin on July 20, 2012 02:26:21

I should also mention the no-faffing way to do this from the command prompt:

nano "$HOME/Library/Application Support/Get iPlayer Automator/options"

Type your input, Ctrl-O to save, Ctrl-X to exit.

[willson556]

From jim.sinf...@gmail.com on July 20, 2012 11:05:22

Still unsure what's happening here, so am being as precise as possible in explaining below...

Objective: make a file titled "options", as kind "Document", with the text "thumbsize 6" saved in it, saved to location "Library/Application Support/Get iPlayer Automator/". I'm testing with RADIO show "Gilles Peterson", and TV show (in case TV worked differently) "Have I Got News For You".

• Tried comment #14 way: opened Terminal, copy/pasted " nano "$HOME/Library/Application Support/Get iPlayer Automator/options" ", hit return key, typed "thumbsize 6", hit Crtl-O, attempted to hit Crtl-X (got bleep sound), couldn't work-out what to do after faffing around in Terminal, so manually hit the red dot to close the Terminal window, that location then had a file called "options.save" in it. Tried RADIO and TV shows above, got same 150x84 thumbnail size in each.
• Tried comment #11/13 way: no file AT ALL was saved to that "Library/Application Support/Get iPlayer Automator/" location?
• I then opened TextEdit, typed "thumbsize 6", went to Format menu to hit "Make plain text", saved file with the .txt on it to location "Library/Application Support/Get iPlayer Automator/", opened Get Info, removed .txt, making sure "Hide extension" was unchecked. Tried RADIO and TV shows above, this time I get 260x146 thumbnail size in each, but not the 640x360??

Have you tested these yourself; do you get the 640x360 thumbs then?

[willson556]

From dinkypumpkin on July 20, 2012 11:46:32

I assure you this works. I've used it for years with GIA. I use the options file for other get_iplayer functions, and it never fails. I tested your 2 programmes with no problems.

How are you determining the size of the image? I don't know how you got a 260x146 thumbnail. That's not one of the sizes available.

Forget TextEdit. If you don't normally use it to work with text files it's not worth learning just for this. It's simpler to coach you through it with nano. I think your nano problem was because you tried to quit before you finished saving the file. So, once more into the breach:

1. Remove any files you already created in "$HOME/Library/Application Support/Get iPlayer Automator/"
2. At a command prompt in Terminal:

nano "$HOME/Library/Application Support/Get iPlayer Automator/options"

1. Type in the nano window:

thumbsize 6

1. Hit Ctrl-O to save the file
2. At the bottom of the nano window you will see the prompt "File Name to Write: /Users//Library/Application Support/Get iPlayer Automator/".
3. Hit Return to accept the file name and save the file to disk.
4. At the bottom of the nano window you will see a message something like "[Wrote 1 line]"
5. Hit Ctrl-X to exit nano
6. Try GIA again.

If you still don't think you're getting the largest thumbnail image, repeat the process above and add this line to the options file:

thumb 1

That tells get_iplayer to save a copy of the thumbnail image in the same directory as the MP4/M4A file. You can open the JPEG file in Preview and use Cmd-I to show the Inspector window, which will show the image dimensions. If you like, you can add the "thumb 1" line when you create the options file, but there is no point in saving JPEG files if you don't need them.

[willson556]

From jim.sinf...@gmail.com on July 20, 2012 15:23:10

You're right: this does work. Even without your comment #16.

To be clear, the third manual method I used above worked anyway. I realised the 260x146 thumb I reported was wrong... I use Path Finder generally and it has its own Get Info window (similar to Finder one) which gives has a thumb Preview pane in it which (in the top right corner of it) has a dimension size given. For some reason, in there it gives the Get Info "preview" size (260x146), rather than the ACTUAL size for the thumbnail.

Weird/stupid/annoyingly misleading if you ask me (I re-checked Path Finder's settings, and AFAICS there is none for this issue!). How are you checking thumb size then (yes your #16 "thumb 1" line works to save an extra image file, BTW)?? [I ended-up chucking the file into iTunes and opened its Get Info, then dragged the image out onto desktop, opened it in Preview, which gave the correct 640x360 resolution I was actually getting.]

As a separate point, as someone who very clearly knows what you're talking about, I'd make a strong suggestion to contact Thomas directly to get these few issues sorted directly. Instead of dealing one-by-one with us non-coder, erm... "plebs".

Thanks again.

[willson556]

From dinkypumpkin on July 21, 2012 08:44:23

RE: checking image size - For your purposes, a simple way to check the size of embedded artwork is to use MetaX (free, find it at MacUpdate). Just open a MP4/M4A file in MetaX and it will show the size of the embedded artwork under the thumbnail preview in the Info panel. Offhand, I don't know of a free equivalent for MP3. There are numerous applications that will display or create metadata, but many don't go to the trouble to show the size of embedded artwork.

[willson556]

From thomas.w...@me.com on September 18, 2012 19:44:13

Status: Accepted

[willson556]

From thomas.w...@me.com on September 18, 2012 19:44:44

Get_iplayer updated

Status: Fixed