ITV Hub is now ITVX

[gacres2]

The ITV Hub is no more - and it is now ITVX with its subscription service overlaid on the old hub.

As a result, searching for ITV programs fails for me as of today - and "Use Current Page" from ITVX does not work. GiA finds 12000 programs but stalls at that point.

Turning off ITV programs in Preferences got me functionality back for iPlayer programs, but it looks like the ITV side of the house will need some repairs.

[nrj53]

I agree. Neither of the last two releases work for ITV anymore. It caches 22,817 programs but fails to start the ITV cache process. As a result the program then hangs indefinitely. If you turn of ITV in the preferences, the program works normally for iPlayer programs. This just started today, since it was working normally 12 hours ago.

[ejstubbs]

I am seeing this as well. The hang is disconcerting since quitting GiA triggers a warning that series link info may be lost, which would create awkwardness should it happen.

[pebceb]

Same problem for me. BBC shows still okay if I turn off ITV like suggested. Hopefully Scott will have a fix.

[skovatch]

The short fix is to turn off ITV in the preferences, but I will release an update that disables ITV caching until I can figure out if ITV will ever work again.

[pebceb]

Just one question Scott. Turning off ITV allows us to get BBC shows but we still cannot use the current web page option for ITV shows. Is there another workaround to get them?

[skovatch]

Command-line yt-dlp won't download from the new ITV, and even Downie can't get a stream to save either, so there's no way to download ITV content at all at this point.

More concerning to me is that Downie can identify that there is a video stream but then fails claiming the video is DRM-protected. If true, that's the end of ITV support.

STV is still a possibility, though.

[dvd403]

I tried STV but it insists I sign in.

So I created an account even giving an accurate but false postcode in the area served by STV to be able to sign up. It says success! but when I try to stream it says I am outside the area.

I am using a UK VPN - I think it's still detecting I am not in the UK - I am in the middle of the Pacific Ocean - but it may be, like 4OD does, also checking the time zone on the computer to see if it matches the postcode. I'll have to try with my other mac set to UK time zone as soon as I finish repairing it.

[pebceb]

I got STV to work in my browser with a VPN. Not a time issue - you must have set up a postcode in Scotland. I picked Edinburgh.

[raythw]

The very latest version of Downie seems to have no problem with downloading from the itvx site. However, you'll get an 'Error: Cannot find download ID' message if it's a show that requires a premium account. I don't know what happens if you have such an account and use Downie. I suspect DRM would be in use for shows that require such a premium account. The problem in GiPA seems more to do with caching than actually downloading...

[hugofalkman]

Submitted a ticket for ITV yesterday to Charlie, the creator of Downie. This morning European time he responded: " Hi, thanks for the report. This issue will be fixed in the next update. Cheers, Charlie Monroe [support@charliemonroe.net]" This afternoon CET, release 4.6.1 shipped fixing ITV. Release notes: "Downie 4.6.1 Nov 18, 2022 New: floracing.com, gamer.tw. New: Custom integrations now support post-load delay for execution. New: Custom integrations now support description metadata field. New: Custom integrations now allow passing a context dictionary between themselves. Improved: arte.tv, bilibili.com, crunchyroll.com, douyin.com, itv.com, iqiyi.com, omny.com, reddit.com, soundcloud.com, uktvplay.co.uk, youtube.com. Improved: Live streams that are downloaded as TS containers will now convert to MP4 when postprocessing is set to MP4."

@raythw Submit a new ticket for your error message -- Charlie willl tell you if its DRM, or can be fixed.

[m1ctk]

I thought it was the launch of ITVX messing up things. Sao that's the end of automater I guess.

[skovatch]

I thought it was the launch of ITVX messing up things. Sao that's the end of automater I guess.

No, GiA will continue on. It may take some time but scanning ITVX for show listings should still be possible. And if Downie can download ITV content I'm pretty confident yt-dlp or youtube-dl can do it too.

[skovatch]

You may want to follow along with this yt-dlp issue tracking ITVX support. It's not encouraging, honestly. https://github.com/yt-dlp/yt-dlp/issues/5570

[ejstubbs]

Living in Edinburgh and already having an STV Player account, I can access programmes on there just fine. But GiA doesn't recognise them when I click the "Use Current Webpage" button, saying:

Programme Page Not Found

Please ensure the frontmost browser tab is open to an iPlayer episode page or ITV Hub episode page.

Did this ever work, or would it need a change to GiA?

(I'm particularly interested in this because I've just discovered that STV Player currently has The Field of Blood available, something I've been trying to get hold of for ages!)

[skovatch]

Programme Page Not Found Please ensure the frontmost browser tab is open to an iPlayer episode page or ITV Hub episode page.

Did this ever work, or would it need a change to GiA?

No, although it has been requested in the past. See https://github.com/Ascoware/get-iplayer-automator/issues/420

[ejstubbs]

OK, thanks. I shall await developments.

[pebceb]

Any update on a possible fix for the ITV problem?

Thanks.

Peter

[m1ctk]

It will be some time before there is an update imo. With so much added content I wonder if any new update will cause GIA to crash !

[pebceb]

Given that ITV is not working at all it might be worth the risk, but agree we don’t want to risk BBC shows. Scott’s call I guess.

[dvd403]

I’ve just subscribed to Downie 4 and have downloaded all the itv shows I would have obtained through iplayer.Automator. Itvx also has many shows from 4OD

[m1ctk]

Just trying Downie which I've never heard of, very cool ! itvx shouldn't have any shows from all4 though?

The test download was news at ten which wasn't HD res btw

[dvd403]

M1ctk - it does have 4OD - it’s advertised as doing it. Do a search for ‘Grand Designs’ and you’ll find all the shows from series 1 through to 19. 40D itself is up to series 22/23.

[dvd403]

m1ctk

https://www.whattowatch.com/features/itvx-everything-you-need-to-know-about-the-new-itv-on-demand-service

[pebceb]

I had a few issues with Downie. Puts the shows as Home Video (easy fix once in apple tv), but I had a few that it downloaded the same episode twice or put the wrong episode number so they were out of order. I also find sometimes I have to put them in Downie twice before they will download. But I manage to work through it, although it gets very confusing sometimes. But at least it's something. Wish I was in the UK, I would just subscribe to the premium service on ITVX. Damn borders.....

[skovatch]

A few updates: I have figured out how to scan ITVx.com for shows, and can successfully rebuild the cache. That's helpful but doesn't get us much further until yt-dlp can actually download content again.

I also looked into using STV for ITV content and I'm happy to report that I have the 'Get Current Webpage' workflow working. I can get the show metadata into a format that looks good in Apple TV, and downloading it with yt-dlp works perfectly. I will likely need to restore the preferences for ITV quality, though, as they offer a full set of resolutions for most shows (1080p, 720p, down to 384x216) and I doubt you want to download 2gb for every show.

I haven't figured out how to get a full library of STV shows. I'm guessing I'd have to do it category by category like I'm doing for ITV, which will be slow.

I have also been doing a lot of under-the-hood work. I moved a fair amount of code to Swift, added a new logging engine, and removed a lot of duplicated code on the BBC side related to getting metadata and searching. It all feels stable in my usage, so I will likely release that before rolling out any of the ITV/STV changes. I'm trying to be very careful to not break things like your stored auto-record settings and saved queues.

Regarding ITVx and premium content: GiA was never intended to be a tool for downloading subscription-only or DRM-protected material. Managing usernames and passwords is an additional level of complexity that I don't want to deal with, and opens up a whole slew of legal issues that I'm not prepared to handle. That may be a disappointment to some of you but it's just the reality of being a part-time open source project.

[pebceb]

Thanks for all your hard work Scott.

[m1ctk]

You have done a excellent job so far and look forward to the finished product :)

[m1ctk]

M1ctk - it does have 4OD - it’s advertised as doing it. Do a search for ‘Grand Designs’ and you’ll find all the shows from series 1 through to 19. 40D itself is up to series 22/23.

You mean All4 of course. Yes but you can't download as it's premium content I noticed

[hugofalkman]

@m1ctk The whole All4 website is DRM-encrypted, so Downie can't download from it.

Downie has the support for user names and passwords Scott references, but I have found it's only useful for free websites that require a signon. All fee websites I have tried downloading from, Downie says they are DRM-protected, so it can't download.

[m1ctk]

Downie doesn't download in HD with me, should it ? I thought all shows were now HD but I am still getting 896 x 504

[hugofalkman]

@m1ctk

[m1ctk]

Hmm shame. I guess 1920 x 1080 will happen from the 8th ?

[hugofalkman]

Don’ t think ITV hub ever had it.

On 23 Nov 2022, at 16:11, m1ctk @.***> wrote:

Hmm shame. I guess 1920 x 1080 will happen from the 8th ?

— Reply to this email directly, view it on GitHub https://github.com/Ascoware/get-iplayer-automator/issues/428#issuecomment-1325229032, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACTV73HFSOEVHP32WRBIYGTWJYX33ANCNFSM6AAAAAASDIUARQ. You are receiving this because you commented.

[m1ctk]

No but ITVX does. With the developer tools on a firestick you can see most shows are now in full HD on itv hub / x but it seems not yet oddly via the website

[hugofalkman]

Charlie just shipped a second Downie release with further changes for ITVX. Release notes: Downie 4.6.2 Nov 21, 2022 Improved: itv.com, odysee.com, youtube.com.

[pebceb]

Any update Scott?

[skovatch]

Any update Scott?

No -- it looks like yt-dlp is waiting until the 'official' rollout of ITVx on Dec. 8 before they try to do anything.

[pebceb]

Okay thanks for the update.

Peter

[AtomicLaser]

Any updates for this at all?

[skovatch]

Sorry, not much to update here. There's discussion about higher resolution downloads and DRM and needing an account. @Vangelis66 has been very helpful here (as usual!)

If you want to follow along with youtube-dl see https://github.com/ytdl-org/youtube-dl/issues/31363 or yt-dlp at https://github.com/yt-dlp/yt-dlp/issues/5570

[hugofalkman]

After shipping 3 releases fine tuning the ITVX support, Downie yesterday shipped a release with full support for STV. Release notes here https://software.charliemonroe.net/trial/downie/v4/rnotes.html

[skovatch]

You can now download most(?) ITV content through STV at all resolutions up to full HD. There is some DRM-protected shows that yt-dlp won't touch, and I added guards so the app won't try to download them.

Does anyone know if STV is a subset or superset of ITV?

[Vangelis66]

Does anyone know if STV is a subset or superset of ITV?

See https://en.wikipedia.org/wiki/STV_(TV_channel) https://en.wikipedia.org/wiki/Scottish_Television https://en.wikipedia.org/wiki/Grampian_Television

[pebceb]

I tried an episode of Emmerdale from STV and it worked until the end and then wouldn't put it in Apple TV and started doing errors.

[ejstubbs]

I've tried the STV download feature with a couple of programmes. In each case I had the following results:

1) The download got to 100% on the Download Queue pane, but then failed with "Problem Unknown" in the Failed Download Solutions window; 2) There was a complete, and playable, .mp4 of the programme at the requested resolution in an appropriately named folder in the "TV Shows" subfolder of the "Movies" folder in my user's home folder; 3) The .mp4 had no subtitles, and there was no .srt file in the programme's folder; 4) As well as the .mp4 file, there was a .en.dfxp file, and a .jpg file in the folder.

I'd say that's an excellent start, all things considered, and I very much appreciate the work that's been done so far to try to address the issues caused by ITV's decision to implement DRM on their streaming service (especially so with a new series of Endeavour due this year.)

[pebceb]

Yes I saw that and was not trying to be critical - just offering feedback. I truly do also appreciate all the work that is being done to fix this.

Cheers.

Peter

[ejstubbs]

Aye, just reporting what happened on my machine in case it's useful.

Another good thing: ITV shows are now downloadable in 720p - previously they only ever downloaded in sub-SD, which I assume was a limitation of the ITV Hub.

I don't use iTunes for video so didn't see anything relating to that.

One other thing: the shows weren't recorded in the Download History - presumably because they were deemed to have failed downloading. Then again, ISTR seeing something similar with ITV shows in the last few months before ITV Hub became ITVX. Didn't think anything of it at the time, though.

[gacres2]

Does anyone know if STV is a subset or superset of ITV?

Those wikipedia links make it clear that this is a tricky question – neither is the right answer I believe. STV overlaps with ITV, but doesn't carry all ITV programming and STV carries some programs that ITV does not. Also, ITVX has numerous, formerly Britbox, programs that are definitely not part of STV's content.

[Vangelis66]

there was no .srt file in the programme's folder; (snipped) As well as the .mp4 file, there was a .en.dfxp file,

.dfxp is a subs format... As I don't run MacOS myself, I can only offer advice on yt-dlp itself 😜 ...

For STV fetches, the yt-dlp command should explicitly request the vtt subs format; then, ffmpeg is required to convert it to .srt:

yt-dlp [options] "https://player.stv.tv/episode/*" --write-sub --sub-format vtt --convert-subs=srt --embed-subs

Probably GiA has its own routine of embedding a .jpg thumbnail into the .mp4 file, but in standalone yt-dlp, thumbnail embedding is delegated to AtomicParsley (if on PATH) or the (optional) python module mutagen:

...  --write-thumbnail --embed-thumbnail

[skovatch]

2. There was a complete, and playable, .mp4 of the programme at the requested resolution in an appropriately named folder in the "TV Shows" subfolder of the "Movies" folder in my user's home folder;

This was due to a bug in how I was passing arguments to AtomicParsley. It should be fixed in v1.25.0. Anything that got added to TV.app is in your Home Movies section.

3. The .mp4 had no subtitles, and there was no .srt file in the programme's folder;
4. As well as the .mp4 file, there was a .en.dfxp file, and a .jpg file in the folder.

I apologize, but I am bad about testing subtitle support as I don't use it myself, so I often forget it's there. But thanks to @Vangelis66 and #443 it's a pretty simple fix.