tv_grab_uk_tvguide can't run --configure

[qdacsvx]

Thanks for taking the time to report an issue. Please take a moment to review our open/closed issues above, in case your issue has already been reported.

If you are reporting a new issue, please give your issue a descriptive title and fill out the blanks below, providing as much information as possible.

XMLTV Version?

head from github.

XMLTV Component?

tv_grab_uk_tvguide

What happened?

running --configure prints an error message.

What did you expect to happen?

Configure runs without errors.

Did you see any warnings/errors?

Can't call lookdown on an undefined value.

What steps are needed to reproduce this issue?

tv_grab_uk_tvguide --configure

Any other information?

new

What other software are you using?

Operating System: Xubuntu

Perl Version: Perl 5 Version 30

[honir]

The website is currently broken. If you go to https://www.tvguide.co.uk/channellistings.asp?ch=74&cTime= you will see the channel dropdown list next to "BBC One London" is empty.

[mansoorhaq]

I am using the tool for the first time, so I am just reporting what I experienced. This issue is also happening on Windows 10, and can't select the channels as it says below. Also, the cache path is not consistent with slashes "\ & /" (maybe, this is okay) and the folder is empty.

I hope this helps...

xmltv.exe tv_grab_uk_tvguide --configure
Timezone is -0800
tv_grab_uk_tvguide uses a cache with files that it has already downloaded. Please specify where the cache shall be stored.
Directory to store the cache in: [C:\Users\User1/.xmltv/cache]
Parsing result: (nothing to do)
Select the channels that you want to receive data for.

[honir]

I've updated the grabber to use an alternative method for fetching the configuration if the main method fails. (Note it doesn't find quite so many channels.)

@mansoorhaq This change will come through the next time the windows .exe is built.

[mkbloke]

It's a bit late to comment on this, but as I needed to use --configure for this grabber a few days ago, I thought I would. I ended up ripping the latest code for the backup method of getting the channels in the fetch_listings() sub and dumping it into my grabber as a workaround to get most of the channels added into tv_grab_uk_tvguide.conf and then manually added a couple more that didn't get put in that I wanted.

I messaged the site via its Facebook page last week about missing schedules for a few channels for this weekend and to their credit, they messaged back saying it was fixed the following day, which it was. I have also messaged them again tonight about the broken drop down menu, so hopefully that could also be fixed soon.

I realise the backup method is just that, so maybe this is unnecessary, but an improvement could be made via a POST request:

$ curl -d 'thisDay=&thisTime=&gridSpan=&emailaddress=&regionid=1&systemid=7&xn=Show+me+the+channels' https://www.tvguide.co.uk/mychannels.asp?gw=1242 | perl -ne "/(?:add|delete)channel\((\d+),'(.*)'\)/i and print \"\$1 \$2\n\"" 
506 4Music
752 4seven
1298 5Action HD
571 5SELECT
1121 5Spike +1
901 5STAR +1
572 5USA +1
...

$ curl -d 'thisDay=&thisTime=&gridSpan=&emailaddress=&regionid=1&systemid=7&xn=Show+me+the+channels' https://www.tvguide.co.uk/mychannels.asp?gw=1242 | perl -ne "/(?:add|delete)channel\((\d+),'(.*)'\)/i and print \"\$1 \$2\n\"" | wc -l
937

If the fetch_listings() sub was changed similar to this (for the request method at least): https://github.com/XMLTV/xmltv/blob/2e5ad3ed96c663121247d8c0f053748c8c7c6000/grab/ar/tv_grab_ar#L909-L922 it would provide a convenient way to make the POST request, get the result and then look down/match against a suitable regex to get a fuller list of channels. By the way, the gw parameter in the POST URL is simply '?gw='+($(window).width()-26).

If you think that's worthwhile, I could take a look and submit a PR. Cheers.

[honir]

Good spot. But I'm unsure what it offers above the current 'alternative' method?

Your example gets the channels for 'Popular Channels' (whatever that means!). So it would still need one POST request for each of the possible groups they have defined. Which is what the current altve method does.

The existing altve method is, umm, 'fragile', but I'm struggling to see the benefit of your POST approach.

[mkbloke]

It gets the extra channels besides the popular channels that are preselected on the page from the 'Add channel' section also, so it's not necessary to loop through all the regions and TV systems.

Having said that, I notice it gets multiple TV Guide channel IDs for a given channel name in some cases, e.g. for 'Sky Sports Golf' there are 13 and another 13 for 'Sky Sports Golf HD', for some reason, which might not be great. That might not be a problem if they all have schedule data though. I need to check that or see if there's a way to otherwise identify the best one to pick.

[mkbloke]

I have some ideas about an improved backup method that should be able to eventually identify all channel IDs with a minimum number of requests. I'll try to put something together and submit a PR for your perusal soon.

I have to say, the TV Guide website is a bit brain-dead. Take the example I mentioned above regarding 'Sky Sports Golf'. Is the working one (i.e. with schedule data) the lowest channel ID? Nope! The highest ID? Nope! The one with a logo image? Nope! It's just one of them! You get 13 to choose from in the case of 'Sky Sports Golf' and no indication of which one is the one (the only one) that actually has schedule data. Even worse, if you select Sky as the TV system, it adds 'Sky Sports Golf' to the lineup with a channel ID that contains no schedule data! Even their own code can't get the right channel ID.

Nobody yet seems to have read the Facebook message I sent to TV Guide regarding the broken drop down menu, even after 4 working days. I can only guess they don't check their Facebook page messages very often and perhaps I was lucky with the response to my first message.