jasueh commented 3 years ago

I've been finding an entry in my logs saying the following: Unable to load channels from [https://www.tvchannellists.com/List_of_channels_on_Virgin_Media_(UK)_-_New_Packages] code [200], trying cache

It seems that around the 27 of August, the table format has changed from

{| class="wikitable sortable" style="border-collapse:collapse; font-size: 90%; text-align:center" width="100%" cellpadding="3" border="1" ! width="5.5%"|HD ! width="5.5%"|SD ! width="5.5%"|+1 ! width="12.5%"|Channel name ! width="14.5%"|Notes ! width="7%"|TSID ! width="8.5%"|SID ! width="12.5%"|Owner/parent company ! width="7.5%"|Broadcast hours ! width="7%"|Package HD ! width="7%"|Package SD ! width="7%"|Package +1

to

{| class="wikitable sortable" style="border-collapse:collapse; font-size: 90%; text-align:center" width="100%" cellpadding="3" border="1" ! width="5%"|TV 360 ! width="7%"|Region ! width="5%"|TV V6 ! width="10%"|Logo ! width="12%"|Channel name ! width="14%"|Notes ! width="8%"|TSID ! width="9%"|SID ! width="13%"|Owner/parent company ! width="8%"|Broadcast hours ! width="9%"|Package

supajason commented 3 years ago

I was about to look into this but I can't access tvchannellists.com (I think it's down)

(row 1043 - media_player.py) It it looks like the below code relates to the column numbers that get scrapped from the site. Some other changes may need to take place as plus one packageHD & SD have been removed...

 if is_tv_table:  # TV Channels
                        CELL_HD = 0
                        CELL_SD = 1
                        CELL_PLUSONE = 2
                        CELL_CHANNELNAME = 3
                        CELL_PACKAGEHD = 9
                        CELL_PACKAGESD = 10
                    else:  # Radio Channels
                        CELL_HD = 0
                        CELL_SD = 0
                        CELL_PLUSONE = 0
                        CELL_CHANNELNAME = 1
                        CELL_PACKAGEHD = 7
                        CELL_PACKAGESD = 7

supajason commented 3 years ago

The site is back up and I can see no changes to the table. I've restart HA and it's working again :-)

90 Fixed

jasueh commented 3 years ago

@supajason It seems tvchannellist.com has brought back the original structure because they have rolled back the site

IMPORTANT: Site has been reverted to August 5. Read more here.

But there isn't any link in the "here", so not clear if this is going to be changed back again or not.

jasueh commented 3 years ago

Ok, I think I found why they rolled back:

https://www.tvchannellists.com/w/User_talk:Tvchannellists

Postmortem: September 1 data loss

bertbert72 commented 3 years ago

Yeah, I had a look at site last night before it crashed. The new table format is a bit insane and is of no use to us. It seems to have been driven by channel number differences between the TV360 and V6/Tivo boxes. However it is all over the show and loses detail such as HD/SD channels.

If they switch to this new format, we'll have to change to another source. I might add the page to the Git repo and people can update as needed. And longer term, change it to a format more specific to what we are using it for. The problem will of course be that it'll need maintained.

jasueh commented 3 years ago

@bertbert72 have you tried contacting tvchannellists admins? Perhaps they can set something like an API instead of scraping the page

bertbert72 commented 3 years ago

The format of the site is no longer compatible with the component. I've created a cut of the relevant tables from the last working copy I have and put it into GitHub. The latest version of the component will now point to that.

I haven't had a running HA environment in quite a while now, so I doubt that I will be in a position to progress this further. The better solution would probably be to redesign the channel bit to use a data file maintained in GitHub.

ColinRobbins commented 3 years ago

@bertbert72 I'm slightly confused with the latest update. I understand you've copied the relevant tables to github, but the update seems to point the URL to https://raw.githubusercontent.com/bertbert72/HomeAssistant_VirginTivo/master/custom_updater.json which does not see to be the github list of channels. Is this the correct URL?

bertbert72 commented 3 years ago

£$!*@!

Wow, just wow! It most certainly isn't the right URL. A new release with hopefully the right URL in it this time. Apologies for that.

bertbert72 / HomeAssistant_VirginTivo

Channel List doesnt seem to parse #90

90 Fixed