search

PredatH0r / ChanSort

TV channel list editor for Samsung, LG, Sony, Hisense, Panasonic, Philips, Sharp, Toshiba and MANY more.
857 stars 115 forks source link

Philips channel-list version 120.0 #371

Open marko-arn opened 11 months ago

marko-arn commented 11 months ago

The Philipps channel-list version 120 is currently not supported by ChanSort. It would ne nice to geht an update for that version.

PredatH0r commented 11 months ago

There is an inofficial ChanSort version supporting list format 120 at https://chansort.com/ChanSort_2023-09-17.zip So far I have only received a single list in this format, and that only contained DVB-C channels. Can you please send me your list so I can improve support for this format?

PredatH0r commented 11 months ago

I'm sorry, but I just realized the linked version above did not add support for format 120, but instead for the old format 2.0. I have not received any files with format 120 yet and can't tell whether support will be possible or not.

madkiss commented 11 months ago

Here is one. Let me know if you need the full folder of the playlist dump.

MtkChannelList.zip

PredatH0r commented 11 months ago

Thanks! The full directory structure would be very helpful. Often there are additional files that hold checksums and without modifying those too, the list will be rejected.

madkiss commented 11 months ago

Here you go. Just realized that my brand new Philips OLED TV cannot be ChanSort-fixed. I'm more than happy to toss in a tenner or several to get support for this out of the door ASAP :)

ChannelMap_120.zip

PredatH0r commented 11 months ago

Thanks! Here's a version of ChanSort that can load and edit the DVBC.xml, DVBT.xml, DVBSall.xml and Favorites.xml: https://chansort.com/ChanSort_2023-10-29.zip, which are the same as in earlier formats (100, 105 and 115).

Your list contains an additional MtkChannelList.xml file, which isn't modified by ChanSort. I hope the TV doesn't really need it. Also, the chanLst.bin, which normally contains model name and checksums for DVBC.xml, DVBT.xml, DVBSall.xml and Favorites.xml, looks similar at first glance, but after a closer look the content doesn't follow any logical structure and certainly doesn't contain any checksums. It looks like the TV's modified software writes some garbage to that file. For that reason this new ChanSort version doesn't modify this file.

I can't tell if the TV will work as expected after loading a modified list. It could work, but it could also be that something is out of sync, like the channel list in the menu, the EPG or the zapping order when using Pr+/-, as some part of the TV might actually be using the data in MtkChannelList.xml.

To make things even more complicated, that MtkChannelList.xml contains everything twice. The first part is readable XML data, but it also contains a data block named "" containing unreadable binary data with 2 data blocks named "" and "". And the latter could very well be the channel list used by the TV tuner board, which might be a standard piece of hardware + software, that Philips bought from a 3rd party company.

Without such a TV at hand, this will be a tedious task to figure out a way that works. I suggest to switch only the first 2 channels, load that list and see if it works. If it doesn't, you could also edit the MtkChannelList.xml with a text edtior (like Notepad++), search for "major_channel_number" and replace the "1" with a "2" and then in the next channel the "2" with a "1". This should swap the numbers of "1-2-3.tv" and "1-2-3.tv HD".

madkiss commented 11 months ago

Thank you. Gave i a shot, but unfortunately to no avail -- created a standard channel list and also manually edited the MtkChannelList.xml manually, installed everything on the TV but the displayed channel list remained completely unchanged. What would be a good way to support you in debugging? Manually create a favourite list and dump to USB and send you? AIUI, this TV doesn't support editing the channel list directly at all, but tries to force users into creating favourite lists.

(Of course, creating a favourite list manually with four satellites in a unicable setup will suck dead elephant through straws, but if it helps in improving ChanSort, I am more than willing to do it.)

marko-arn commented 11 months ago

With my TV (65PUS8808) I can edit the favourites on the TV. I created a favourite-list and uploaded the complete folder. I hope it helps you to find a solution.

PhilipsChannelMaps.zip

madkiss commented 11 months ago

Just for the record: I can also edit channels on TV to add them to favourite lists. I just think the whole purpose of ChanSort is sort of not having to do that. ;-)

PredatH0r commented 11 months ago

Are your TV set to country "GB" or other countries? I know from previous Philips TVs that they disabled channel editing only when the country was set to "GB" (or "UK" ... not sure) as it seems to be a license requirement for one of the stickers they put on their TVs that they use a fixed channel list. The fav list is the only way to allow customization with that country setting - at least it was like that with formats 110 and 115 (maybe even 105).

Before I go down the rabbit hole and write code to edit the 120 fav list, can anyone please try out if the TV can actually import its own unmodified list? To test that, export the list, change some favorites through the TV's menu and then re-import the previous list. I'm concerned that the TV tries to verify checksums of the files (including Favorites.xml) that are normally stored in chanLst.bin, but the firmware with the 120 format writes garbage, which will probably also cause the import to fail.

madkiss commented 11 months ago

My TV has its country code set to DE for Germany. I will perform the re-import test as requested.

marko-arn commented 11 months ago

I also have the country code 'DE'. I tried the following:

  1. Exporting the channels to USB, swap favourite 1 and 2 at the TV and import to the TV -> import successful and fav. changed
  2. Exporting the channels to USB, edit the favorite.xml and just swap the channel-number of 1 and 2, import to the TV successfull- fav. not changed
  3. Exporting the channels to USB, edit the favorite.xml and just swap the whole block of the channel 1 and 2, import to the TV successfull- fav. not changed

@madkiss could you confirm the beaviour from your side?

PredatH0r commented 11 months ago

Thanks to your files I was able to find out that chanLst.bin in version 120 contains only the lower 8 bits of the 16bit CRC16-IBM/Modbus checksum of the DVBC.xml, DVBT.xml, DVBSall.xml and Favorite.xml files.

https://chansort.com/ChanSort_2023-11-02.zip now reads/validates/updates the chanBin.lst file to ensure correct checksums. It doesn't update MtkChannelList.xml. Hopefully this is not needed.

Can you give this new version a try please?

madkiss commented 11 months ago

Will do right away.

madkiss commented 11 months ago

When trying to open the original channel list, I now get an error message according to which the size of the file exceeds the allowed maximum size.

madkiss commented 11 months ago

Scratch that, my bad. With the new version, the problem is the same though. Changing the list in Chansort and then importing it into the TV doesn't change the all channels list at all.

PredatH0r commented 11 months ago

sigh

I'll have to leave now for some hours. You could give it another try to edit the MtkChannelList.xml too, so it matches the changes done with ChanSort. e.g. swapping program number 1 and 2. There is no checksum for the MtkChannelList.xml in chanLst.bin.

marko-arn commented 11 months ago 
2. Exporting the channels to USB, edit the favorite.xml and just swap the channel-number of 1 and 2, import to the TV successfull- fav. not changed

3. Exporting the channels to USB, edit the favorite.xml and just swap the whole block of the channel 1 and 2, import to the TV successfull- fav. not changed

I was wrong, the favourites were changed, but the TV was set back to show all programmes instead of the fav-list. Sorry it was my mistake.

marko-arn commented 11 months ago

Thanks to your files I was able to find out that chanLst.bin in version 120 contains only the lower 8 bits of the 16bit CRC16-IBM/Modbus checksum of the DVBC.xml, DVBT.xml, DVBSall.xml and Favorite.xml files.

https://chansort.com/ChanSort_2023-11-02.zip now reads/validates/updates the chanBin.lst file to ensure correct checksums. It doesn't update MtkChannelList.xml. Hopefully this is not needed.

Can you give this new version a try please?

I tried it. First thing I discovered is that the current favorite.xml is not loaded. But I´m not sure if this is right or if it should be loaded.

Even though the service-type ist not read properly and the flag for enrypted is not set: grafik grafik

The favourites are not imported to the TV after saving within the programme. With the original files there was a message that the import was successful, but with the edited files by chansort not. The favourites were afterwards deleted and emtpy on the TV.

I re-imported the previous favourite-list from my backup and left the ChanBin.lst like saved from ChanSort. It works, so maybe there is no change needed for the ChanBin.lst.

PredatH0r commented 11 months ago

Thank for the info. I could fix the service type (they are now writing the actual DVB service type value to the file and not just 1=TV, 2=Radio). The "Scrambled" attribute has value 0 for all channels, so there is no way to know which ones are encrypted.

I also noticed that there's something wrong with Umlaut characters in the channel names that I have to look into.

And they seem to have added extra data to the Favorite.xml file that ChanSort didn't write to the file. I'll also fix that and there should be a new build later today.

madkiss commented 11 months ago

I did note the Umlaut problem but forgot to mention that, thank you.

PredatH0r commented 11 months ago

new build: https://chansort.com/ChanSort_2023-11-03.zip

I tested this version with marko's file, loading it and immediately saving it again and comparing the modified files against the original ones. There are 0 differences now, which should increase chances that actual changes are now working.

marko's list seems to be a predefined List. I guess you selected something like "Astra 19.2 Senderliste Deutschland". With most TVs, such predefined lists cannot be edited or just cause random behavior after imported in the TV. You can keep such a list around as a reference list for applying the order to another file, but the actual file being edited with ChanSort should always come from a "Full search" (Blindsuche).

These predefined lists are also the reason why you are not allowed to edit the channel numbers in the TV's menu. (Although I can't guarantee that it will allow you to edit the numbers after running a full search).

But either way, this new version should now work at least for editing the favorite list.

madkiss commented 11 months ago

Will test right away. Thank you for the hint concerning the channel list -- I did not choose any of the lists when the TV asked me to (there is no pre-defined list for my setup I think). I know for sure my TV did a full search, I just don't know in what way it stored it. I was a bit flabbergasted by the fact that even though all satellites were searched (and the time it took to search was adequate for four satellites), but the channels were still listed per-satellite in the overall list after the scan.

Will report back after testing.

marko-arn commented 11 months ago

Thanks @PredatH0r!

I did two tests. First I exported the current setting of my tv and edited it with the newest version of ChanSort. I changed the favourites and these change were imported into the tv. I also changed the full list (just swapped two programmes) and this was not imported.

When I installed the tv is discovers automatically that it is an Astra 19.2 Sat and gives the option to choose either German, Austrian etc. lists and on the buttom a fast scan or others. When selecting others it starts a full search and every programme is only once in the list.

I opened this list with ChanSort and tried to create a new fav-list, which was not imported into the tv. I also tried to change the full list an re-order some prgrammes. This was also not imported, but an change to one name was. I changed from 1-2-3.tv to 11-2-3.tv which works, but the coding of the hyphen was not correct. grafik

I also tried to find out where the information about a encrypted programme is saved, as the TV shows it as a little key-symbol next to the name. But I was not successful.

My conclusion is: If the fav-list is created you can edit it with ChanSort, which is more then enough for me. I prefer the fav-list beside the full list, so I can have all programmes in one and my favs in the other.

tanGERra commented 10 months ago

If someone needs another philips channel-list in version 120 from a 55OLED908, just let me know.

Thanks so far for the test-build which handels that type of lists. I will try my luck later.

tanGERra commented 10 months ago

So, I sorted my favorites list and recognized an issue:

If I rename a channel with ChanSort, the corresponding xml field "ChannelName" in the file "DVBSall.xml" is updated. While updating the ascii 'utf-8' code in hex format, ChanSort uses capital letters like '0x6E' for 'n'. This leads to a wrong depiction on the tv. Manual editing all the capital letters in the "DVBSall.xml" field "ChannelName" to lowercase letters (for the example above '0x6e') fixes that issue in my case.

May sound weird, but this is my observation. ;-)

PredatH0r commented 10 months ago

Good observation, thanks for letting me know. With formats 115 and earlier, Philips hat lowercase hex letters in Favorite.xml and only wrote as many characters as needed, while the DVB*.xml files had uppercase hex digits and were padded to a fixed number of characters.

Now with format120 it seems they switched to lowercase hex everywhere, but still pad the channel names with 0x00 to a fixed length while favorite list names are only as long as needed.

Philips always finds new ways to mess things up...

tanGERra commented 10 months ago

Thanks for your information.

Philips always finds new ways to mess things up...

That´s a really stupid implementation of philips. Doesn´t matter if the tvs write their files with lowercase letters, interpreting all files with both, lower- and uppercase letters, should be standard.

Another thing that I recognized is, that changing the channel names with ChanSort, has only effect on the "DVBSall.xml" field "ChannelName". The "service_name" field in "MtkChannelList.xml" stays original. After transfering the list to the tv and again copying it to an usb-drive, the "service_name" from "MtkChannelList.xml" is updated with the changed name from "ChannelName" out of "DVBSall.xml". A bit offtopic: For now, I don´t know, which name field is relevant for switching to a channel via alexa voice command. Do you have any idea how to figure out?

PredatH0r commented 10 months ago

The "encoding" used by Philips is the most stupid possible. They use 2 bytes, but that's not UTF-16 as one might guess, but instead the first byte is based on some locale specific code page (i.e. CP-1252) and the 2nd byte always 0x00. Add to that some random uppercase/lowercase handling that's inconsistent betweeen files in the same format version and across different format versions.

And then they always duplicate the channel information. One time in DVB*xml files and then depending on the format version as MtkChannelList.xml or some encrypted binary files.

Currently ChanSort does not modifiy the MtkChannelList.xml at all. I can't tell what the TV really does with this file. In prior formats it wasn't required to modify the binary files and the import accepted lists that only changed the DVB*.xml files, including the main channel number.

You could try and change the main channel number with Chansort in the DVB*.xml file and then manually set the same value in MtkChannelList.xml in the major_channel_number tag. The user_edit_flag might be another value that could play a role. But setting "UserReorderChannel=1" inside the DVB*.xml file seem to cause the TV to reject the import complete, from what I've been told from users. Maybe both flags need to have the same value, maybe only the user_edit_flag is relevant, maybe none.

Unfortunately I don't have a Philips TV to test all of this myself.

I don't have any knowledge about Alexa. My best guess is that the TV imports some hardware-platform dependent files first (MtkChannelList.xml in case of format 120) and then applies the Philips-specific "DVB*.xml" over it to allow renaming/reordering. That would explain why the re-export then has the edited name in both files.

Philipsistupid commented 2 months ago

Hello guys

have you found a solution to the problem? I bought a Philips 8800 series and unfortunately the sorting does not work.

It works approximately when I sort the favorites with Chansort and transfer them to the TV via USB.

If I then type the number on the remote control for the memory location I have selected, the stored channel is switched on but the TV says it is on a completely different channel number. Totally strange...

PredatH0r commented 2 months ago

Unfortunately I'm at a dead end.

When you reorder the favorites and type the number on the remote, does it show the channel's main program number or a completely different one from both main and favorite?

The only new information I gathered is that the "MtkChannelList.xml" seems to be (almost) identical to what a Sony Bravia 7 exports. I guess both use MediaTek hardware and firmware inside. But also a manual attempt to rearrange 2 channels for Sony also failed.