Movistar+ 19.2 missing channels

[miberian]

A number of channels are not added to the bouquet. All of them are SD and SD only. For example:

[ABM-DvbScanner] SDT service {'transport_stream_id': 1060, 'original_network_id': 1, 'service_id': 30613, 'service_type': 1, 'free_ca': 1, 'service_name': 'CANAL DECASA', 'provider_name': 'PRISA TV', 'logical_channel_number': 0, 'bouquets_id': 0, 'service_group_id': 0, 'category_id': 0, 'region_code': 0, 'city_code': 0}
[ABM-DvbScanner] Above service not in LCN dict

This particular channel should be placed at number 93 according to the provider: https://www.movistarplus.es/canal/canal-decasa?id=DECASA

Full debug attached for a single run using default provider configuration.

debug.log

[miberian]

The information of the missing channels is actually available in descriptor_tag 0x82:

13:14:51.6588 [ABM-DvbScanner] hexcontent {'descriptor_tag': 130, 'descriptor_length': 56, 'hexcontent': '8238778E0130778F003F77890043778D0052778B00BE7796002C7795005D778A0BB87790002A7797003677930BB8778C00057792000477980034'} 13:14:51.6632 [ABM-DvbScanner] hexcontent {'descriptor_tag': 130, 'descriptor_length': 56, 'hexcontent': '8238746B006F746C0015746D0016746E0017747100117468004274770027746F002674690060746A004E7478000E747200537473009C74740002'} 13:14:51.6642 [ABM-DvbScanner] hexcontent {'descriptor_tag': 130, 'descriptor_length': 72, 'hexcontent': '824875FB01DE760E0BB876010BB8760A0BB876040BB87607003875F90BB875FE0BB875FF00C2760200C3760B0BB8760D0BB8760F0BB876100BB87600001376080BB875FA003B760600B2'} 13:14:51.6648 [ABM-DvbScanner] hexcontent {'descriptor_tag': 130, 'descriptor_length': 56, 'hexcontent': '823877C1001F77C40BB877BC003177C6001477BD00A777C3003D77D800B477C7007377C2004177BF00BF77C800C077BE00C477C000C577C90BB8'} 13:14:51.6653 [ABM-DvbScanner] hexcontent {'descriptor_tag': 130, 'descriptor_length': 56, 'hexcontent': '823876C0000C76CA000D76C3007476CC004B76C7001E76C8002976C9003776CE005C76CF001976C50BB876C2000176C4006376C1007676CB0028'} 13:14:51.6662 [ABM-DvbScanner] hexcontent {'descriptor_tag': 130, 'descriptor_length': 60, 'hexcontent': '823C756800A2756900A375700BB8756C0131757B003C75740BB875820BB8758401327577007D75870032756B0024757F001275880007756D004475790BB8'} 13:14:51.6667 [ABM-DvbScanner] hexcontent {'descriptor_tag': 130, 'descriptor_length': 60, 'hexcontent': '823C7730002B772B004D773100547738007F772D0BB87736001A772A004777320BB8772F005577290097772E007277390040772C00037733000877340035'} 13:14:51.6672 [ABM-DvbScanner] hexcontent {'descriptor_tag': 130, 'descriptor_length': 56, 'hexcontent': '8238749A005F749B0BB8749C006E749E0075749F003074A00BB874A2007874A3004A74A1004874A40045749D000F74A6000674A7004974A80025'} 13:14:51.6676 [ABM-DvbScanner] hexcontent {'descriptor_tag': 130, 'descriptor_length': 60, 'hexcontent': '823C768E00C6769500A476920BB876A3000B769600187699007B76A4004676A50BB876A70BB8769A0BB8769700717691003A769B00C1769D0039769F003E'} 13:14:51.6681 [ABM-DvbScanner] hexcontent {'descriptor_tag': 130, 'descriptor_length': 100, 'hexcontent': '826475AA0BB875CF0BB875A900C875F20BB875EE0BB875F10BB875F30BB875D90BB875DD01BC75AF0BB875CA0BB875B200A575C70BB875DE025975DF025A75E0025B75E1025C75E20BB875E30BB875C600C775AB0BB875F50BB875EF0BB875960BB8759E000A'}

For the example above (service 30613 or 0x7795), the information available in the logs matches the provider information (channel number 93):

hexcontent {'descriptor_tag': 130, 'descriptor_length': 56, 'hexcontent': '8238778E0130778F003F77890043778D0052778B00BE7796002C7795005D778A0BB87790002A7797003677930BB8778C00057792000477980034'}

0x005D = 93.

[miberian]

Code changes in pull request #277

[AbuBaniaz]

If there are no objections by Friday, I will merge it.

For which architecture did you compile the reader? Maybe you can share it.

[miberian]

Compiled for arm and tested in openatv 7.3 + Gigablue UE 4K.

arm-linux-gnueabihf-gcc -mcpu=cortex-a15 -mfpu=neon-vfpv4 -Wall -O

dvbreader.so.zip

[AbuBaniaz]

Sorry, I missed you post and run a build with your changes.

There are lost of extra channels. I end up missing following channels 418, 432, 441, 471, 473

Is that to be expected?

With changes hd_sat_192_movistar_plus_esp_CustomLCN_post_miberain.xml.txt

Without changes hd_sat_192_movistar_plus_esp_CustomLCN_pre-miberian.xml.txt

Debug log. (Sorry also had 28.2 in there) Enigma2_debug_2023-09-19_00-42-27.log.txt

[miberian]

Thanks. No, that is not expected. Those are SD versions of HD channels. In your run this is affecting the following channels:

<configuration lcn="418" channelnumber="418" serviceid="30088" description="Movistar Plus+"></configuration>
<configuration lcn="432" channelnumber="432" serviceid="30061" description="M+GOLF 2"></configuration>
<configuration lcn="441" channelnumber="441" serviceid="30079" description="M+ Música"></configuration>
<configuration lcn="472" channelnumber="472" serviceid="30073" description="Movistar Plus+ 2"></configuration>
<configuration lcn="473" channelnumber="473" serviceid="30075" description="M+LCAMPEONES"></configuration>

If we take serviceid 30088, information is present in the log:

{'bouquet_id': 33, 'transport_stream_id': 1042, 'original_network_id': 1, 'service_id': 30088, 'visible_service_flag': 0, 'logical_channel_number': 418, 'descriptor_tag': 131}
{'bouquet_id': 33, 'transport_stream_id': 1042, 'original_network_id': 1, 'service_id': 30088, 'logical_channel_number': 7, 'descriptor_tag': 130}

There must be some part of the code overriding one of the records. The service hacks in the xml should ignore/skip the second registry given that there is a HD version of the channel in the same LCN:

{'bouquet_id': 33, 'transport_stream_id': 1016, 'original_network_id': 1, 'service_id': 29908, 'visible_service_flag': 0, 'logical_channel_number': 7, 'descriptor_tag': 131}
<configuration lcn="7" channelnumber="7" serviceid="29908" description="Movistar Plus+ HD"></configuration>

This is better behaviour that before the changes, but should be looked at.

[miberian]

I believe the issue is this piece of code in dvbscanner.py:

    def readLCNBAT(self, bouquet_id, descriptor_tag, TSID_ONID_list):
...

        logical_channel_number_dict = {}

        for service in bat_content:
            if service["descriptor_tag"] != descriptor_tag:
                continue

            key = "%x:%x:%x" % (service["transport_stream_id"], service["original_network_id"], service["service_id"])
            TSID_ONID = "%x:%x" % (service["transport_stream_id"], service["original_network_id"])

            logical_channel_number_dict[key] = service

key is the same for a channel in different regions so only the last one coming remains in the dictionary. This is consistent with the debug log which shows that the missing channel is processed in the first place (and hence overriden):

00:43:16.5241 [ABM-DvbScanner] LCN entry 412:1:7588 {'bouquet_id': 33, 'transport_stream_id': 1042, 'original_network_id': 1, 'service_id': 30088, 'visible_service_flag': 0, 'logical_channel_number': 418, 'descriptor_tag': 131}
...
00:43:16.5251 [ABM-DvbScanner] LCN entry 412:1:7588 {'bouquet_id': 33, 'transport_stream_id': 1042, 'original_network_id': 1, 'service_id': 30088, 'logical_channel_number': 7, 'descriptor_tag': 130}

So I am going to try the following:

• For all regions in scope, apply different priorities for processing. In the case of Movistar, region 0x83 should prevail over 0x82.
• Add to logical_channel_number_dict only those registries with higher priority.

Service hacks still needed for edge case (M+ Vamos SD).

[miberian]

Issue should be fixed in pull request #278. No additional changes to reader so the same binary above can be used for testing: https://github.com/oe-alliance/AutoBouquetsMaker/files/12612149/dvbreader.so.zip

[AbuBaniaz]

Thanks.

We had a few issues on 0.8w if you want to look at it too.

[miberian]

Great, thanks. Happy to have a look at 0.8w but please note I can't tune that. But if there is a debug log and description of the issue at least I can try to understand what's going on.

[AbuBaniaz]

There are a couple of issues open. But no logs. Thanks for the offer to look at it, but looks like it wont be possible.

[miberian]

@AbuBaniaz thanks for the feedback. Created pull request #280 to achieve same results for Movistar+ without code changes to dvbscanner. I am sorry I guess I was not familiar enough with the plugin when I started looking at it.

[AbuBaniaz]

This is how the channels appear for me with those changes. Let us wait for a while for any constructive feedback/criticism from others.

EXAMPLE_hd_sat_192_movistar_plus_esp_CustomLCN.xml.txt

[miberian]

Brilliant. I get the same results.