Duplicate programme element in generated epg file

[arjunior26]

Hi,

In case two or more channels use the same tvg-id value in m3u file list, programme element is created several times. This happens for channels with multiple streams for example (SD and HD).

Programme elements are actually created for every channel present in the m3u_entries. Maybe programme tag could be stored and then compared with the next tag before append it to epg file ?

What do you think ? Could you take a look at this ?

Thanks.

[bebo-dot-dev]

Hi again, this one sounds like an m3u file data quality induced bug. The fix will be something similar to your suggestion. I'll take a look.

[arjunior26]

Great ! Maybe just store then compare tvg-id value before creating programme element should be enough...

[bebo-dot-dev]

I've struggled a little bit with this bug because I've not been able to get data into the required state to reproduce this problem with confidence.

I have created a branch here https://github.com/jjssoftware/m3u-epg-editor/tree/epg_dupe_fix and I've committed a candidate fix into that branch that I think might be the fix.

Please give this branch version a try and let me know if it works / if you see any issues. Much appreciated, thanks.

[arjunior26]

Works like a charm !

However, you should store values of tvg-id with lowercased characters to avoid duplicates in case we have same tvg-id defined with and without capital letters (like in my malformed list ;)

tvg_id_unique_entries = {e.tvg_id.lower(): e for e in m3u_entries}.values()

Thank you for your wonderful script ;)

[bebo-dot-dev]

Thanks for confirming and for your tip on forcing toLowerCase. I'll apply that change, merge and we can call this done.

[arjunior26]

Just one thing. Now, channels with tvg-id="" are no more added to no_epg_channels.txt file I don't know exactly if it's a choice but maybe it's better to see them in that file. However, not a real problem for me...

[bebo-dot-dev]

That's an interesting observation because I'm seeing that work as expected here at the moment.

Perhaps one thing to consider is:

https://github.com/jjssoftware/m3u-epg-editor/blob/master/m3u-epg-editor-py3.py#L910

In other words m3u entries with no tvg-id attribute value are only written to no_epg_channels.txt when:

1. The --no_tvg_id / -nt argument is set to true (consider m3u entries with no tvg-id attribute value as valid) and
2. The --force_epg / -feargument is not set to true (do not force epg entries to be created for m3u entries with no tvg-id attribute)

Perhaps there's something subtle going on.

[arjunior26]

Hi,

In other words m3u entries with no tvg-id attribute value are only written to no_epg_channels.txt when:

1. The `--no_tvg_id / -nt` argument is set to true (consider m3u entries with no tvg-id attribute value as valid) **and**

2. The `--force_epg / -fe`argument is not set to true (do not force epg entries to be created for m3u entries with no tvg-id attribute)

Hmm not agree with this or maybe I don't understand what you mean. Actual script contains this:

    for entry in tvg_id_unique_entries:
        if entry.tvg_id is not None and entry.tvg_id != "" and entry.tvg_id != "None":
        ...
        else:
            if not args.no_tvg_id or not args.force_epg:
                    no_epg_channels.append(entry)

So entry should be written to no_epg_channels.txt file if tvg-id value is not set or empty (or value="None")

But of course, programme elements are not written as expected. Maybe is what you mean ?

[bebo-dot-dev]

What I meant by saying this:

1. The --no_tvg_id / -nt argument is set to true (consider m3u entries with no tvg-id attribute value > as valid) OR
2. The --force_epg / -feargument is not set to true (do not force epg entries to be created for m3u > entries with no tvg-id attribute)

..is that it's line 910 that enforces this rule:

https://github.com/jjssoftware/m3u-epg-editor/blob/master/m3u-epg-editor-py3.py#L910-L911

Or to put it another way, it's not correct to say that an entry is written to the no_epg_channels.txt file simply when the tvg-id value is not set or empty (or value="None") because the args / configuration passed to the script still need to be aligned correctly to allow this to happen.

I'm seeing this work as expected here. It might be the case that your args are currently set correctly to allow this and that you're still not seeing an entry written to the no_epg_channels.txt file as expected. If that is the case, it would need further investigation, probably with both your data and the exact args that you're using.

[bebo-dot-dev]

Apologies, one edit applied to my last comment, it's a logical OR on line 910, not a logical AND as at first stated (incorrectly).

The rest remains unchanged.

[arjunior26]

OK...just trying to understand your last post as I was pretty sure to read AND in your previous answer :)

[arjunior26]

So in your case, suppose you have this entry in m3u: #EXTINF:-1 tvg-name="FRANCE 4" tvg-id="" tvg-logo="FRANCE4HD.png" group-title="FRANCE TNTSAT",FRANCE 4 Is information appended to your no_epg_channels.txt file if "force_epg" option is set to false ?

[arjunior26]

My fault ! Didn't paid attention that I had set "no_tvg_id": true in my JSON file !!

[bebo-dot-dev]

No worries, it's a fairly complex set of conditions. I do think that the no_epg_channels.txt logic is ok as-is at the moment but if it isn't, it's been like this for a while ;)