search

bebo-dot-dev / m3u-epg-editor

a python m3u / epg optimizer
120 stars 27 forks source link

groupmode argument is unsusable #53

Closed trix7777 closed 4 years ago

trix7777 commented 4 years ago

Hi! Thanks for a very useful script! I use it in conjuction with Tvheadend to only get the groups that are of interest to me. Also I use it for the Kodi PVR IPTV Simple Client for VOD and TV-Series. I am using the Latest commit ff974d6 on May 4.

  1. GOAL: Test the argument "groupmode": "keep"

{ "m3uurl": "file:///home//m3u-epg-editor/big.m3u8", "epgurl": "file:///home//m3u-epg-editor/big.xml", "groups": [ "Sweden", "\-\ Music Package \-\", "United States", "United Kingdom" ], "groupmode": "keep", "outdirectory": "/home/user/m3u-epg-editor", "outfilename": "test" }

RESULT: ############################################################################### user@server:~$ /usr/bin/python3 /home/user/m3u-epg-editor/m3u-epg-editor-py3.py --json_cfg /home/user/m3u-epg-editor/args_test.json 2020-07-03T13:18:32.706156 m3u-epg-editor-py3.py process started with Python v3.7.3 (default, Dec 20 2019, 18:57:59) [GCC 8.3.0] 2020-07-03T13:18:32.706211 performing HTTP GET request to file:///home/user/m3u-epg-editor/big.m3u8 2020-07-03T13:18:32.739438 saving retrieved m3u file: /home/user/m3u-epg-editor/original.m3u8 2020-07-03T13:18:32.748955 parsing m3u into a list of objects 2020-07-03T13:18:33.550053 m3u contains 3096 items 2020-07-03T13:18:33.551272 keeping channel groups in this list ['Sweden', '\-\ Music Package \-\', 'United States', 'United Kingdom'] 2020-07-03T13:18:33.577094 filtered m3u contains 0 items 2020-07-03T13:18:33.577717 process completed ############################################################################### ERROR:

  1. GOAL: Test the argument "groupmode": "discard" - same config as above but changed "keep" for "discard". RESULT:
    • Exactly the same console output as in first test.
    • Still missing test.m3u8 and test.xml. CONCLUSION: groupmode argument is unusable. The only way to use the script is to remove the argument completely and settle for the default, which is "keep", when the argument is not added at all.

One more thing I've noticed also, not having to do with the above, there seem to be no way to sort the group order in the output m3u8? In my case I want Sweden first, then the other ones, but no matter what I try in the config, the order remains the same with - Music Package - first. This has no consequence when used with Tvheadend but for the Kodi plugin, it would be easier if the group list order could be changed at least for an alphabetical order. As it is now, different VOD-groups does not follow the order I put them in under the groups argument, but more likely follow the original order from the provider.

There are too many channels to put in the sortchannels argument when using it in the VOD and TV-series context so this is not a solution for me.

BTW, am I correct there is no way to keep comments in the json-config? Also, the "range" argument, is that an hours value to cut down the epg to? xml_sort_type as I get it, is to either do not sort at all (with argument missing), "alpha" as in alphabetical order in the epg.xml, and m3u as in order taken from the m3u (input or output m3u?).

Thanks again! Will check back if you need logs or anything.

bebo-dot-dev commented 4 years ago

You found a bug in the json args parse bit - fixed in https://github.com/jjssoftware/m3u-epg-editor/commit/11dd78fceec542096fd00f0ccf92c869b3d08611

Please pull the latest and give it another try, thanks.

trix7777 commented 4 years ago

Thank you! "keep" now works as expected. "discard" discarded the group "Sweden" (I used only this group in this test) in the "groups" list plus a no-named group with 42 channels. Should there be a way to keep even the no-named group or a search and replace for 'group-title=""' with 'group-title="no-group"' within your script?

Logfile bugs when using "discard": 2020-07-05T14:02:34.905893 keeping channel groups in this list ['Sweden'] <-- shouldn't it say "discarding channel groups in this list"? 2020-07-05T14:02:34.923333 desired channel sort order: ['Sweden'] <-- Sorting by a group discarded?

bebo-dot-dev commented 4 years ago

Should there be a way to keep even the no-named group or a search and replace for 'group-title=""' with 'group-title="no-group"' within your script?

m3u entries that have no populated group-title attribute are treated as invalid at source data processing time so these entries don't make it into the newly generated target m3u file at the moment. Since group transforms are applied after m3u source parsing there's no way to convert an empty group name into some named group. I guess that there is an argument to say that something should / could happen but it is what it is at the moment.

2020-07-05T14:02:34.905893 keeping channel groups in this list ['Sweden'] <-- shouldn't it say "discarding channel groups in this list"? 2020-07-05T14:02:34.923333 desired channel sort order: ['Sweden'] <-- Sorting by a group discarded?

Fair comment I'll take a look, these are likely to be minor output logic bugs.

bebo-dot-dev commented 4 years ago

The log output issues that you mentioned should be fixed in https://github.com/jjssoftware/m3u-epg-editor/commit/e38e50230997509f51b055734d03ad413100bf23

A tweak has been applied to item sorting too -> if you don't supply a specific channel sort order and rely on script default sort behaviour, you might find that this behaves in a slightly more sane way now, sorting alphabetically by group and channel.

Please pull the latest and give it another try, thanks.

trix7777 commented 4 years ago

Great work! =) Thank you very much! I love this script! The sorting seem to be after the eventual group_transforms which is just the way I like it!

You can close this issue! Thanks again!

Also, to give other people answers to my own questions above;

bebo-dot-dev commented 4 years ago

Thank you for the feedback, it is much appreciated and issues like this help to improve things for future users.

I didn't address your questions and comments earlier because I was keen to focus on the issues you had raised rather than to try and engage in a Q+A at the same time as attempting to address the bugs. To address your points:

  1. Comments are generally not supported in vanilla json, this feature was never introduced into the json spec by its creator Douglas Crockford. Some efforts have been made since to introduce support for comments in json and there are some flavours of json and json parsers in existence today that have support for comments. The default python json parser doesn't support comments at this time.
  2. You are correct about the range argument, this parameter is an a hour value that can be used to reduce the volume of data output into the generated xml epg data file. When this argument is not specified, the value defaults to 168 hours (7 days). https://github.com/jjssoftware/m3u-epg-editor/blob/master/m3u-epg-editor-py3.py#L811
  3. The xml_sort_type argument was an option introduced due to a suggestion / request made by somebody in this issue, there are some comments in that issue that describe rationale for the feature: https://github.com/jjssoftware/m3u-epg-editor/issues/36

Thanks again, closing.