EPG Not Loading - Shows Failed

[Aersaud]

Hello I am trying to use an external EPG from my Threadfin docker however I am getting "Failed" on the latest build. The logs do not show anything useful to help us figure out what is happening. The XML is accessible and works via Plex and Jellyfin or via my web browser.

2023-12-30 00:59:16,926 - Run Python 3.11.7 x64
2023-12-30 00:59:16,927 - Build by Nuitka
2023-12-30 00:59:17,207 - Mitmproxy process started
2023-12-30 00:59:17,208 - Epg updater started
2023-12-30 00:59:17,208 - Load epg channels from 'http://REDACTED/xmltv/threadfin.xml'
2023-12-30 00:59:24,814 - Epg updater stopped
2023-12-30 00:59:24,814 - Mitmproxy process exit

Am I doing something wrong or is there anything else I can try?

[sebdelsol]

If I understand correctly this an xml file produced by a Plex server : Could you please share the resulting file produced by this url so that I can check why it fails (it should download in your browser except if there's an authentification required).

EDIT: From what I can seen in the epg in threadfin source it seems to be a correct XML epg format. So it's probably an issue with authentification (I don't do any) so maybe you should allow access from the local network without authentication or add your plex token @ the end of the url

[Aersaud]

Hello! Thanks for the response, I have uploaded the XML - it does not require any authentication to access on my local network. The URL I am inputting into sfvip is the same one I am using in both Jellyfin and Plex. Both are able to pull the xml file. threadfin.zip

I had to zip the file as github does not allow xml upload.

[sebdelsol]

mmm this is a proper xmltv file: my test code finds programmes for 400 channels. Have you tested this url in your browser to check the file is correctly downloaded ?

[Aersaud]

Yes I have that is how I downloaded the xml to upload it.

Is there any way to enable a more verbose log on the epg loader so we can find the issue?

[sebdelsol]

Is there any way to enable a more verbose log on the epg loader so we can find the issue?

yes I could that

I still have to finish the the all category cache for MAC accounts first. I might release a version tomorrow where I'll add more logs for this case. I have a hunch it's because the file is downloaded with a stream request and threadfin might not support it... EDIT: I was wrong

[sebdelsol]

So I've released v1.4.12.5 where errors are logged when downloading the EPG. You'll find the logs files in the logs folder:

cd "%LocalAppData%\Programs\Sfvip All*\logs" 
dir *.log /OD

You'll find the error(s?) in one of the last 2 logs.

[Aersaud]

I see in the logs it is not downloading the file completely? As it shows a size of 0

2024-01-02 12:00:23,593 - Run Python 3.11.7 x64
2024-01-02 12:00:23,593 - Build by Nuitka
2024-01-02 12:00:23,887 - Mitmproxy process started
2024-01-02 12:00:23,888 - Epg updater started
2024-01-02 12:00:23,889 - Load epg channels from 'http://172.30.1.90:30044/xmltv/threadfin.xml'
2024-01-02 12:00:23,928 - Download epg from 'http://172.30.1.90:30044/xmltv/threadfin.xml' has a size of 0
2024-01-02 12:00:37,167 - Epg updater stopped
2024-01-02 12:00:37,167 - Mitmproxy process exit

Also just to verify here is the XML url using Firefox where it does load the full xml file.

[sebdelsol]

~Ok your xmltv server doesn't handle streaming hence the 0 size~. I'll add a fallback without streaming (download progress won't be available for this case).

[Aersaud]

I appreciate it! Are there any other negatives to it not being done via stream? I assume if the xml updates it would have to repull (aka close and reopen the program) ?

[sebdelsol]

Are there any other negatives to it not being done via stream?

My bad it's just that some servers don't inform the 'Content-Length' in their response headers. I'm sure your server streams just fine. Anyway I just use a stream to show download progress information. But in your case you won't miss it since it's a local server that should download the file quasi instantly.

I assume if the xml updates it would have to repull (aka close and reopen the program) ?

No the whole EPG is stored in memory so that you can query the xml data.

Ok the EPG should download in the new upcoming 1.4.12.6 version. EDIT: it's available Now I hope the EPG channel ids will match those given by the XCode API. :sweat_smile: I'm not sure: I have only a few server available to test and those API have very lax specifications... You'll tell me

[Aersaud]

Can confirm it is downloading the EPG now

2024-01-02 15:11:58,300 - Run Python 3.11.7 x64
2024-01-02 15:11:58,300 - Build by Nuitka
2024-01-02 15:11:58,592 - Mitmproxy process started
2024-01-02 15:11:58,592 - Epg updater started
2024-01-02 15:11:58,593 - Load epg channels from 'http://172.30.1.90:30044/xmltv/threadfin.xml'
2024-01-02 15:11:59,208 - Epg channels updated from 'http://172.30.1.90:30044/xmltv/threadfin.xml'
2024-01-02 15:11:59,209 - 400 Epg channels found
2024-01-02 15:15:45,483 - Load epg channels from 'http://172.30.1.90:30044/xmltv/threadfin.xml'
2024-01-02 15:15:46,017 - Epg channels updated from 'http://172.30.1.90:30044/xmltv/threadfin.xml'
2024-01-02 15:15:46,017 - 400 Epg channels found
2024-01-02 15:17:42,651 - Epg updater stopped
2024-01-02 15:17:42,651 - Mitmproxy process exit

However I'm guessing the ID's don't match because there is nothing displaying when accessing the EPG.

Just to verify the EPG is viewed by pressing E on the keyboard while the spvip player is focused correct?

[sebdelsol]

Just to verify the EPG is viewed by pressing E on the keyboard while the spvip player is focused correct?

yes

However I'm guessing the ID's don't match because there is nothing displaying when accessing the EPG.

Ok could you try this url in your browser:

http://IPTV_SERVER:PORT/player_api.php?username=USERNAME&password=PASSWORD&action=get_live_streams Where IPTV_SERVER:PORT is your iptv server url (with its port). USERNAME & PASSWORD are your server IDs.

It should give a huge response (that's the list of all channels). And I'm interested in some channel attribute (epg_channel_id) that helps with finding the correct EPG for each channel. There's no private information like your id & password in this response so it's safe to share. If you can share it I might be able to fix the issue.

[Aersaud]

Heres the entire JSON -- giant file.

player_api.php.json

[sebdelsol]

There are no matches indeed (epg_channel_id is either null or in a very different format than the ones given in the xmltv EPG where channels' ids are mere integers). Funnily enough the channel icons urls seem to match for both but that's a very poor heuristic (I'm sure a lot of channels in the same network share their logos).

I'll check if I can find a way to find some matches with other channels' attributes (prolly XCode channels name and xmltv channels display-name) hoping for enough matches without too much false positives.

I'm interested to know whether you've found another player where this external epg worked with this XCode iptv provider :)

[Aersaud]

I haven't used any external players aside from Plex and Jellyfin. They auto map and work correctly there. I am also using the same m3u that Threadfin created with that xml file.

I haven't found any other external players (aside from Plex and Jellyfin) that have epg/xml capability on Windows hence why I am here 😀

[sebdelsol]

I am also using the same m3u that Threadfin created with that xml file.

Ok I understand: you use Threadfin that generates an xml and an m3u url. I'm sure both share their channel ids. This epg is only meant for this specific m3u (the ids used are integers without any semantic).

Anyway m3u are transparents for Sfvip All: I plan to handle the online ones in the future (I can't do anything for the local ones since there are no web requests to intercept)...

Now I guess the Xtream Code Api response (the json file) you gave me doesn't come from the same Threadfin ?

[Aersaud]

No it does not, I am not sure if Threadfin can produce a similar json. I was able to grab how it does display the info in json format however.

  "x-ID.89": {
    "_file.m3u.id": "MFYYW4XUO28IMYHYM90Q",
    "_file.m3u.name": "[REDACTED]",
    "_file.m3u.path": "/home/threadfin/conf/data/MFYYW4XUO28IMYHYM90Q.m3u",
    "group-title": "USA Entertainment",
    "name": "Fuse",
    "tvg-id": "fusetv.us",
    "tvg-logo": "http://181.233.124.55:80/images/a3cf68cde4ce9930116bba5496717257.png",
    "tvg-name": "Fuse",
    "url": "[REDACTED]",
    "_uuid.key": "tvg-id",
    "_uuid.value": "fusetv.us",
    "_values": "fusetv.us Fuse USA Entertainment Fuse",
    "x-active": true,
    "x-category": "",
    "x-channelID": "1089",
    "x-epg": "x-ID.89",
    "x-group-title": "USA Entertainment",
    "x-mapping": "fusetv.us",
    "x-xmltv-file": "XHA2NLF7VGMPKB8125VC.xml",
    "x-ppv-extra": "",
    "x-backup-channel-1": "",
    "x-backup-channel-2": "",
    "x-backup-channel-3": "",
    "x-hide-channel": false,
    "x-name": "Fuse",
    "x-update-channel-icon": false,
    "x-update-channel-name": false,
    "x-description": "",
    "live": false,
    "is_backup_channel": false,
    "backup_channel_1_url": "",
    "backup_channel_2_url": "",
    "backup_channel_3_url": ""
  },
  "x-ID.9": {
    "_file.m3u.id": "MFYYW4XUO28IMYHYM90Q",
    "_file.m3u.name": "[REDACTED]",
    "_file.m3u.path": "/home/threadfin/conf/data/MFYYW4XUO28IMYHYM90Q.m3u",
    "group-title": "USA Entertainment",
    "name": "A\u0026E SD",
    "tvg-id": "aande.us",
    "tvg-logo": "http://23.227.147.170:80/images/0344d9238ba4ae5e410487ba4c3e8240.png",
    "tvg-name": "A\u0026E SD",
    "url": "[REDACTED]",
    "_uuid.key": "tvg-id",
    "_uuid.value": "aande.us",
    "_values": "aande.us A\u0026E SD USA Entertainment A\u0026E SD",
    "x-active": true,
    "x-category": "",
    "x-channelID": "1009",
    "x-epg": "x-ID.9",
    "x-group-title": "USA Entertainment",
    "x-mapping": "aande.us",
    "x-xmltv-file": "XHA2NLF7VGMPKB8125VC.xml",
    "x-ppv-extra": "",
    "x-backup-channel-1": "",
    "x-backup-channel-2": "",
    "x-backup-channel-3": "",
    "x-hide-channel": false,
    "x-name": "A\u0026E SD",
    "x-update-channel-icon": false,
    "x-update-channel-name": false,
    "x-description": "",
    "live": false,
    "is_backup_channel": false,
    "backup_channel_1_url": "",
    "backup_channel_2_url": "",
    "backup_channel_3_url": ""
  },

I have redacted the URL's and M3U name but the rest is how the info is served. I hope this can help.

The m3u name and xml name are displayed with their ID's here however it just ends up parsing out to threadfin.m3u for the actual IPTV streams and threadfin.xml for the EPG

Perhaps x-mapping can be used? as that is where it actually maps the channel to the xml EPG from the provider's m3u.

In regards to:

Anyway m3u are transparents for Sfvip All: I plan to handle the online ones in the future (I can't do anything for the local ones since there are no web requests to intercept)...

Threadfin does produce have a local webserver that exposes the m3u and xml file in my case http://172.30.1.90:30044/m3u/threadfin.m3u and http://172.30.1.90:30044/xmltv/threadfin.xml

Here is an example of how the m3u and xml/epg are imported and displayed in Jellyfin.

[sebdelsol]

Perhaps x-mapping can be used? as that is where it actually maps the channel to the xml EPG from the provider's m3u.

Yes it would be helpful but I don't have access to that.

It's a shame threadfin can't produce an epg with correct channels id (eg. "fusetv.us" instead of "1234"). I see it has all the correct data internally (the "tvg-id" attribute).

EDIT: I've tried to use epg display's names: it gives some results with way too much false positive. EDIT2: I'm trying now with a fuzzy search to see if I can have results AND false positives cut off. EDIT3: This looks promising, expect a version tomorrow.

[sebdelsol]

There are ~500 channels in your epg, only ~400 have programmes. There are ~8000 channels in your json. ~400 find a match in the epg.

There are still ~10% of probable false positives: So I might add some UI to show the epg match confidence and a confidence slider. The higher, the laxer, which means asking for:

• 0% confidence will give less matches but better matches.
• 100% confidence will always give a match but often false.

EDIT: ~Anyway UI is always a PITA so I'll do that later~. I think I'll publish v1.4.12.7 with this feature ~today~ soon.

EDIT2: The default value for EPG confidence is 30% which seems the sweet spot between enough matches without too much false positives.

[sebdelsol]

Ok it took way longer than I though. Anyway it should be way better in v1.4.12.7. Please check the changelog and report back any issue.

EDIT: thanks a lot for your help. EDIT2: v1.4.12.9 fix yet another EPG loading bug. :sweat_smile:

[Aersaud]

Not sure what I am doing wrong not getting any EPG info when I press 'e'. I have tried confidence down to 1% and the EPG states is loaded.

2024-01-09 13:57:29,065 - Run Python 3.11.7 x64
2024-01-09 13:57:29,065 - Build by Nuitka
2024-01-09 13:57:29,429 - Mitmproxy process started
2024-01-09 13:57:29,429 - Epg confidence updater started
2024-01-09 13:57:29,430 - Epg updater started
2024-01-09 13:57:29,431 - Load epg channels from 'http://172.30.1.90:30044/xmltv/threadfin.xml'
2024-01-09 13:57:30,057 - 497 Epg channels found from 'http://172.30.1.90:30044/xmltv/threadfin.xml'
2024-01-09 13:58:31,778 - Epg updater stopped
2024-01-09 13:58:31,778 - Epg confidence updater stopped
2024-01-09 13:58:31,779 - Mitmproxy process exit

[sebdelsol]

I have a lot of matches with the files you uploaded. I've you tried with a lot of channels ?

[Aersaud]

I went through about 20 channels and 'e' did not show anything for me, I also tested 'c' which brings up the clock so I know the keyboard commands are working atleast.

[sebdelsol]

It's weird. it works fine for me. OK could you please upload again the combination of files that fails:

• the Threadfin.xml
• the json file for the provider channels, see here https://github.com/sebdelsol/sfvip-all/issues/3#issuecomment-1874574885

[Aersaud]

Here are 3 files threadfin.xml threadfin.m3u JSON from provider

I provided the m3u that is proxied thru my local server so its just my lan IP showing. This is also the same m3u I am feeding into sfvip

3files.zip

[sebdelsol]

I have tried confidence down to 1% and the EPG states is loaded

1% it means you barely trust the EPG match so you'll have way less matches. 30% is ok (lot of results, not too many false positives). 100% means you trust so much the epg that you always have a match (and those might be of very poor quality).

Maybe I should add a tooltip to make it clear.

This is also the same m3u I am feeding into sfvip

There are no epg at all for m3u in Sfvip Player :). That explain a lot :sweat_smile: I'm adding it at the moment in Sfvip All, but it's a lot of UI... so quite painful to do.

Anyway external EPG works for XC accounts (like you player_api.php.json) or MAC accounts.

[Aersaud]

There are no epg at all for m3u in Sfvip Player :). That explain a lot 😅 I'm adding it at the moment in Sfvip All, but it's a lot of UI... so quite painful to do.

Ah that explains a lot why EPG isn't showing. I look forward to the release whenever its done. Thank you!

[sebdelsol]

I look forward to the release whenever its done.

v1.4.12.11 has a basic epg for online m3u accounts: Only the current programme is shown at the moment as a proof of concept. I'll add the whole schedule (shortcut e) in a later version.

EDIT: v1.4.12.12 shows the whole schedule with the e shortcut. Still a lot to do....

EDIT2: In the current dev version: Tool-tip added to explain(?) what the the EPG confidence level actually means.

EDIT3: v1.4.12.20 should do the job, please tell me if that's not the case.