search

kodi-pvr / pvr.iptvsimple

IPTV Simple client for Kodi PVR
GNU General Public License v2.0
774 stars 372 forks source link

Infinite loading of a large compressed EPG on Android 9. #680

Open ivanmara555 opened 1 year ago

ivanmara555 commented 1 year ago

Hello. My EPG provider provides two versions of compressed renewable xmltv EPG:

https://iptvx.one/EPG_LITE # no descriptions, just the title of the show, uncompressed size: ~200Mb and https://iptvx.one/EPG # full version, uncompressed size: ~450Mb.

All this versions work fine in kodi on Linux, Windows, Android 11.

But FULL version of EPG leads to endless loading of EPG in Kodi (all verions) and IPTV Simple (all versions) on Android 9 TV box (x92 3/16Gb). Nevertheless lite version works fine!

I started looking into the issue. Create local web server on my lan and received compressed and uncompressed variants of FULL epg:

wget -qO - https://iptvx.one/EPG | zcat > epg.xml # get UNcompressed epg file: epg.xml
wget https://iptvx.one/EPG                        # get   compressed epg file: epg.xml.gz

After converting URLs to local web server I added one by one to the IPTV Simple configuration file, restarts kodi, clear the EPG cache and get some statistisc:

http://router.lan:8000/epg.xml     # UNcompressed: works fine
http://router.lan:8000/epg.xml.gz  #   compressed: INFINITE EPG LOADING.

After load it locally on tv box and editing config file:

epg.xml     # UNcompressed: works fine
epg.xml.gz  #   compressed: INFINITE EPG LOADING.

I thought there was not enough memory and turned on the swap 1G but swap does not used while infinite epg loading (cpu usage 100%!):

X92:/ # free -h
        total        used        free      shared     buffers
Mem:             2.8G        2.1G        718M        3.5M         25M
-/+ buffers/cache:           2.1G        743M
Swap:            1.2G           0        1.2G

Something wrong with uncompression operation in IPTV Simple or missing component on Android 9 or something alse.

I am attaching the log file to the post with all category logging enabled + settings file.

Can you help me please ?

log_and_config.zip

phunkyfish commented 1 year ago

What do you mean by infinite? You mean you have left it for days to load? Likely it is limited by processor speed.

Also look at the settings for Channel groups, they can help reduce load times. Also Nexus has a new feature for VOD which should also help.

phunkyfish commented 1 year ago

BTW 450mb is insanely huge.

ivanmara555 commented 1 year ago

What do you mean by infinite?

i mean: endless loading epg (sorry my english) ... the process of import epg will never ends ... i wait waited about an hour and process not stoped ... this is not normal behavior ... import of uncompressed epg from network going ~3 min ... in log you can see some looping process after downloading .... i don't know what is it ... any ideas ?

ps: as i said this full epg import normal on other devices and OSs ... i use m3u IPTV play list with 2500+ channels and this epg for it.

phunkyfish commented 1 year ago

Leave it for a day and see if it loads.

ivanmara555 commented 1 year ago

Leave it for a day and see if it loads.

Ok ... but I've been trying to solve this problem for a couple of years ...

phunkyfish commented 1 year ago

Then what’s one more day? 😉

ivanmara555 commented 1 year ago

Hello. I wanted to ask if you looked further into my problem ?

phunkyfish commented 1 year ago

What happened when you waited a day for the EPG to load?

ivanmara555 commented 1 year ago

Nothing ... compressed epg is imported endlessly ...

phunkyfish commented 1 year ago

Can you upload the debug log to a paste site?

ivanmara555 commented 1 year ago

https://pastebin.com/UdT0tM4Y https://pastebin.com/qNwGMrCH https://pastebin.com/C6SubUst https://pastebin.com/S5CJFDbZ

Complete log and settings file attached to first message (log_and_config.zip).

phunkyfish commented 1 year ago

All I see is this log message, which doesn’t explain anything but may be an issue with the network.

2023-01-10 01:20:52.419 T:5739    DEBUG <general>: CFileCache::Process - <http://router.lan:8000/epg.xml.gz> source read hit eof

Have you tried copying the epg file to the device locally and loading it from there?

ivanmara555 commented 1 year ago

Yes ... i tried to load it localy (I wrote about it in the first post). The results are:

epg.xml     # UNcompressed: works fine
epg.xml.gz  #   compressed: ENDLESS EPG LOADING.

Any ideas ?

phunkyfish commented 1 year ago

Can we find out what is the maximum size compressed file that works?

Uncompress the file, take 50%, compress it and see what happens. If it works try 75%, if not 25%. Keep on dividing and conquering until you find the limit.

My guess is you require (compressed file size * 2) for it to work. But let’s see what the results are.