search

K-vanc / Tempest-EPG-Generator

The fastest XMLTV formatted EPG Generator with GUI and the lowest memory usage
https://hub.docker.com/r/kvanc/tempest_epg
Other
108 stars 19 forks source link

Grabs data slow #27

Closed ghost closed 1 year ago

ghost commented 1 year ago

I set 8 channels to grab 4 days worth of data mostly from ontvtonight.com. I had one grab from tvguide but it said url could not be found. Naturally, I can change this one to point to a different source. I had another from tvpassport. This grab took about 30 minutes. I would like to do more channels but if I did 100 I am looking at 5 hours. Is my number of days too long, bad sources, or this is just how long it takes?

ghost commented 1 year ago

I thought I had searched here better before I posted.

I see you have answered this already.

https://github.com/K-vanc/Tempest-EPG-Generator/issues/26

K-vanc commented 1 year ago

Hi @stuckinthe. Speed of the grabbing is usually related with type of the server data, server's response speed and your connection speed. Regarding server data type, I explained how some siteconfigs are working on below comment;

https://github.com/K-vanc/Tempest-EPG-Generator/issues/15#issuecomment-1467329319

All the siteconfigs that you use, are with detail pages which means for each grabbed show, an additional webpage more has to be called.For 500 grabbed show, Tempest has to call minimum 501 webpages. It is usually considered usable when server response speed is less than 1 sec for each show but there are also some nightmare servers such as tvtv.us or tivo which page speed > 2secs even in USA. Unfortunately there is no way of changing server's data structure from my side and you need to find best and fastest alternatives based on your needs. I made some quick comparison with several siteconfigs you mentioned.

For ontvtonight (page speed < 1s) image

For tvguide.com (page speed < 1s) image

For tvtv.us and tvpassport, I need vpn which currently not installed my pc but more or less you know their response speed..

You can consider some faster siteconfigs such as zap2it(no detail page/VPN or US ip required) or tvguide(with detail page but faster response speed(around 0.5 secs per show)).