tvgrabbers / tvgrabpyAPI

An xmltv-API for extracting and merging tv programme information from several sources
https://github.com/tvgrabbers/tvgrabpyAPI/releases/latest
GNU General Public License v3.0
27 stars 8 forks source link

This version is exceedingly slow #12

Closed pensionado closed 7 years ago

pensionado commented 7 years ago

Yesterday I used this version for my daily updates, and these are the summary statistics: 2017-02-23 07:13:51 : Verzamel statistieken van 1721 programma's voor 16 zenders: 2017-02-23 07:13:51 : Start tijd: 2017-02-23 06:52 2017-02-23 07:13:51 : Eind tijd: 2017-02-23 07:13 2017-02-23 07:13:51 : Duur: 0:21:26.437174 2017-02-23 07:13:51 : 114 pagina('s) opgehaald, waarvan 5 faalden 2017-02-23 07:13:51 : 49 cache vonst(en) 2017-02-23 07:13:51 : 180 succesvolle ttvdb.com verwijzingen 2017-02-23 07:13:51 : 75 misluktte ttvdb.com verwijzingen 2017-02-23 07:13:51 : Tijd/pagina: 11.284536614 seconds 2017-02-23 07:13:51 : 84 pagina('s) opgehaald van theTVDB.com Today I ran the 2.2 version again and these are the summary stattistics: 2017-02-24 06:46:55 : Fetch statistics for 1606 programms on 16 channels: 2017-02-24 06:46:55 : Start time: 2017-02-24 06:45 2017-02-24 06:46:55 : End time: 2017-02-24 06:46 2017-02-24 06:46:55 : Duration: 0:01:53.098212 2017-02-24 06:46:55 : 107 page(s) fetched, of which 0 failed 2017-02-24 06:46:55 : 564 cache hits 2017-02-24 06:46:55 : 10 succesful ttvdb.com lookups 2017-02-24 06:46:55 : 8 failed ttvdb.com lookups 2017-02-24 06:46:55 : Time/fetch: 1.05699263551 seconds

So looks like this version needs 20 times more elapsed for roughly the same number of pages. Will stay on 2.2 until this version is indeed faster / comparable to 2.2

hikavdh commented 7 years ago

Yes it is slower, but you are also not properly comparing. Once you use it for a week it will get faster as the database starts filling up. Also if you compare correctly it does 20 times more ttvdb lookups, it finds 7% more programmes, etc. etc. Also it is 11 times slower in these two runs (11,28 sec /page and 1,05 sec/page) and not 20!

hikavdh commented 7 years ago

Secondly you have, especially if you use TVheadend the --use-only-cache option, which is blitzingly fast, for direct access, letting the actual fetching run, run at night at leasure!