XMLTV / xmltv

Utilities to obtain, generate, and post-process TV listings data in XMLTV format
GNU General Public License v2.0
283 stars 93 forks source link

is it normal speed? #207

Closed ovladuk closed 1 year ago

ovladuk commented 1 year ago

im using version 1.2.1 on windows 10 and i was just wondering if it's normal for it to be slow? I'm grabbing uk listings and it's been running for about an hour and it's only at 3%. is that normal?

honir commented 1 year ago

That's to be expected if you are fetching lots of channels and many days worth of listings. We follow best practice when it comes to grabbing this data, so it may appear "slow" but that is deliberate.

You didn't fill the form in so I don't know what you are attempting to fetch, but as a general rule you should only fetch data for the channels you actually watch (rather than all possible channels).

Similarly don't try and fetch 14 days' worth of listings every day - there's no need. Listings beyond 3 days hence rarely change until the last minute. Fetching everything just wastes your time. For example, I fetch days 1-3 + day 8. This gives me an up-to-date schedule for the next 3 days and adds one new day onto the end. Fetching the days in between would not change anything.

rmeden commented 1 year ago

you didn't mention what grabber you're using.. I assume you're using tv_grab_uk_tvguide. That's a web scraper, so it has to grab slowly to avoid getting blocked from the target website.

Another (non-free) option is to use Schedules Direct (tv_grab_zz_sdjson, and tv_grab_zz_sdjson_sqlite). It should have data for the UK and run fast.

Disclaimer: I'm president of non-profit Schedules Direct. SD was created by OSS project leaders to provide solid, non-scraped data for US/Canadian guide data.. but now has international data.