search

XMLTV / xmltv

Utilities to obtain, generate, and post-process TV listings data in XMLTV format
GNU General Public License v2.0
269 stars 93 forks source link

Grabbing tvguide.co.uk very slow #155

Closed misar1 closed 2 years ago

misar1 commented 2 years ago

XMLTV Version? 1.0.0 (xmltv-1.0.0-win32.zip)

XMLTV Component? tv_grab_uk_tvguide

What happened? It works perfectly but is very slow. Grabbing a 1 day EPG for 50 channels takes about 70 min.

What did you expect to happen? Same thing but faster. I tested the same website and channels with WebGrabPlus (not Open Source and paid licence) and it does the same EPG grab in about one tenth of the time.

What other software are you using?

Operating System: Windows 10

Perl Version: As provided by extracting XMLTV zip

honir commented 2 years ago

Hi. Thanks for your comment.

Yes I've had discussions with the WG+ guys about this before, and they seem to think that screen-scraping as fast as you can is "good" behaviour. There are reasons why not even GobbleBot does that.

If you want to upset TVGuide and risk getting all of us banned, then go ahead with WG+.

tv_grab_uk_tvguide adopts best practices for screen grabbing.

Schedule your fetch (Windows Task Scheduler) for a time when you are doing something else and it doesn't matter whether it takes 7 mins or 70 mins - your data will be ready for you when you want it.