kgroeneveld / tv_grab_sd_json

XMLTV grabber for the Schedules Direct JSON API
8 stars 6 forks source link

Timeout issue #6

Open half4u opened 8 years ago

half4u commented 8 years ago

Hi,

i was trying to setup your grabber. Looked very promising and thanks for your work. My issue right now is that everytime i try to dry run it, it timeouts at different POST sets.

I tryed to limit the channels to see if that is the problem. So trying to grab 10 channels

\ POST https://json.schedulesdirect.org/20141201/schedules ==> Timeout after 180.

I would like to narrow it down further. But i dont know where to start looking at.


Limit to 1 Channel:

/tv_grab_sd_json --days 2 --debug --output test.xml Initializing... * POST https://json.schedulesdirect.org/20141201/token ==> 200 OK * GET https://json.schedulesdirect.org/20141201/status ==> 200 OK Updating lineups... lineup USA-KS57402-DEFAULT: current lineup USA-KS57402-X: current Updating schedules... * POST https://json.schedulesdirect.org/20141201/schedules/md5 ==> 200 OK (1s) channel 15425 2016-03-26: new channel 15425 2016-04-02: new channel 15425 2016-03-15: new channel 15425 2016-04-03: new channel 15425 2016-03-18: new channel 15425 2016-03-19: new channel 15425 2016-03-25: new channel 15425 2016-03-22: new channel 15425 2016-04-04: new channel 15425 2016-03-30: new channel 15425 2016-03-27: new channel 15425 2016-03-21: new channel 15425 2016-03-17: new channel 15425 2016-04-01: new channel 15425 2016-03-20: new channel 15425 2016-03-31: new channel 15425 2016-03-28: new channel 15425 2016-03-23: new channel 15425 2016-03-24: new channel 15425 2016-03-29: new channel 15425 2016-03-16: new channel 15425 2016-04-05: new * POST https://json.schedulesdirect.org/20141201/schedules ==> 200 OK Updating programs... program EP003248143445: new program EP017612940193: new program EP017612940191: new program EP019641150074: new program EP019641150079: new program EP005200942532: new program EP005200942502: new program SH007669620000: new \ POST https://json.schedulesdirect.org/20141201/programs ==> Timeout after 180.


conf:

cache=/root/.xmltv/tv_grab_sd_json.cache channel-id-format=zap2it username=xxxx password=xxxx mode=channels channels=USA-KS57402-DEFAULT channels!USA-KS57402-X channel=15425 channel!10393 channel!12424 channel!10764

kgroeneveld commented 8 years ago

I just tried your config file and it worked okay for me.

Do you have a slow internet connection?

When the grabber downloads data it tries to get up to 5000 items in one request (the max allowed by the server). Maybe if you tried downloading less it wouldn't timeout? You can try this by looking for and changing the following line in the grabber:

my $sd_json_request_max = 5000;

Maybe I should make that a configurable item. Maybe I should also look into changing the timeout for large requests. I don't recall setting a timeout so "Timeout after 180" is probably the default.

Maybe the server was just responding slowly when you tried it and it would just work now.