Wait more before downloading another subtitle

[heciritoglu]

Hi,

I have over 100 files, I am running the script on CLi. But I believe there is a request limit for API (40 HTTP requests per 10 seconds per IP address)[1]. So I think it could be nice to increase the wait time before start downloading another subtitle. Therefore, we don't get: xmlrpclib.ProtocolError: <ProtocolError for api.opensubtitles.org/xml-rpc: 429 Too Many Requests>

[1] https://trac.opensubtitles.org/projects/opensubtitles/wiki/XMLRPC --

[emericg]

Hi! Allright, there is probably some room for improvement here. You are using automatic selection & downloading right ?

By looking at the code, a limitation should already be in place, good enough for 40 searchs per 10 seconds. If it's not working, then it's probably because by HTTP requests they mean connection, search, download and disconnection are counted instead of just searches. So I moved the timer to take account the first search too (4a0687953ad32c9005c46ba4c4ca828e2b1aad58), but can you try what timer value is working out for you? If it's 10s / 4 requests then the timer should be at 1.0s (latencies here and there should ensure we don't hit the limit). Right now it's at 0.33s.

Of course another way too look at this without resorting to huge timer value is to move the dispatch into the main loop so we can connect/disconnect just once, and avoid a couple of unnecessary requests, like it was actually done years ago. It's only possible for CLI (or GUI full auto) requests. I'll do that later though, because it require a little (probably a lot) more testing...

[muzzol]

I'm having this problem too. I'm using a registered user and I configured correctly osd_username and osd_password but I still get

xmlrpclib.ProtocolError: <ProtocolError for api.opensubtitles.org/xml-rpc: 429 Too Many Requests>

maybe you could add a parameter to specify miliseconds to wait between searches.

[shoon]

Would also like to see this as an argument. As a hack, the -i argument can be used to provide a resume-like functionality as it will skip the GET request for any scanned file that already has a corresponding srt file:

./OpenSubtitlesDownload.py --cli -i /PATH/OR/FILE

 ... 429 Too Many Requests>

./OpenSubtitlesDownload.py --cli -i /PATH/OR/FILE

[emericg]

Ok I've upped the wait time to 1s between two searches, that should be enough. I'll let this issue open a bit to see if we still have problems here.

[abarisain]

Hi,

I still get 429s, even on a VIP account. I can get ~20 requests in before getting throttled.

I don't mind waiting so I bumped the wait to 5s, and it seems fine.

[emericg]

Hum :( what's the lowest value that seems to work for you?

[abarisain]

Two seconds seem to work fine!

[emericg]

Allright I've finally set the timer to 2s (it was 0.33s before this bug report) let's see if it's better.