bpolaszek / guzzle-throttle-middleware

A Guzzle 6+ Middleware that can delay requests before sending them.
MIT License
15 stars 9 forks source link

Does this work with async requests? #2

Open Dylan-DutchAndBold opened 5 years ago

Dylan-DutchAndBold commented 5 years ago

I read in the docs This means throttling will also block Guzzle's asynchronous requests when using CurlMultiHandler.. This is actually wanted behavior for me. But it doesn't seem to work. I have put an breakpoint on the sleep method within the Middleware. But it always breaks on the 'Developer over Qps' exception first which the api throws when more than 4 request per second are made.

Code snippet

$handlerStack = HandlerStack::create(new CurlMultiHandler());

        $throttle = new ThrottleMiddleware(new ArrayAdapter());
        $throttle->registerConfiguration(new ThrottleConfiguration(new HotelbedsRequestMatcher(), 1, 1, 'throttle'));

        $handlerStack->push($throttle, 'throttle');
bpolaszek commented 5 years ago

Hi @Dylan-DutchAndBold,

I have indeed a little doubt that it really works with CurlMultiHandler. It seems that the blocking effectively occurs but all requests are sent at once with that handler. I have found cases where it works, and cases where it doesn't. I still don't have the clear explanation.

I suggest using the normal CurlHandler until a solution is found 😕

Dylan-DutchAndBold commented 5 years ago

@bpolaszek Well, the RequestMatcher might have been at fault here. Will keep you posted.

bpolaszek commented 5 years ago

Thank you.

Dylan-DutchAndBold commented 5 years ago

Nope, wasn't the request matcher. The api keeps throwing the limit exception. My api client, which extends the guzzle client is a Singleton. It's actually trying to download static data in all languages. Where we always synchronously download the first page so we can guess the subsequent pages. We then create 7 promises for the others pages (Their are 8000 items, 1000 per page), and do this for each language. The we gather all those promises and wait for all of them to complete, this is actually also limited to 4 promises at a time. But since the requests complete too fast this middleware is necessary to throttle with time intervals. Will keep looking into it. But I'm afraid it just doesn't work.