search

resilient-http / resilient.js

Fault tolerant and reactive HTTP client for node.js and browsers
183 stars 13 forks source link

How to handle rate limiting? #128

Closed cmeans closed 9 years ago

cmeans commented 9 years ago

Sorry, I couldn't find a place to get support/help.

I'm calling an API via Resilient that has rate limiting. Is there a standard way to have the Resilient client resend a request that was previously subject to rate limiting?

In this case, the response includes a header that indicates the rate limiting was hit, so I'd just like to be able to "automatically" resubmit the request should I get back that specific header/value.

h2non commented 9 years ago

Resilient handles 429 status codes as failed request. The API service you're using is responding with that status?

Ideally, you should plug in your own strategies to evaluate if a request was failed or not, but unfortunately this is not supported yet.

cmeans commented 9 years ago

No, unfortunately, the API I'm using returns 200. It does however, include a specific header/value combination that indicates that the call was subjected to rate limiting.

h2non commented 9 years ago

I think I can do something to cover that need... I'll let you know.

cmeans commented 9 years ago

Thanks. No rush, I've implemented a work-around that appears to be working for the moment.

On Fri, Sep 4, 2015 at 2:14 PM Tomás notifications@github.com wrote:

I think I can do something to cover that need... I'll let you know.

— Reply to this email directly or view it on GitHub https://github.com/resilient-http/resilient.js/issues/128#issuecomment-137828563 .

h2non commented 9 years ago

You got it! I've just supported this feature, and I believe that the versatility of the feature should cover all your current and potentially future needs.

Take a look to the docs, the test I've wrote and the example.

To be honest, I had in the roadmap features like for a while, so I'm glad to finally support it to help someone. With this feature, you can for instance plug in a set of strategies for different cases, like using this module.

Worth to say that I've just released a fresh version, so feel free to update via npm or bower.

cmeans commented 9 years ago

Fantastic. Thanks!

On Fri, Sep 4, 2015 at 2:51 PM Tomás notifications@github.com wrote:

You got it! I've just supported it, and I believe that the versatility of the feature should cover all your current and potentially future needs.

Take a look to the docs https://github.com/resilient-http/resilient.js#resilientaddfailstrategystrategy, the test https://github.com/resilient-http/resilient.js/blob/master/test/fail-strategies.js I've wrote and the example https://github.com/resilient-http/resilient.js/blob/master/examples/custom-fail-strategies.js .

To he honest, I had in the roadmap features like for a while, so I'm glad to finally support it to help someone. With this feature, you can for instance plug in a set of strategies for different cases, like using this module https://github.com/h2non/is-fail.

Worth to say that I've just released a fresh version, so feel free to update via npm or bower.

— Reply to this email directly or view it on GitHub https://github.com/resilient-http/resilient.js/issues/128#issuecomment-137837714 .

cmeans commented 9 years ago

Possibly, I've not implemented the pattern correctly, but the callback for the request that failed is still being called, and without the "status" property being present.

On Fri, Sep 4, 2015 at 2:52 PM Chris Means chris.a.means@gmail.com wrote:

Fantastic. Thanks!

On Fri, Sep 4, 2015 at 2:51 PM Tomás notifications@github.com wrote:

You got it! I've just supported it, and I believe that the versatility of the feature should cover all your current and potentially future needs.

Take a look to the docs https://github.com/resilient-http/resilient.js#resilientaddfailstrategystrategy, the test https://github.com/resilient-http/resilient.js/blob/master/test/fail-strategies.js I've wrote and the example https://github.com/resilient-http/resilient.js/blob/master/examples/custom-fail-strategies.js .

To he honest, I had in the roadmap features like for a while, so I'm glad to finally support it to help someone. With this feature, you can for instance plug in a set of strategies for different cases, like using this module https://github.com/h2non/is-fail.

Worth to say that I've just released a fresh version, so feel free to update via npm or bower.

— Reply to this email directly or view it on GitHub https://github.com/resilient-http/resilient.js/issues/128#issuecomment-137837714 .

cmeans commented 9 years ago

The status property of the response object.

On Fri, Sep 4, 2015 at 3:13 PM Chris Means chris.a.means@gmail.com wrote:

Possibly, I've not implemented the pattern correctly, but the callback for the request that failed is still being called, and without the "status" property being present.

On Fri, Sep 4, 2015 at 2:52 PM Chris Means chris.a.means@gmail.com wrote:

Fantastic. Thanks!

On Fri, Sep 4, 2015 at 2:51 PM Tomás notifications@github.com wrote:

You got it! I've just supported it, and I believe that the versatility of the feature should cover all your current and potentially future needs.

Take a look to the docs https://github.com/resilient-http/resilient.js#resilientaddfailstrategystrategy, the test https://github.com/resilient-http/resilient.js/blob/master/test/fail-strategies.js I've wrote and the example https://github.com/resilient-http/resilient.js/blob/master/examples/custom-fail-strategies.js .

To he honest, I had in the roadmap features like for a while, so I'm glad to finally support it to help someone. With this feature, you can for instance plug in a set of strategies for different cases, like using this module https://github.com/h2non/is-fail.

Worth to say that I've just released a fresh version, so feel free to update via npm or bower.

— Reply to this email directly or view it on GitHub https://github.com/resilient-http/resilient.js/issues/128#issuecomment-137837714 .

h2non commented 9 years ago

Only two params are passed to the strategy function: Error and http.IncomingMessage. Note that Resilient uses the raw node.js http module API, so you have to check the statusCode property in this case.

Just to clarify, if Error is present, the second argument is undefined. If http.IncomingMessage is present, first argument is null.

cmeans commented 9 years ago

I'm referring to the callback of the client.get call. But I believe I understand now. Thanks.

On Fri, Sep 4, 2015 at 3:20 PM Tomás notifications@github.com wrote:

Only two params are passed to the strategy function: Error and http.IncomingMessage https://nodejs.org/api/http.html#http_http_incomingmessage. Note that Resilient uses the raw node.js http module API, so you have to check the statusCode property.

Just to clarify, if Error is present, the second argument is undefined. If http.IncomingMessage is present, error is null.

— Reply to this email directly or view it on GitHub https://github.com/resilient-http/resilient.js/issues/128#issuecomment-137843894 .

h2non commented 9 years ago

That's the same, the interface is exactly equal.

cmeans commented 9 years ago

Should the callback from the client.get call still be called if there's a Rate Limit "failure"?

I had assumed that Resilient would automatically retry (options.retry times) before actually failing.

On Fri, Sep 4, 2015 at 3:23 PM Tomás notifications@github.com wrote:

That's the same, the interface is exactly equal.

— Reply to this email directly or view it on GitHub https://github.com/resilient-http/resilient.js/issues/128#issuecomment-137844293 .

h2non commented 9 years ago

It depends on your specific configuration, but generally yes: when the max amount of retries exceeds and the error status remains across all the attempts, the request callback will be called with an error status.

cmeans commented 9 years ago

My mistake, I was not configuring Resilient correctly (I'd missed the extra level of braces necessary for the config object). All good.

On Fri, Sep 4, 2015 at 3:28 PM Chris Means chris.a.means@gmail.com wrote:

Should the callback from the client.get call still be called if there's a Rate Limit "failure"?

I had assumed that Resilient would automatically retry (options.retry times) before actually failing.

On Fri, Sep 4, 2015 at 3:23 PM Tomás notifications@github.com wrote:

That's the same, the interface is exactly equal.

— Reply to this email directly or view it on GitHub https://github.com/resilient-http/resilient.js/issues/128#issuecomment-137844293 .

h2non commented 9 years ago

Excellent, no worries!