ferdn4ndo / candlestick-data-lake

A data lake application for candlestick crawling, storage and syncronous (API) and asyncronous (WebSockets) serving
MIT License
0 stars 0 forks source link

Handle the exchange API limits #5

Open ferdn4ndo opened 3 years ago

ferdn4ndo commented 3 years ago

Binance has a complex API limiting system that uses HTTP headers to inform the number of seconds to wait before a new request is made. This must be considered when implementing the CronJob.

FROM: https://binance-docs.github.io/apidocs/spot/en/#limits

General Info on Limits

The following intervalLetter values for headers: SECOND => S MINUTE => M HOUR => H DAY => D intervalNum describes the amount of the interval. For example, intervalNum 5 with intervalLetter M means "Every 5 minutes".

IP Limits

Every request will contain X-MBX-USED-WEIGHT-(intervalNum)(intervalLetter) in the response headers which has the current used weight for the IP for all request rate limiters defined. Each route has a weight which determines for the number of requests each endpoint counts for. Heavier endpoints and endpoints that do operations on multiple symbols will have a heavier weight. When a 429 is received, it's your obligation as an API to back off and not spam the API. Repeatedly violating rate limits and/or failing to back off after receiving 429s will result in an automated IP ban (HTTP status 418). IP bans are tracked and scale in duration for repeat offenders, from 2 minutes to 3 days. A Retry-After header is sent with a 418 or 429 responses and will give the number of seconds required to wait, in the case of a 429, to prevent a ban, or, in the case of a 418, until the ban is over. The limits on the API are based on the IPs, not the API keys.