search

umee-network / umee

A Golang implementation of the Umee network, a decentralized universal capital facility in the Cosmos ecosystem.
Apache License 2.0
224 stars 170 forks source link

price-feeder: risks of rate limiting when using public APIs #509

Closed alexanderbez closed 2 years ago

alexanderbez commented 2 years ago

From ToB Audit

Description

Price providers used by the price-feeder tool may limit the amount of served requests. After reaching the limit, certain actions should be taken by the tool to avoid prolonged or even permanent ban. Moreover, API keys or not-HTTP access channels should be used to decrease the chance of being rate-limited. Every API has its own rules which should be reviewed and respected. For example: ● Binance - has hard, machine-learning, and web application firewall limits. Users are required to respect 429 HTTP response code. ● Kraken - does rate limiting based on “call counters”. Recommends using websockets API instead of REST API. ● Huopi - restricts requests to 10 per second. Recommends using an API key.

Exploit Scenario

A price-feeder exceeds limits for the Binance API. It gets rate-limited, 429 HTTP code is returned by the API. The tool doesn’t notice this event and continues to spam the API. It gets a permanent ban. The validator using the price-feeder starts reporting imprecise exchange rates and gets slashed.

Recommendations

Short term, review requirements and recommendations provided by all supported APIs. Implement them in a user-friendly manner – for example, allow users to set and rotate API keys, use heuristic for delaying HTTP requests so that rate limits are not reached but the price-feeder still reports precise prices, and log informative error messages upon reaching rate limits.

Main takeaway here is that we should opt for websockets where possible and supported. If WS are not supported, attempt to use an API key (if provided).

RafilxTenfen commented 2 years ago

All the providers are using WS connection (Okx, Huobi, Binance, and Kraken), only the osmosis provider that just has a simple API and doesn't implement WS

We probably could close this one ;D @adamewozniak @alexanderbez

adamewozniak commented 2 years ago

All the providers are using WS connection (Okx, Huobi, Binance, and Kraken), only the osmosis provider that just has a simple API and doesn't implement WS

We probably could close this one ;D @adamewozniak @alexanderbez

Super agreed! Do you know if the osmosis api has rate limiting? If not we can definitely close this @RafilxTenfen

alexanderbez commented 2 years ago

I doubt they are. I think their API is run by Imperator.co, which is also an Umee validator. So we can ask them. But regardless, yes, let's close this issue.