search

freqtrade / freqtrade

Free, open source crypto trading bot
https://www.freqtrade.io
GNU General Public License v3.0
28.76k stars 6.16k forks source link

CoinGecko rate limit on fiat convert #10076

Closed xaviergxf closed 6 months ago

xaviergxf commented 6 months ago

Hi everyone,

I had deployed several instances of freqtrade using docker and i hit the API rate limit of CoinGecko that does the fiat conversion. I've started investigating what could be done to bypass this limit and found the following:

  1. Not use fiat conversion. The doc says it's required, but on github i found that it could be removed. Can it?
  2. Introduce the possibility of using a CoinGecko payed API key on the config.
  3. Introduce the possibility of using a reverse proxy to cache these responses and avoid getting rate limited.

Did anyone else had this problem? How did you solve it? What do you think about the propositions above?

Thanks

stash86 commented 6 months ago

It can be removed. I believe we should have said so in latest docs. I'll check the docs.

I don't think reverse proxy will solve it. I have had the issue on vps with only 1-2 bots. I'm not sure why coingecko suddenly throwing this error lately, but using proxy won't help imo

xmatthias commented 6 months ago

Not use fiat conversion. The doc says it's required, but on github i found that it could be removed. Can it?

i'd be curious where you found this. it's not mentioned to be explicitly "non-required" - but it's not required.

The docs also mention this

Removing fiat_display_currency completely from the configuration will skip initializing coingecko, and will not show any FIAT currency conversion. This has no importance for the correct functioning of the bot.

xaviergxf commented 6 months ago

@stash86 a reverse proxy would allow to capture any requests intended to coingecko and cache them. If you have multiple bot instances they could use the cache instead of relaying the request to the real coingecko api. Lowering the amount of requests would lower the rate limit issues.

How about letting users use their real, paid api key?

xmatthias commented 6 months ago

If you look deep enough - then pycoingecko (with key) is pretty broken.

Coingecko offers demo keys now - but using demo keys (with some workarounds) yields this: If you are using Pro API key, please change your root URL from api.coingecko.com to pro-api.coingecko.com'

Using the "pro" url - we get this: If you are using Demo API key, please change your root URL from pro-api.coingecko.com to api.coingecko.com

Now that's obviously a pycoingecko fault, not a problem with coingecko itself - but i also don't think that fiat conversion justifies "inlining" the whole coingecko communication - even though it'd probably be pretty simple considering that we're only using very minor things off of it.