[BitFinex] ERR_RATE_LIMIT

[JCBauza]

Having some sort of error handling when there is an ERR_RATE_LIMIT coming from bitfinex would be good. Probably sleep and try again after x amount of time:

Traceback (most recent call last):
  File "lendingbot.py", line 76, in <module>
    analysis = MarketAnalysis(Config, api)
  File "/home/jcarlos/poloniexlendingbot/modules/MarketAnalysis.py", line 36, in __init__
    + str(cur_ex))
Exception: ERROR: You entered an incorrect currency: 'DSH' to analyse the market of, please check your settings. Error message: API Error 429: { "error": "ERR_RATE_LIMIT"}

[rnevet]

'Probably sleep and try again after x amount of time' - the bot sleeps after an error, then runs a new cycle.

[mahiso]

Bitfinex doc says: "Our rate limit policy can vary in a range of 10 to 90 requests per minute depending on some factors (e.g. servers load, endpoint, etc.)."

@JCBauza: What are your sleeptimeactive, sleeptimeinactive and analyseUpdateInterval values?

[rnevet]

'If an IP address exceeds 90 requests per minute to a specific REST API endpoint (ie. /ticker), the requesting IP address will be blocked for 10-60 seconds on that endpoint and the JSON response {"error": "ERR_RATE_LIMIT"} will be returned. Please note the exact logic and handling for such DDoS defenses may change over time to further improve reliability'

For Polo we have a 6 requests per second limitation implemented - but that's ~360 requests per minute... should reduce that on Bitfinex to 1.5 requests per second or 90 per min.

https://github.com/BitBotFactory/poloniexlendingbot/blob/master/modules/Poloniex.py#L52

[mahiso]

I'm not sure if it helps implementing a limitation, because of "vary in a range of 10 to 90" and "to a specific REST API", therefore it is not global to whole API over all endpoints.

How often occurs it? I myself have not seen it on my running bot. My times are sleeptimeactive = 90, sleeptimeinactive = 90 and analyseUpdateInterval = 60.

[JCBauza]

I am using the default settings: sleeptimeinactive = 300 sleeptimeactive = 60 analyseUpdateInterval = 60

[JCBauza]

It doesn't occur very often. Usually it happens when I boot up the bot. Looking quickly through the log it seems it is just at the beginning. It is almost a guarantee at the beginning and maybe later on very seldomly

I get more read timeouts later on:

2017-08-25 09:34:05 Error HTTPSConnectionPool(host='api.bitfinex.com', port=443): Read timed out. (read timeout=30) Requesting https://api.bitfinex.com/v1/pubticker/dshbtc Timed out, will retry in 60.0sec

[mfbrj]

I have thousands of this:

Error in returning data from exchange: API Error 429: { "error": "ERR_RATE_LIMIT"} Requesting https://api.bitfinex.com/v1/lendbook/BTC?limit_asks=3&limit_bids=3 Traceback (most recent call last): File "/home/fredbeltrao/poloniexlendingbot/modules/MarketAnalysis.py", line 160, in update_market_thread raw_data = self.api.return_loan_orders(cur, levels)['offers'] File "/home/fredbeltrao/poloniexlendingbot/modules/Bitfinex.py", line 150, in return_loan_orders bfx_resp = self._get(command) File "/home/fredbeltrao/poloniexlendingbot/modules/Bitfinex.py", line 120, in _get return self._request('get', request) File "/home/fredbeltrao/poloniexlendingbot/modules/ExchangeApi.py", line 29, in new_method return method(self, *arg, **kws) File "/home/fredbeltrao/poloniexlendingbot/modules/Bitfinex.py", line 109, in _request raise ex ApiError: API Error 429: { "error": "ERR_RATE_LIMIT"}

My config file

[rnevet]

@fredbeltrao is your bot up to date with the recent version? as Bitfinex has a dynamic rate limit we can't prevent those only mitigate.

[mfbrj]

@rnevet yes. Updated from scratch yesterday. These errors disappear when analysis currency is disable.

[graidl]

With the main branch, I also have the problem of a huge amount of "ERR_RATE_LIMIT" errors. Using the ERR_RATE_LIMIT_fix branch helps, but to me the initial value and the step size for the dynamic update (3) are clearly too small. I need an api_req_period of about 1200 to get no ERR_RATE_LIMIT errors, and starting from 850, over 110 errors/updates are produced until this target level is reached. And this means a lot of time if you consider that after each three or four errors, the 130s ban takes place.

To make it short, my suggestion is to make the update step size significantly larger, maybe 50, and to possibly start already with a more meaningful value. Ideally, the dynamic mechanism might also consider to decrease the time occasionally, if no errors are produced.

Anyway, thanks for the fix!

[laxdog]

@graidl can you try the head of that branch, I've already made some updates including raising the default from 850 to 1000.

I've a feeling it doesn't need to go as high as 1200, but because the requests are all getting queue and released at the same time (with a request period that's too small) then it's hitting the rate limit again straight away.

Currently the timer is reduced if it ever gets above 1500. Reasons being if it ever gets that high then it's likely something else has gone wrong.

Agreed though that there should be something to bring it down gradually if no error are encountered. However I need to change a few other things first for that to happen.

Feedback much appreciated by the way. Hoping we can find a solution that suits everyone.

[graidl]

I've tried the new version, and it looks good. There are still ERR_RATE_LIMIT errors, but just nine at the beginning, the limit is increased to no more than 1027. Looks like a solid fix! Thank you for your work!

[mike4001]

Just wanted to say that I hope the ERR_RATE_LIMIT fix is implemented soon.

The bot is basically working, but I am getting the error every few minutes and then no stats are displayed in the web-interface. I have to wait a couple of minutes to "catch" the bot when he is able to pull the necessary data.

[coulterj]

@rnevet Do you know when you might merge the changes? I am having the same issue as @mike4001

[CptanPanic]

Cloned the other day, and am getting " Error API Error 403: {"message":"This API key does not have permission for this action"} Requesting https://api.bitfinex.com/v1/offer/new" every once in a while. Is this due to rate limit also?

My settings are:

sleeptimeactive = 300
sleeptimeinactive = 300

[MarketAnalysis]
# PLEASE refer to the docs before attempting to use any of this. There are a lot of things here that will not work
# correctly unless you understand what you are doing.
analyseUpdateInterval=60
analyseCurrencies = BTC,ETH
lendingStyle = 90
MACD_long_win_seconds = 1800
#MACD_short_win_seconds = 150
# 3 days = 60 * 60 * 24 * 3 = 259200
percentile_seconds = 259200

[laxdog]

Unlikely to be anything to do with that @CptanPanic

Have you checked the permissions on the key are correct?

Is the bot still lending?

[CptanPanic]

Just went to bitfinex, and they are starting to block US customers, starting with funding. So I maybe out of luck in terms of bitfinex.

[veeprox]

I get this error if I enable the ACCOUNTSTATS and the CHARTS plugin. Without those two plugins enabled the bot works fine. I think it's just the initial startup hammering the API which leads to Bitfinex banning your IP for a time.

[mc-knife]

@veeprox: same here. i realize it since around december 6th. but in my case it's not the initial hammering. plugin was running since weeks and suddenly started to cause the errors. sleeptimeactive = 60 sleeptimeinactive = 600

[mike4001]

Same here.

They must have changed something during their DDoS attack.

I constantly get the ERR_RATE_LIMIT message for a couple of days now. Doesn't go away even when the bot is running for hours/days.

[mc-knife]

in statements bitfinex changed this:

If an IP address exceeds 90 requests per minute to the REST APIs, the requesting IP address will be blocked for 10-60 seconds and the JSON response {"error": "ERR_RATE_LIMIT"} will be returned. [1]

to that:

If an IP address exceeds a certain number of requests per minute (between 10 and 90) to a specific REST API endpoint (ie. /ticker), the requesting IP address will be blocked for 10-60 seconds on that endpoint and the JSON response {"error": "ERR_RATE_LIMIT"} will be returned. [2]

can we somehow messure the amount of request that we send? is there a switch to limit the requests to https://api.bitfinex.com/v1/history?

what about the proposed switch to WebSockets APIs?

[besoeasy]

i think websockets api will require HTTPS, so its not good yet

[johnny-cash]

Any news about this?

[Evanito]

@rnevet What do you think about being more aggressive with adjusting to the dynamic limit? Starting slow with higher increments, by messages/min instead of ms/message.

[dobcrypto]

Same errors =(

2018-09-18 01:07:09 Error Error retrieving ticker for bcheth: API Error 429: { "error": "ERR_RATE_LIMIT" } Requesting https://api.bitfinex.com/v1/pubticker/bcheth. Continue with next currency.

2018-09-18 01:07:03 Error Error retrieving ticker for bchbtc: API Error 429: { "error": "ERR_RATE_LIMIT" } Requesting https://api.bitfinex.com/v1/pubticker/bchbtc. Continue with next currency

2018-09-18 01:07:35 Error API Error 429: { "error": "ERR_RATE_LIMIT"} Requesting https://api.bitfinex.com/v1/balances