search

DeviaVir / zenbot

Zenbot is a command-line cryptocurrency trading bot using Node.js and MongoDB.
MIT License
8.22k stars 2.04k forks source link

GDAX constant status 429 (api limit exceeded) with very reasonable poll times applied #862

Open formyhealth opened 6 years ago

formyhealth commented 6 years ago

sample config that encounters rate limit errors:

order time adjust= 8000 order poll time= 2000 settlement= 5000 trade poll= 2000

What times should I use to get as close to real time as possible and not run into 429's? To be clear it appears trades are still posting but I imagine I am missing some opportunities when I get these recurring api limit status messages.

MrSiNiSt3R commented 6 years ago

Rate Limits

When a rate limit is exceeded, a status of 429 Too Many Requests will be returned. REST API Public endpoints

We throttle public endpoints by IP: 3 requests per second, up to 6 requests per second in bursts. Private endpoints

We throttle private endpoints by user ID: 5 requests per second, up to 10 requests per second in bursts. Financial Information eXchange API

The FIX API throttles each command type (eg.: NewOrderSingle, OrderCancelRequest) to 50 commands per second.

StraggleCraft commented 6 years ago

I'm also unable to use paper trading because of this... whenever the bot decides to make a transaction, the throttle is hit. I'm unclear from the comments above. Does this issue require new development work to fix the amount of requests being sent, or is this an end user configuration problem? As it currently stands, gdax support is broken and unusable for anything other than sims.

formyhealth commented 6 years ago

@MrSiNiSt3R

Right.. I am aware of the api limitations, but my 2000ms to 8000ms polls don't (or shouldn't) come close to 3 requests per second.

It happens intermittently it seems, it was running fine for 4 hours and then started hitting limits. There must be other functions or calls that get triggered based on trading strategy (using neural).

unixhaj commented 6 years ago

I get banned by gdax using buy / sell commands, settings are; c.order_adjust_time = 15000 c.order_poll_time = 15000 c.wait_for_settlement = 15000

typicalaimster commented 6 years ago

So FWIW I tested a few things tonight while the market was sideways.. My settings are the following..

c.order_adjust_time = 1000 c.order_poll_time = 1000 c.wait_for_settlement = 2000

I was getting the 429's from GDAX.. Until I killed my other bot that was trading LTC. After that I could trade BTC without getting the message. Are you running two bots and are they coming from the same IP address?

formyhealth commented 6 years ago

Thanks for testing @typicalaimster In my case I am only running the one bot in ETH-USD and funny enough in sideways markets(like now) it seems ok (no 429's)

unixhaj commented 6 years ago

@typicalaimster Can't imagine that it makes less API requests if the timing is set lower? wouldn't that make it worse?

MrSiNiSt3R commented 6 years ago

GDAX: =-=-= 3 per second (6 burst) --> 3000 ms 5 pre second (10 burst) --> 5000 ms

--poll_trades=6000 --order_poll_time=3000 --order_adjust_time=3000

Bitfinex: =-=-=-= 90 per minute 1.5 per second --> 1500 per milisecond

--poll_trades=1500 --order_poll_time=1500 --order_adjust_time=1500

Poloniex: =-=-=-= 6 calls per second --> 6000 ms

--poll_trades=6000 --order_poll_time=6000 --order_adjust_time=6000

formyhealth commented 6 years ago

Ok so at 3000ms it shouldn't hit rate limits?

On Dec 16, 2017 12:57 PM, "SiNiSt3R" notifications@github.com wrote:

GDAX: =-=-= 3 per second (6 burst) --> 3000 ms 5 pre second (10 burst) --> 5000 ms

--poll_trades=6000 --order_poll_time=3000 --order_adjust_time=3000

Bitfinex: =-=-=-= 90 per minute 1.5 per second --> 1500 per milisecond

--poll_trades=1500 --order_poll_time=1500 --order_adjust_time=1500

Poloniex: =-=-=-= 6 calls per second --> 6000 ms

--poll_trades=6000 --order_poll_time=6000 --order_adjust_time=6000

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/carlos8f/zenbot/issues/862#issuecomment-352211317, or mute the thread https://github.com/notifications/unsubscribe-auth/AcJSDDdWwZlOezwxyecCJZwJ75Kof4Swks5tBC63gaJpZM4RBloW .

unixhaj commented 6 years ago

Using tcpdump I see quite a lot of requests going to gdax when executing an order using buy or sell commands... Quite a lot more than one every 6000 MS... it shows at least 40 every second (if one outgoing line in tcpdump counts as one request)

unixhaj commented 6 years ago

02:09:33.601727 IP api.gdax.com.https > anon.34327: Flags [P.], seq 169:1321, ack 1005, win 31, length 1152 02:09:33.601763 IP api.gdax.com.https > anon.34327: Flags [F.], seq 1321, ack 1005, win 31, length 0 02:09:33.601776 IP anon.34327 > api.gdax.com.https: Flags [.], ack 1322, win 130, length 0 02:09:33.602120 IP anon.34327 > api.gdax.com.https: Flags [P.], seq 1005:1036, ack 1322, win 130, length 31 02:09:33.602168 IP anon.34327 > api.gdax.com.https: Flags [F.], seq 1036, ack 1322, win 130, length 0 02:09:33.603431 IP api.gdax.com.https > anon.34327: Flags [.], ack 1036, win 31, length 0 02:09:33.603470 IP api.gdax.com.https > anon.34327: Flags [.], ack 1037, win 31, length 0 02:09:33.606510 IP anon.34331 > api.gdax.com.https: Flags [S], seq 1366738637, win 29200, options [mss 1460,nop,wscale 8], length 0 02:09:33.607756 IP api.gdax.com.https > anon.34331: Flags [S.], seq 2071628512, ack 1366738638, win 29200, options [mss 1460,nop,wscale 10], length 0 02:09:33.607780 IP anon.34331 > api.gdax.com.https: Flags [.], ack 1, win 115, length 0 02:09:33.607897 IP anon.34331 > api.gdax.com.https: Flags [P.], seq 1:490, ack 1, win 115, length 489 02:09:33.609064 IP api.gdax.com.https > anon.34331: Flags [.], ack 490, win 30, length 0 02:09:33.610195 IP api.gdax.com.https > anon.34331: Flags [P.], seq 1:169, ack 490, win 30, length 168 02:09:33.610209 IP anon.34331 > api.gdax.com.https: Flags [.], ack 169, win 119, length 0 02:09:33.610434 IP anon.34331 > api.gdax.com.https: Flags [P.], seq 490:1005, ack 169, win 119, length 515 02:09:33.652086 IP api.gdax.com.https > anon.34331: Flags [.], ack 1005, win 31, length 0 02:09:33.714677 IP api.gdax.com.https > anon.34331: Flags [P.], seq 169:1321, ack 1005, win 31, length 1152 02:09:33.714701 IP api.gdax.com.https > anon.34331: Flags [F.], seq 1321, ack 1005, win 31, length 0 02:09:33.714715 IP anon.34331 > api.gdax.com.https: Flags [.], ack 1322, win 130, length 0 02:09:33.715090 IP anon.34331 > api.gdax.com.https: Flags [P.], seq 1005:1036, ack 1322, win 130, length 31 02:09:33.715167 IP anon.34331 > api.gdax.com.https: Flags [F.], seq 1036, ack 1322, win 130, length 0 02:09:33.716259 IP api.gdax.com.https > anon.34331: Flags [.], ack 1036, win 31, length 0 02:09:33.716326 IP api.gdax.com.https > anon.34331: Flags [.], ack 1037, win 31, length 0 02:09:33.717818 IP anon.34333 > api.gdax.com.https: Flags [S], seq 2493029835, win 29200, options [mss 1460,nop,wscale 8], length 0 02:09:33.719029 IP api.gdax.com.https > anon.34333: Flags [S.], seq 1426147908, ack 2493029836, win 29200, options [mss 1460,nop,wscale 10], length 0 02:09:33.719054 IP anon.34333 > api.gdax.com.https: Flags [.], ack 1, win 115, length 0 02:09:33.719185 IP anon.34333 > api.gdax.com.https: Flags [P.], seq 1:490, ack 1, win 115, length 489 02:09:33.720428 IP api.gdax.com.https > anon.34333: Flags [.], ack 490, win 30, length 0 02:09:33.721326 IP api.gdax.com.https > anon.34333: Flags [P.], seq 1:169, ack 490, win 30, length 168 02:09:33.721338 IP anon.34333 > api.gdax.com.https: Flags [.], ack 169, win 119, length 0 02:09:33.721559 IP anon.34333 > api.gdax.com.https: Flags [P.], seq 490:1005, ack 169, win 119, length 515 02:09:33.762949 IP api.gdax.com.https > anon.34333: Flags [.], ack 1005, win 31, length 0 02:09:33.832473 IP api.gdax.com.https > anon.34333: Flags [P.], seq 169:1321, ack 1005, win 31, length 1152 02:09:33.832500 IP api.gdax.com.https > anon.34333: Flags [F.], seq 1321, ack 1005, win 31, length 0 02:09:33.832515 IP anon.34333 > api.gdax.com.https: Flags [.], ack 1322, win 130, length 0 02:09:33.832852 IP anon.34333 > api.gdax.com.https: Flags [P.], seq 1005:1036, ack 1322, win 130, length 31 02:09:33.832909 IP anon.34333 > api.gdax.com.https: Flags [F.], seq 1036, ack 1322, win 130, length 0 02:09:33.834196 IP api.gdax.com.https > anon.34333: Flags [.], ack 1036, win 31, length 0 02:09:33.834234 IP api.gdax.com.https > anon.34333: Flags [.], ack 1037, win 31, length 0 02:09:33.835852 IP anon.34334 > api.gdax.com.https: Flags [S], seq 738684404, win 29200, options [mss 1460,nop,wscale 8], length 0 02:09:33.837037 IP api.gdax.com.https > anon.34334: Flags [S.], seq 2332439829, ack 738684405, win 29200, options [mss 1460,nop,wscale 10], length 0 02:09:33.837062 IP anon.34334 > api.gdax.com.https: Flags [.], ack 1, win 115, length 0 02:09:33.838036 IP anon.34334 > api.gdax.com.https: Flags [P.], seq 1:490, ack 1, win 115, length 489 02:09:33.839184 IP api.gdax.com.https > anon.34334: Flags [.], ack 490, win 30, length 0 02:09:33.840101 IP api.gdax.com.https > anon.34334: Flags [P.], seq 1:169, ack 490, win 30, length 168 02:09:33.840113 IP anon.34334 > api.gdax.com.https: Flags [.], ack 169, win 119, length 0 02:09:33.840666 IP anon.34334 > api.gdax.com.https: Flags [P.], seq 490:1005, ack 169, win 119, length 515 02:09:33.882707 IP api.gdax.com.https > anon.34334: Flags [.], ack 1005, win 31, length 0 [849/9923] 02:09:33.954344 IP api.gdax.com.https > anon.34334: Flags [P.], seq 169:1321, ack 1005, win 31, length 1152 02:09:33.954376 IP api.gdax.com.https > anon.34334: Flags [F.], seq 1321, ack 1005, win 31, length 0 02:09:33.954391 IP anon.34334 > api.gdax.com.https: Flags [.], ack 1322, win 130, length 0 02:09:33.954754 IP anon.34334 > api.gdax.com.https: Flags [P.], seq 1005:1036, ack 1322, win 130, length 31 02:09:33.954815 IP anon.34334 > api.gdax.com.https: Flags [F.], seq 1036, ack 1322, win 130, length 0 02:09:33.955901 IP api.gdax.com.https > anon.34334: Flags [.], ack 1036, win 31, length 0 02:09:33.955952 IP api.gdax.com.https > anon.34334: Flags [.], ack 1037, win 31, length 0 02:09:33.957495 IP anon.34336 > api.gdax.com.https: Flags [S], seq 1367822836, win 29200, options [mss 1460,nop,wscale 8], length 0 02:09:33.958644 IP api.gdax.com.https > anon.34336: Flags [S.], seq 1930541578, ack 1367822837, win 29200, options [mss 1460,nop,wscale 10], length 0 02:09:33.958666 IP anon.34336 > api.gdax.com.https: Flags [.], ack 1, win 115, length 0 02:09:33.958773 IP anon.34336 > api.gdax.com.https: Flags [P.], seq 1:490, ack 1, win 115, length 489 02:09:33.959931 IP api.gdax.com.https > anon.34336: Flags [.], ack 490, win 30, length 0 02:09:33.960946 IP api.gdax.com.https > anon.34336: Flags [P.], seq 1:169, ack 490, win 30, length 168 02:09:33.960959 IP anon.34336 > api.gdax.com.https: Flags [.], ack 169, win 119, length 0 02:09:33.961170 IP anon.34336 > api.gdax.com.https: Flags [P.], seq 490:1005, ack 169, win 119, length 515 02:09:34.002980 IP api.gdax.com.https > anon.34336: Flags [.], ack 1005, win 31, length 0 02:09:34.070479 IP api.gdax.com.https > anon.34336: Flags [P.], seq 169:1321, ack 1005, win 31, length 1152 02:09:34.070507 IP api.gdax.com.https > anon.34336: Flags [F.], seq 1321, ack 1005, win 31, length 0 02:09:34.070517 IP anon.34336 > api.gdax.com.https: Flags [.], ack 1322, win 130, length 0 02:09:34.070886 IP anon.34336 > api.gdax.com.https: Flags [P.], seq 1005:1036, ack 1322, win 130, length 31 02:09:34.070945 IP anon.34336 > api.gdax.com.https: Flags [F.], seq 1036, ack 1322, win 130, length 0 02:09:34.072157 IP api.gdax.com.https > anon.34336: Flags [.], ack 1036, win 31, length 0 02:09:34.072199 IP api.gdax.com.https > anon.34336: Flags [.], ack 1037, win 31, length 0 02:09:34.073775 IP anon.34339 > api.gdax.com.https: Flags [S], seq 1440517702, win 29200, options [mss 1460,nop,wscale 8], length 0 02:09:34.074984 IP api.gdax.com.https > anon.34339: Flags [S.], seq 3288221005, ack 1440517703, win 29200, options [mss 1460,nop,wscale 10], length 0 02:09:34.075007 IP anon.34339 > api.gdax.com.https: Flags [.], ack 1, win 115, length 0 02:09:34.075221 IP anon.34339 > api.gdax.com.https: Flags [P.], seq 1:490, ack 1, win 115, length 489 02:09:34.076491 IP api.gdax.com.https > anon.34339: Flags [.], ack 490, win 30, length 0 02:09:34.077411 IP api.gdax.com.https > anon.34339: Flags [P.], seq 1:169, ack 490, win 30, length 168 02:09:34.077423 IP anon.34339 > api.gdax.com.https: Flags [.], ack 169, win 119, length 0 02:09:34.077687 IP anon.34339 > api.gdax.com.https: Flags [P.], seq 490:1005, ack 169, win 119, length 515 02:09:34.119203 IP api.gdax.com.https > anon.34339: Flags [.], ack 1005, win 31, length 0 02:09:34.197908 IP api.gdax.com.https > anon.34339: Flags [P.], seq 169:1321, ack 1005, win 31, length 1152 02:09:34.197937 IP api.gdax.com.https > anon.34339: Flags [F.], seq 1321, ack 1005, win 31, length 0 02:09:34.197956 IP anon.34339 > api.gdax.com.https: Flags [.], ack 1322, win 130, length 0 02:09:34.198324 IP anon.34339 > api.gdax.com.https: Flags [P.], seq 1005:1036, ack 1322, win 130, length 31 02:09:34.198377 IP anon.34339 > api.gdax.com.https: Flags [F.], seq 1036, ack 1322, win 130, length 0 02:09:34.199436 IP api.gdax.com.https > anon.34339: Flags [.], ack 1036, win 31, length 0 02:09:34.199520 IP api.gdax.com.https > anon.34339: Flags [.], ack 1037, win 31, length 0 02:09:34.201007 IP anon.34341 > api.gdax.com.https: Flags [S], seq 3451143788, win 29200, options [mss 1460,nop,wscale 8], length 0 02:09:34.202228 IP api.gdax.com.https > anon.34341: Flags [S.], seq 1628317278, ack 3451143789, win 29200, options [mss 1460,nop,wscale 10], length 0 02:09:34.202255 IP anon.34341 > api.gdax.com.https: Flags [.], ack 1, win 115, length 0 02:09:34.202336 IP anon.34341 > api.gdax.com.https: Flags [P.], seq 1:490, ack 1, win 115, length 489 02:09:34.203536 IP api.gdax.com.https > anon.34341: Flags [.], ack 490, win 30, length 0 02:09:34.205238 IP api.gdax.com.https > anon.34341: Flags [P.], seq 1:169, ack 490, win 30, length 168 02:09:34.205252 IP anon.34341 > api.gdax.com.https: Flags [.], ack 169, win 119, length 0 02:09:34.205476 IP anon.34341 > api.gdax.com.https: Flags [P.], seq 490:1005, ack 169, win 119, length 515 02:09:34.247682 IP api.gdax.com.https > anon.34341: Flags [.], ack 1005, win 31, length 0 02:09:34.309206 IP api.gdax.com.https > anon.34341: Flags [P.], seq 169:1321, ack 1005, win 31, length 1152 [800/9923] 02:09:34.309234 IP api.gdax.com.https > anon.34341: Flags [F.], seq 1321, ack 1005, win 31, length 0 02:09:34.309250 IP anon.34341 > api.gdax.com.https: Flags [.], ack 1322, win 130, length 0 02:09:34.309594 IP anon.34341 > api.gdax.com.https: Flags [P.], seq 1005:1036, ack 1322, win 130, length 31 02:09:34.309641 IP anon.34341 > api.gdax.com.https: Flags [F.], seq 1036, ack 1322, win 130, length 0 02:09:34.310755 IP api.gdax.com.https > anon.34341: Flags [.], ack 1036, win 31, length 0 02:09:34.310794 IP api.gdax.com.https > anon.34341: Flags [.], ack 1037, win 31, length 0 02:09:34.312339 IP anon.34342 > api.gdax.com.https: Flags [S], seq 2859919495, win 29200, options [mss 1460,nop,wscale 8], length 0 02:09:34.313868 IP api.gdax.com.https > anon.34342: Flags [S.], seq 2449711986, ack 2859919496, win 29200, options [mss 1460,nop,wscale 10], length 0 02:09:34.313897 IP anon.34342 > api.gdax.com.https: Flags [.], ack 1, win 115, length 0 02:09:34.313985 IP anon.34342 > api.gdax.com.https: Flags [P.], seq 1:490, ack 1, win 115, length 489 02:09:34.315183 IP api.gdax.com.https > anon.34342: Flags [.], ack 490, win 30, length 0 02:09:34.316152 IP api.gdax.com.https > anon.34342: Flags [P.], seq 1:169, ack 490, win 30, length 168 02:09:34.316163 IP anon.34342 > api.gdax.com.https: Flags [.], ack 169, win 119, length 0 02:09:34.316382 IP anon.34342 > api.gdax.com.https: Flags [P.], seq 490:1005, ack 169, win 119, length 515 02:09:34.358636 IP api.gdax.com.https > anon.34342: Flags [.], ack 1005, win 31, length 0 02:09:34.441089 IP api.gdax.com.https > anon.34342: Flags [P.], seq 169:1321, ack 1005, win 31, length 1152 02:09:34.441117 IP api.gdax.com.https > anon.34342: Flags [F.], seq 1321, ack 1005, win 31, length 0 02:09:34.441132 IP anon.34342 > api.gdax.com.https: Flags [.], ack 1322, win 130, length 0 02:09:34.441482 IP anon.34342 > api.gdax.com.https: Flags [P.], seq 1005:1036, ack 1322, win 130, length 31 02:09:34.441528 IP anon.34342 > api.gdax.com.https: Flags [F.], seq 1036, ack 1322, win 130, length 0 02:09:34.442772 IP api.gdax.com.https > anon.34342: Flags [.], ack 1036, win 31, length 0 02:09:34.442832 IP api.gdax.com.https > anon.34342: Flags [.], ack 1037, win 31, length 0 02:09:34.443041 IP anon.34343 > api.gdax.com.https: Flags [S], seq 2893999165, win 29200, options [mss 1460,nop,wscale 8], length 0 02:09:34.444241 IP api.gdax.com.https > anon.34343: Flags [S.], seq 3207526472, ack 2893999166, win 29200, options [mss 1460,nop,wscale 10], length 0 02:09:34.444266 IP anon.34343 > api.gdax.com.https: Flags [.], ack 1, win 115, length 0 02:09:34.444354 IP anon.34343 > api.gdax.com.https: Flags [P.], seq 1:490, ack 1, win 115, length 489 02:09:34.445588 IP api.gdax.com.https > anon.34343: Flags [.], ack 490, win 30, length 0 02:09:34.446692 IP api.gdax.com.https > anon.34343: Flags [P.], seq 1:169, ack 490, win 30, length 168 02:09:34.446704 IP anon.34343 > api.gdax.com.https: Flags [.], ack 169, win 119, length 0 02:09:34.446971 IP anon.34343 > api.gdax.com.https: Flags [P.], seq 490:1005, ack 169, win 119, length 515 02:09:34.488773 IP api.gdax.com.https > anon.34343: Flags [.], ack 1005, win 31, length 0 02:09:34.559053 IP api.gdax.com.https > anon.34343: Flags [P.], seq 169:1321, ack 1005, win 31, length 1152 02:09:34.559075 IP api.gdax.com.https > anon.34343: Flags [F.], seq 1321, ack 1005, win 31, length 0 02:09:34.559086 IP anon.34343 > api.gdax.com.https: Flags [.], ack 1322, win 130, length 0 02:09:34.559661 IP anon.34343 > api.gdax.com.https: Flags [P.], seq 1005:1036, ack 1322, win 130, length 31 02:09:34.559719 IP anon.34343 > api.gdax.com.https: Flags [F.], seq 1036, ack 1322, win 130, length 0 02:09:34.560878 IP api.gdax.com.https > anon.34343: Flags [.], ack 1036, win 31, length 0 02:09:34.560922 IP api.gdax.com.https > anon.34343: Flags [.], ack 1037, win 31, length 0 02:09:34.562860 IP anon.34346 > api.gdax.com.https: Flags [S], seq 3294841707, win 29200, options [mss 1460,nop,wscale 8], length 0 02:09:34.564044 IP api.gdax.com.https > anon.34346: Flags [S.], seq 3930178261, ack 3294841708, win 29200, options [mss 1460,nop,wscale 10], length 0 02:09:34.564068 IP anon.34346 > api.gdax.com.https: Flags [.], ack 1, win 115, length 0 02:09:34.564191 IP anon.34346 > api.gdax.com.https: Flags [P.], seq 1:490, ack 1, win 115, length 489 02:09:34.565421 IP api.gdax.com.https > anon.34346: Flags [.], ack 490, win 30, length 0 02:09:34.566753 IP api.gdax.com.https > anon.34346: Flags [P.], seq 1:169, ack 490, win 30, length 168 02:09:34.566764 IP anon.34346 > api.gdax.com.https: Flags [.], ack 169, win 119, length 0 02:09:34.567070 IP anon.34346 > api.gdax.com.https: Flags [P.], seq 490:1005, ack 169, win 119, length 515 02:09:34.609404 IP api.gdax.com.https > anon.34346: Flags [.], ack 1005, win 31, length 0 02:09:34.675866 IP api.gdax.com.https > anon.34346: Flags [P.], seq 169:1321, ack 1005, win 31, length 1152 02:09:34.675887 IP api.gdax.com.https > anon.34346: Flags [F.], seq 1321, ack 1005, win 31, length 0 [751/9923] 02:09:34.675895 IP anon.34346 > api.gdax.com.https: Flags [.], ack 1322, win 130, length 0 02:09:34.676268 IP anon.34346 > api.gdax.com.https: Flags [P.], seq 1005:1036, ack 1322, win 130, length 31 02:09:34.676325 IP anon.34346 > api.gdax.com.https: Flags [F.], seq 1036, ack 1322, win 130, length 0 02:09:34.677479 IP api.gdax.com.https > anon.34346: Flags [.], ack 1036, win 31, length 0 02:09:34.677514 IP api.gdax.com.https > anon.34346: Flags [.], ack 1037, win 31, length 0 02:09:34.680594 IP anon.34348 > api.gdax.com.https: Flags [S], seq 2006911760, win 29200, options [mss 1460,nop,wscale 8], length 0 02:09:34.681917 IP api.gdax.com.https > anon.34348: Flags [S.], seq 3198759906, ack 2006911761, win 29200, options [mss 1460,nop,wscale 10], length 0 02:09:34.681943 IP anon.34348 > api.gdax.com.https: Flags [.], ack 1, win 115, length 0 02:09:34.682032 IP anon.34348 > api.gdax.com.https: Flags [P.], seq 1:490, ack 1, win 115, length 489 02:09:34.683148 IP api.gdax.com.https > anon.34348: Flags [.], ack 490, win 30, length 0 02:09:34.684203 IP api.gdax.com.https > anon.34348: Flags [P.], seq 1:169, ack 490, win 30, length 168 02:09:34.684213 IP anon.34348 > api.gdax.com.https: Flags [.], ack 169, win 119, length 0 02:09:34.684417 IP anon.34348 > api.gdax.com.https: Flags [P.], seq 490:1005, ack 169, win 119, length 515 02:09:34.726145 IP api.gdax.com.https > anon.34348: Flags [.], ack 1005, win 31, length 0 02:09:34.797920 IP api.gdax.com.https > anon.34348: Flags [P.], seq 169:1321, ack 1005, win 31, length 1152 02:09:34.797952 IP api.gdax.com.https > anon.34348: Flags [F.], seq 1321, ack 1005, win 31, length 0 02:09:34.797967 IP anon.34348 > api.gdax.com.https: Flags [.], ack 1322, win 130, length 0 02:09:34.798366 IP anon.34348 > api.gdax.com.https: Flags [P.], seq 1005:1036, ack 1322, win 130, length 31 02:09:34.798416 IP anon.34348 > api.gdax.com.https: Flags [F.], seq 1036, ack 1322, win 130, length 0 02:09:34.799472 IP api.gdax.com.https > anon.34348: Flags [.], ack 1036, win 31, length 0 02:09:34.799506 IP api.gdax.com.https > anon.34348: Flags [.], ack 1037, win 31, length 0 02:09:34.800446 IP anon.34351 > api.gdax.com.https: Flags [S], seq 3694345709, win 29200, options [mss 1460,nop,wscale 8], length 0 02:09:34.801648 IP api.gdax.com.https > anon.34351: Flags [S.], seq 3813541753, ack 3694345710, win 29200, options [mss 1460,nop,wscale 10], length 0 02:09:34.801673 IP anon.34351 > api.gdax.com.https: Flags [.], ack 1, win 115, length 0 02:09:34.801759 IP anon.34351 > api.gdax.com.https: Flags [P.], seq 1:490, ack 1, win 115, length 489 02:09:34.802997 IP api.gdax.com.https > anon.34351: Flags [.], ack 490, win 30, length 0 02:09:34.803970 IP api.gdax.com.https > anon.34351: Flags [P.], seq 1:169, ack 490, win 30, length 168 02:09:34.803980 IP anon.34351 > api.gdax.com.https: Flags [.], ack 169, win 119, length 0 02:09:34.804269 IP anon.34351 > api.gdax.com.https: Flags [P.], seq 490:1005, ack 169, win 119, length 515 02:09:34.846196 IP api.gdax.com.https > anon.34351: Flags [.], ack 1005, win 31, length 0 02:09:34.919677 IP api.gdax.com.https > anon.34351: Flags [P.], seq 169:1321, ack 1005, win 31, length 1152 02:09:34.919702 IP api.gdax.com.https > anon.34351: Flags [F.], seq 1321, ack 1005, win 31, length 0 02:09:34.919720 IP anon.34351 > api.gdax.com.https: Flags [.], ack 1322, win 130, length 0 02:09:34.920153 IP anon.34351 > api.gdax.com.https: Flags [P.], seq 1005:1036, ack 1322, win 130, length 31 02:09:34.920214 IP anon.34351 > api.gdax.com.https: Flags [F.], seq 1036, ack 1322, win 130, length 0 02:09:34.921473 IP api.gdax.com.https > anon.34351: Flags [.], ack 1036, win 31, length 0 02:09:34.921482 IP api.gdax.com.https > anon.34351: Flags [.], ack 1037, win 31, length 0 02:09:34.924093 IP anon.34354 > api.gdax.com.https: Flags [S], seq 1192034706, win 29200, options [mss 1460,nop,wscale 8], length 0 02:09:34.925321 IP api.gdax.com.https > anon.34354: Flags [S.], seq 1466594637, ack 1192034707, win 29200, options [mss 1460,nop,wscale 10], length 0 02:09:34.925347 IP anon.34354 > api.gdax.com.https: Flags [.], ack 1, win 115, length 0 02:09:34.925463 IP anon.34354 > api.gdax.com.https: Flags [P.], seq 1:490, ack 1, win 115, length 489 02:09:34.926640 IP api.gdax.com.https > anon.34354: Flags [.], ack 490, win 30, length 0 02:09:34.928012 IP api.gdax.com.https > anon.34354: Flags [P.], seq 1:169, ack 490, win 30, length 168 02:09:34.928026 IP anon.34354 > api.gdax.com.https: Flags [.], ack 169, win 119, length 0 02:09:34.928300 IP anon.34354 > api.gdax.com.https: Flags [P.], seq 490:1005, ack 169, win 119, length 515 02:09:34.969994 IP api.gdax.com.https > anon.34354: Flags [.], ack 1005, win 31, length 0 02:09:35.036675 IP api.gdax.com.https > anon.34354: Flags [P.], seq 169:1321, ack 1005, win 31, length 1152 02:09:35.036696 IP api.gdax.com.https > anon.34354: Flags [F.], seq 1321, ack 1005, win 31, length 0 02:09:35.036706 IP anon.34354 > api.gdax.com.https: Flags [.], ack 1322, win 130, length 0 [702/9923] 02:09:35.037169 IP anon.34354 > api.gdax.com.https: Flags [P.], seq 1005:1036, ack 1322, win 130, length 31 02:09:35.037229 IP anon.34354 > api.gdax.com.https: Flags [F.], seq 1036, ack 1322, win 130, length 0 02:09:35.038451 IP api.gdax.com.https > anon.34354: Flags [.], ack 1036, win 31, length 0 02:09:35.038483 IP api.gdax.com.https > anon.34354: Flags [.], ack 1037, win 31, length 0 02:09:35.041044 IP anon.34356 > api.gdax.com.https: Flags [S], seq 3410081104, win 29200, options [mss 1460,nop,wscale 8], length 0 02:09:35.042215 IP api.gdax.com.https > anon.34356: Flags [S.], seq 70634635, ack 3410081105, win 29200, options [mss 1460,nop,wscale 10], length 0 02:09:35.042241 IP anon.34356 > api.gdax.com.https: Flags [.], ack 1, win 115, length 0 02:09:35.042353 IP anon.34356 > api.gdax.com.https: Flags [P.], seq 1:490, ack 1, win 115, length 489 02:09:35.043452 IP api.gdax.com.https > anon.34356: Flags [.], ack 490, win 30, length 0 02:09:35.044542 IP api.gdax.com.https > anon.34356: Flags [P.], seq 1:169, ack 490, win 30, length 168 02:09:35.044555 IP anon.34356 > api.gdax.com.https: Flags [.], ack 169, win 119, length 0 02:09:35.044893 IP anon.34356 > api.gdax.com.https: Flags [P.], seq 490:1005, ack 169, win 119, length 515 02:09:35.087197 IP api.gdax.com.https > anon.34356: Flags [.], ack 1005, win 31, length 0 02:09:35.161937 IP api.gdax.com.https > anon.34356: Flags [P.], seq 169:1321, ack 1005, win 31, length 1152 02:09:35.161955 IP api.gdax.com.https > anon.34356: Flags [F.], seq 1321, ack 1005, win 31, length 0 02:09:35.161964 IP anon.34356 > api.gdax.com.https: Flags [.], ack 1322, win 130, length 0 02:09:35.162501 IP anon.34356 > api.gdax.com.https: Flags [P.], seq 1005:1036, ack 1322, win 130, length 31 02:09:35.162571 IP anon.34356 > api.gdax.com.https: Flags [F.], seq 1036, ack 1322, win 130, length 0 02:09:35.163567 IP api.gdax.com.https > anon.34356: Flags [.], ack 1036, win 31, length 0 02:09:35.163605 IP api.gdax.com.https > anon.34356: Flags [.], ack 1037, win 31, length 0 02:09:35.165979 IP anon.34358 > api.gdax.com.https: Flags [S], seq 1598155541, win 29200, options [mss 1460,nop,wscale 8], length 0 02:09:35.167313 IP api.gdax.com.https > anon.34358: Flags [S.], seq 11847506, ack 1598155542, win 29200, options [mss 1460,nop,wscale 10], length 0 02:09:35.167338 IP anon.34358 > api.gdax.com.https: Flags [.], ack 1, win 115, length 0 02:09:35.167529 IP anon.34358 > api.gdax.com.https: Flags [P.], seq 1:490, ack 1, win 115, length 489 02:09:35.168836 IP api.gdax.com.https > anon.34358: Flags [.], ack 490, win 30, length 0 02:09:35.169843 IP api.gdax.com.https > anon.34358: Flags [P.], seq 1:169, ack 490, win 30, length 168 02:09:35.169857 IP anon.34358 > api.gdax.com.https: Flags [.], ack 169, win 119, length 0 02:09:35.170743 IP anon.34358 > api.gdax.com.https: Flags [P.], seq 490:1005, ack 169, win 119, length 515 02:09:35.212927 IP api.gdax.com.https > anon.34358: Flags [.], ack 1005, win 31, length 0 02:09:35.285270 IP api.gdax.com.https > anon.34358: Flags [P.], seq 169:1321, ack 1005, win 31, length 1152 02:09:35.285296 IP api.gdax.com.https > anon.34358: Flags [F.], seq 1321, ack 1005, win 31, length 0 02:09:35.285310 IP anon.34358 > api.gdax.com.https: Flags [.], ack 1322, win 130, length 0 02:09:35.285667 IP anon.34358 > api.gdax.com.https: Flags [P.], seq 1005:1036, ack 1322, win 130, length 31 02:09:35.285721 IP anon.34358 > api.gdax.com.https: Flags [F.], seq 1036, ack 1322, win 130, length 0 02:09:35.286764 IP api.gdax.com.https > anon.34358: Flags [.], ack 1036, win 31, length 0 02:09:35.286847 IP api.gdax.com.https > anon.34358: Flags [.], ack 1037, win 31, length 0 02:09:35.288659 IP anon.34361 > api.gdax.com.https: Flags [S], seq 2569070405, win 29200, options [mss 1460,nop,wscale 8], length 0 02:09:35.289956 IP api.gdax.com.https > anon.34361: Flags [S.], seq 1798302649, ack 2569070406, win 29200, options [mss 1460,nop,wscale 10], length 0 02:09:35.289978 IP anon.34361 > api.gdax.com.https: Flags [.], ack 1, win 115, length 0 02:09:35.290068 IP anon.34361 > api.gdax.com.https: Flags [P.], seq 1:490, ack 1, win 115, length 489 02:09:35.291263 IP api.gdax.com.https > anon.34361: Flags [.], ack 490, win 30, length 0 02:09:35.292214 IP api.gdax.com.https > anon.34361: Flags [P.], seq 1:169, ack 490, win 30, length 168 02:09:35.292231 IP anon.34361 > api.gdax.com.https: Flags [.], ack 169, win 119, length 0 02:09:35.292464 IP anon.34361 > api.gdax.com.https: Flags [P.], seq 490:1005, ack 169, win 119, length 515 02:09:35.334456 IP api.gdax.com.https > anon.34361: Flags [.], ack 1005, win 31, length 0 02:09:35.405367 IP api.gdax.com.https > anon.34361: Flags [P.], seq 169:1321, ack 1005, win 31, length 1152 02:09:35.405394 IP api.gdax.com.https > anon.34361: Flags [F.], seq 1321, ack 1005, win 31, length 0 02:09:35.405410 IP anon.34361 > api.gdax.com.https: Flags [.], ack 1322, win 130, length 0

Observe .. seems a bit busy for; c.order_adjust_time = 20000 c.order_poll_time = 20000 c.wait_for_settlement = 20000

formyhealth commented 6 years ago

@unixhaj it does look to busy for 20 second intervals

StraggleCraft commented 6 years ago

Issuing a manual market buy command while in paper mode immediately triggers the rate limit. @DeviaVir can this be marked as a bug?

kiancross commented 6 years ago

I'm also experiencing the same problem.

acecilia commented 6 years ago

Same here. Default configuration, one bot running.

c.order_adjust_time = 5000
c.order_poll_time = 5000
c.wait_for_settlement = 5000
c.poll_trades = 30000

After 5 hours, got several 429 from gdax.ETH-EUR

formyhealth commented 6 years ago

I found that with:

GDAX: =-=-= 3 per second (6 burst) --> 3000 ms 5 per second (10 burst) --> 5000 ms

--poll_trades=6000 --order_poll_time=3000 --order_adjust_time=3000

as per @SiNiSt3R seemed to work without rate limiting. (albeit a little slower to refresh)

On Sun, Dec 24, 2017 at 10:13 PM, Andrés Cecilia Luque < notifications@github.com> wrote:

Same here. Default configuration, one bot running.

c.order_adjust_time = 5000 c.order_poll_time = 5000 c.wait_for_settlement = 5000 c.poll_trades = 30000

After 5 hours, got several 429 from gdax.ETH-EUR

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/carlos8f/zenbot/issues/862#issuecomment-353835332, or mute the thread https://github.com/notifications/unsubscribe-auth/AcJSDH8KW2Mwsb9swR9qUifEx_6kvJwsks5tDz0igaJpZM4RBloW .

acecilia commented 6 years ago

@formyhealth I will give it a try. Thanks! :)

acecilia commented 6 years ago

Despite using:

--order_poll_time=3000 --order_adjust_time=3000

I keep getting 429 errors from GDAX in paper mode, 2m period:

captura de pantalla 2017-12-25 a las 16 28 33

The buy order from the image was completed after six 429 errors, with an execution time of 10 minutes and 0.3391% slippage (using maker mode).

formyhealth commented 6 years ago

@acecilia try also using: --poll_trades=6000

xmaslightguy commented 6 years ago

I'm also having this issue, and I'm wondering why the output of the bot in paper mode has lines in groups of 5, where time, price, everything is the same. Mongodb console says 5 connections are open, but I know nothing about mongodb so I don't know how or if I should fix that.

acecilia commented 6 years ago

@formyhealth I have it running with 3min period and:

--order_poll_time=3000 --order_adjust_time=3000 --poll_trades=6000

Just got two 429 errors after 24h running without issues :(

@xmaslightguy had that issue some days ago, solved here: https://github.com/carlos8f/zenbot/issues/983

StraggleCraft commented 6 years ago

This should probably be closed now that the websocket functionality has been merged in.

shri046 commented 6 years ago

Looks like this is still an issue, first time I attempted to run the bot today under live trading I got 429s

GDAX API is down! unable to call cancelOrder, retrying in 10s
{ Error: non-200 status: 429
    at statusErr (/Users/shri/crypto/zenbot/extensions/exchanges/gdax/exchange.js:65:17)
    at /Users/shri/crypto/zenbot/extensions/exchanges/gdax/exchange.js:212:25
    at Request._callback (/Users/shri/crypto/zenbot/node_modules/gdax/lib/clients/public.js:53:9)
    at Request.self.callback (/Users/shri/crypto/zenbot/node_modules/gdax/node_modules/request/request.js:188:22)
    at Request.emit (events.js:159:13)
    at Request.<anonymous> (/Users/shri/crypto/zenbot/node_modules/gdax/node_modules/request/request.js:1171:10)
    at Request.emit (events.js:159:13)
    at IncomingMessage.<anonymous> (/Users/shri/crypto/zenbot/node_modules/gdax/node_modules/request/request.js:1091:12)
    at Object.onceWrapper (events.js:254:19)
    at IncomingMessage.emit (events.js:164:20)
    at endReadableNT (_stream_readable.js:1062:12)
    at process._tickCallback (internal/process/next_tick.js:152:19)
  code: 'HTTP_STATUS',
  body: 
   { message: 'Rate limit exceeded. For rate limit information see https://docs.gdax.com/#rate-limits' } }
[ { order_id: 'e8976461-e378-42f7-be9c-8a605d44f7ad',
    product_id: 'ETH-USD' } ]

Haven't played with any of the poll time settings just yet.

micheledicosmo commented 6 years ago

Any news on this? I'm also hitting the same issue. 

2018-01-17 14:08:36  7201.15 BTC-GBP   -0.14%        0    -- 38        sell  0.00012 BTC  88.09 GBP  -11.04%  +11.18%

GDAX API is down! unable to call getBalance, retrying in 10s
{ Error: non-200 status: 429
    at statusErr (/home/ubuntu/zenbot/extensions/exchanges/gdax/exchange.js:65:17)
    at /home/ubuntu/zenbot/extensions/exchanges/gdax/exchange.js:161:25
    at Request._callback (/home/ubuntu/zenbot/node_modules/gdax/lib/clients/public.js:53:9)
    at Request.self.callback (/home/ubuntu/zenbot/node_modules/gdax/node_modules/request/request.js:188:22)
    at emitTwo (events.js:126:13)
    at Request.emit (events.js:214:7)
    at Request.<anonymous> (/home/ubuntu/zenbot/node_modules/gdax/node_modules/request/request.js:1171:10)
    at emitOne (events.js:116:13)
    at Request.emit (events.js:211:7)
    at IncomingMessage.<anonymous> (/home/ubuntu/zenbot/node_modules/gdax/node_modules/request/request.js:1091:12)
    at Object.onceWrapper (events.js:313:30)
    at emitNone (events.js:111:20)
    at IncomingMessage.emit (events.js:208:7)
    at endReadableNT (_stream_readable.js:1055:12)
    at _combinedTickCallback (internal/process/next_tick.js:138:11)
    at process._tickCallback (internal/process/next_tick.js:180:9)
  code: 'HTTP_STATUS',
  body:
   { message: 'Rate limit exceeded. For rate limit information see https://docs.gdax.com/#rate-limits' } }
[ { currency: 'GBP', asset: 'BTC' } ]
zakriyarahman commented 6 years ago

Hello fellows, was this issue resolved? I have been getting this same error. image

micheledicosmo commented 6 years ago

@zakriyarahman Hey Zak, I worked around it using these parameters: ./zenbot.sh trade --order_poll_time=3000 --order_adjust_time=3000 --poll_trades=6000 Make sure you are using the latest commit on the unstable branch. I'm on 1207e62bbc9cf6270ba906d180d5538c0afcbfd9

zakriyarahman commented 6 years ago

@micheledicosmo Thanks!

GerdC commented 6 years ago

My experience from a completely different project:

I have no problem when I set my rate limiter to 3 API calls every 2.1 seconds. I have some occasional problems when I set my rate limiter to 3 API calls every 1.8 seconds. I have tested these values with automated tests.

Important: all my calls to GDAX share the same rate limiter(!)

To the maintainers of the project: If you don't have a rate limiter across your API calls, the easiest and best algorithm I found is called TokenBasket. Google for it. I can't tell you which implementation is the best since Javascript is not part of my main languages.

Hope that helps, Gerd

Netzarim commented 6 years ago

I am still having rate limiting issues with GDAX:

./zenbot.sh trade gdax.ETH-BTC --order_poll_time=18000 --order_adjust_time=18000 --poll_trades=18000 --period_length=15m --min_periods=94

I was running them will lower numbers, but was trying to get past the rate limit... These are ridiculous numbers and still getting the 429 error...

Netzarim commented 5 years ago

I am still seeing this... I had stopped the bot on pro.coinbase.com (gdax) months ago... but wanted to do some trading in it today and the bot is still giving the rate limit error.