search

jaggedsoft / node-binance-api

Node Binance API is an asynchronous node.js library for the Binance API designed to be easy to use.
MIT License
1.58k stars 767 forks source link

websockets.depthCache silently drops connection #214

Closed dakovi closed 5 years ago

dakovi commented 6 years ago

Title

Short Description: First of all I am very sorry for the short description but I am in a rush because of this error and the resulting problems with my trading results. Let's get to the point:

I am using a combined websockets.depthCache stream to update my price calculations with the always latest version of this repository. Since two days (as of my knowledge, maybe longer since I wasn't trading much before the recent BTC dip) I am encountering a silent drop of the stream connection (verbose is always true).

Additionally I saw a maybe connected error which I mentioned in this closed issue in a reply: https://github.com/jaggedsoft/node-binance-api/issues/207

Everything worked perfectly for weeks/months and now this gamechanger occurs and I cannot figure out why. Do you have any idea if there are existing problems with Binance itself or was something changed to the socket management part of the script?

Right now I am fighting this error by using an interval of restarting my script but this is of course hardly a long term solution...

Any help is appreciated!

EDIT: After seeing a similar problem in the issues with the userdata I am not sure if it drops the connection or just doesn't receive any updates...is this all connected?

Platform:

node version:

dakovi commented 6 years ago

Short update (sry for the mess but it's really some intense stuff if you're dealing with real money here ;)): After a few tests I saw the following pattern in my bot:

  1. combined stream is running perfectly
  2. bot buys a new coin, restarts the combined stream
  3. stream delivers one data update for every active coin stream
  4. then it stays silent and doesn't provide any further update
  5. a new restart can (but does not have to) fix the broken stream

This isn't happening all the time when the bot is restarting the stream but it is surely annoying enough...

jaggedsoft commented 6 years ago

Thank you for the report. I am not aware of any issues to depthCache at this time. But I will keep an eye on it Obviously you are having an issue so I will take a look at everything but I have not been having any problems myself

panda2048 commented 6 years ago

Hi, sorry to hijack this but I am facing similar issue with depthCache I got “too many request” error and ip is being ban for short period every time

After some study, I found that after I started the websocket for depthCache, the connection count is keep increasing (checked via netstat) I subscribe 125 pairs and it jump up to like 2000 connections shortly This is not happening to other websocket call like candles

Is it binance problem or the api logic problem? Thanks

panda2048 commented 6 years ago

Some update to my reported issue

I tried “depth” instead of “depthCache” And there is no issue with increasing number of connections Thus this can conclude that it is the bug on depthCache implementation

bmino commented 6 years ago

To answer your concern, Panda, each time you open a websocket connection to track depthCache a REST call is made with a default depth of 500 to initialize the cache (binance rate weight of 5). You are currently allowed a weighted amount of 1200 requests per minute so 125 pairs will create 600 (125 x 5) weighted requests. If you do this twice in a minute, maybe because you are restarting your program, you will hit the weighted api limit and receive the ban.

bmino commented 6 years ago

I experience a similar problem to you dakovi, when opening multiple depthCache websockets at once. Sometimes a few depth caches will not be populated with the 500 initial depths. All depths caches will receive and update with subsequent updates from the websocket connection.

I also experience this problem occasionally when toggling the default depth population to something smaller like 100.

bmino commented 6 years ago

I am now opening depth cache connections individually by ticker with a small delay between each. This has fixed the issue for me and I would propose it as a workaround.

jaggedsoft commented 6 years ago

Thanks again for your excellent work @bmino This will likely replace depthCache in a coming release

jaggedsoft commented 6 years ago

I suppose there would still be an issue if it tries to reconnect to all of them at once unless throttling REST requests (while downloading all the previous order book information) is already supported, I'll have to check