bmoscon / cryptofeed

Cryptocurrency Exchange Websocket Data Feed Handler
Other
2.19k stars 679 forks source link

Binance using REST API instead of websocket #899

Closed iamkucuk closed 2 years ago

iamkucuk commented 2 years ago

Hello all. I'm trying to get cryptostore and cryptofeed work with QuestDB backend. I use cryptostore docker container to invoke cryptofeed feeds. I'm trying to test cryptofeed's limits with various cases. However, I encountered an error.

I'm using the following docker command to run a fetching operation:

docker run -e EXCHANGE=BINANCE -e CHANNELS=l2_book -e SYMBOLS='ETH-BTC,LTC-BTC,BNB-BTC,NEO-BTC,GAS-BTC,BTC-USDT,ETH-USDT,WTC-BTC,LRC-BTC,QTUM-BTC,OMG-BTC,ZRX-BTC,KNC-BTC,SNM-BTC,IOTA-BTC,LINK-BTC,MTL-BTC,EOS-BTC,SNT-BTC,ETC-BTC,DNT-BTC,ZEC-BTC,BNT-BTC,AST-BTC,DASH-BTC,OAX-BTC,BTG-BTC,REQ-BTC,VIB-BTC,TRX-BTC,POWR-BTC,ARK-BTC,XRP-BTC,ENJ-BTC,STORJ-BTC,BNB-USDT,KMD-BTC,NULS-BTC,XMR-BTC,AMB-BTC,BAT-BTC,NEO-USDT,BTS-BTC,LSK-BTC,MANA-BTC,ADX-BTC,ADA-BTC,XLM-BTC,WABI-BTC,LTC-USDT,WAVES-BTC,GTO-BTC,ICX-BTC,ELF-BTC,AION-BTC,NEBL-BTC,RLC-BTC,PIVX-BTC,IOST-BTC,STEEM-BTC,BLZ-BTC,ZIL-BTC,ONT-BTC,QTUM-USDT,XEM-BTC,WAN-BTC,QLC-BTC,SYS-BTC,ADA-USDT,LOOM-BTC,XRP-USDT,REP-BTC,BTC-TUSD,ETH-TUSD,ZEN-BTC,EOS-USDT,CVC-BTC,THETA-BTC,TUSD-USDT,IOTA-USDT,XLM-USDT,IOTX-BTC,QKC-BTC,DATA-BTC,ONT-USDT,TRX-USDT,ETC-USDT,ICX-USDT,ARDR-BTC,NULS-USDT,VET-BTC,VET-USDT,DOCK-BTC,POLY-BTC,RVN-BTC,DCR-BTC,MITH-BTC,REN-BTC,BNB-TUSD,XRP-TUSD,BNB-USDC,BTC-USDC,ETH-USDC,XRP-USDC,EOS-USDC,USDC-USDT,ADA-TUSD,TRX-TUSD,LINK-USDT,LINK-USDC,WAVES-USDT,LTC-USDC,TRX-USDC,ONG-BTC,ONG-USDT,HOT-USDT,ZIL-USDT,ZRX-USDT,FET-BTC,FET-USDT,BAT-USDT,XMR-USDT,ZEC-USDT,ZEC-USDC,IOST-USDT,CELR-BTC,CELR-USDT,ADA-USDC,DASH-USDT,OMG-USDT,THETA-USDT,ENJ-USDT,MITH-USDT' -e BACKEND=QUEST -e HOST='localhost' ghcr.io/bmoscon/cryptostore:latest

I get the following error: ERROR : BINANCE.http.0: Resp: {"code":-1003,"msg":"Too much request weight used; current limit is 1200 request weight per 1 MINUTE. Please use the websocket for live updates to avoid polling the API.

The error obviously urges me to use websocket. I thought, Cryptofeed should use websocket when it's available. However, cryptofeed apparently gets this error by trying to make a request from this link: https://api.binance.com/api/v3/depth?symbol=ENJUSDT&limit=1000

Did I do something wrong?

bmoscon commented 2 years ago

you have to get the book snapshot with the rest endpoint. you cant pull that many books at once unless you are using multiple ip addresses