search

ccxt / ccxt

A JavaScript / TypeScript / Python / C# / PHP cryptocurrency trading API with support for more than 100 bitcoin/altcoin exchanges
https://docs.ccxt.com
MIT License
31.96k stars 7.38k forks source link

watch_order_book with Binance - Connection timed out due to a ping-pong keepalive missing on time followed by NetworkError: 1006 #9456

Closed Kinzowa closed 3 years ago

Kinzowa commented 3 years ago

Hello,

I would like to share a problem I encounter when monitoring order books of markets in the spot, future and delivery API at Binance. It happens that exceptions are thrown with the messages bellow. Please find a short extract of the loops and client initialization (1 per API).

During this code execution there were 6 markets from the spot API, 2 markets from the delivery API and 4 markets from the future API. Because there are 4 iterations per error messages it could be a problem with the 4 markets of the future API.

Help would be appreciated,

Best Kinzowa

async def watch_book(client, market):

    log.info('Start market loop')
    wallet = market.default_type
    symbol = market.symbol

    while True:
        try:
            ob = await client.watch_order_book(symbol)  
            if ob:
                do_stuff()

            await client.sleep(1000)

        except Exception as e:
            traceback.print_exc()
            raise e

async def wallet_loop(loop, i, wallet):

    log.info('Start wallet loop')
    client = getattr(ccxtpro, exchange.exid)({'enableRateLimit': True, 'asyncio_loop': loop, })
    client.options['defaultType'] = wallet
    ws_loops = [watch_book(client, market) for market in markets]

    await asyncio.gather(*ws_loops)
    await client.close()

async def main(loop):
    log.info('Start main loop')
    wallet_loops = [wallet_loop(loop, wallet) for wallet in ['spot', 'future', 'delivery']]
    await asyncio.gather(*wallet_loops)

loop = asyncio.get_event_loop()
gp = asyncio.wait([main(loop)])
loop.run_until_complete(gp)

This is the error I get : 

2021-06-27 07:00:21.466954 [info     ] Start main loop                             [trading.tasks] 
2021-06-27 07:00:21.512848 [info     ] Start wallet loop                           [trading.tasks] 
2021-06-27 07:00:21.556965 [info     ] Start wallet loop                           [trading.tasks] 
2021-06-27 07:00:21.624180 [info     ] Start wallet loop                           [trading.tasks] 

Traceback (most recent call last):
  File "/home/bragar/python/capital/trading/tasks.py", line 2928, in watch_book
    ob = await client.watch_order_book(symbol)  
  File "/home/bragar/python/envdev/lib/python3.6/site-packages/ccxtpro/binance.py", line 144, in watch_order_book
    orderbook = await self.watch(url, messageHash, message, messageHash, subscription)
ccxt.base.errors.RequestTimeout: Connection to wss://dstream.binance.com/ws timed out due to a ping-pong keepalive missing on time
Traceback (most recent call last):
  File "/home/bragar/python/capital/trading/tasks.py", line 2928, in watch_book
    ob = await client.watch_order_book(symbol)  
  File "/home/bragar/python/envdev/lib/python3.6/site-packages/ccxtpro/binance.py", line 144, in watch_order_book
    orderbook = await self.watch(url, messageHash, message, messageHash, subscription)
ccxt.base.errors.RequestTimeout: Connection to wss://stream.binance.com:9443/ws timed out due to a ping-pong keepalive missing on time
Traceback (most recent call last):
  File "/home/bragar/python/capital/trading/tasks.py", line 2928, in watch_book
    ob = await client.watch_order_book(symbol)  
  File "/home/bragar/python/envdev/lib/python3.6/site-packages/ccxtpro/binance.py", line 144, in watch_order_book
    orderbook = await self.watch(url, messageHash, message, messageHash, subscription)
  File "/home/bragar/python/capital/trading/tasks.py", line 3033, in watch_book
    raise e
  File "/home/bragar/python/capital/trading/tasks.py", line 2928, in watch_book
    ob = await client.watch_order_book(symbol)  
  File "/home/bragar/python/envdev/lib/python3.6/site-packages/ccxtpro/binance.py", line 144, in watch_order_book
    orderbook = await self.watch(url, messageHash, message, messageHash, subscription)
ccxt.base.errors.RequestTimeout: Connection to wss://stream.binance.com:9443/ws timed out due to a ping-pong keepalive missing on time
Traceback (most recent call last):
  File "/home/bragar/python/capital/trading/tasks.py", line 2928, in watch_book
    ob = await client.watch_order_book(symbol)  
  File "/home/bragar/python/envdev/lib/python3.6/site-packages/ccxtpro/binance.py", line 144, in watch_order_book
    orderbook = await self.watch(url, messageHash, message, messageHash, subscription)
  File "/home/bragar/python/capital/trading/tasks.py", line 3033, in watch_book
    raise e 
  File "/home/bragar/python/capital/trading/tasks.py", line 2928, in watch_book
    ob = await client.watch_order_book(symbol)  
  File "/home/bragar/python/envdev/lib/python3.6/site-packages/ccxtpro/binance.py", line 144, in watch_order_book
    orderbook = await self.watch(url, messageHash, message, messageHash, subscription)
  File "/home/bragar/python/capital/trading/tasks.py", line 3033, in watch_book
    raise e 
  File "/home/bragar/python/capital/trading/tasks.py", line 2928, in watch_book
    ob = await client.watch_order_book(symbol)  
  File "/home/bragar/python/envdev/lib/python3.6/site-packages/ccxtpro/binance.py", line 144, in watch_order_book
    orderbook = await self.watch(url, messageHash, message, messageHash, subscription)

2021-06-27 07:00:21.687871 [info     ] Start market loop                           [trading.tasks] 
2021-06-27 07:00:21.688395 [info     ] Start market loop                           [trading.tasks] 
2021-06-27 07:00:21.688728 [info     ] Start market loop                           [trading.tasks] 
2021-06-27 07:00:21.689067 [info     ] Start market loop                           [trading.tasks] 
2021-06-27 07:00:21.689379 [info     ] Start market loop                           [trading.tasks] 
2021-06-27 07:00:21.689697 [info     ] Start market loop                           [trading.tasks] 
2021-06-27 07:00:21.690032 [info     ] Start market loop                           [trading.tasks] 
2021-06-27 07:00:21.690359 [info     ] Start market loop                           [trading.tasks] 
2021-06-27 07:00:21.690663 [info     ] Start market loop                           [trading.tasks] 
2021-06-27 07:00:21.690995 [info     ] Start market loop                           [trading.tasks] 
2021-06-27 07:00:21.691321 [info     ] Start market loop                           [trading.tasks] 
2021-06-27 07:00:21.691645 [info     ] Start market loop                           [trading.tasks]

Traceback (most recent call last):
  File "/home/bragar/python/capital/trading/tasks.py", line 2928, in watch_book
    ob = await client.watch_order_book(symbol)  
  File "/home/bragar/python/envdev/lib/python3.6/site-packages/ccxtpro/binance.py", line 144, in watch_order_book
    orderbook = await self.watch(url, messageHash, message, messageHash, subscription)
ccxt.base.errors.NetworkError: 1006
Traceback (most recent call last):
  File "/home/bragar/python/capital/trading/tasks.py", line 2928, in watch_book
    ob = await client.watch_order_book(symbol)  
  File "/home/bragar/python/envdev/lib/python3.6/site-packages/ccxtpro/binance.py", line 144, in watch_order_book
    orderbook = await self.watch(url, messageHash, message, messageHash, subscription)
  File "/home/bragar/python/capital/trading/tasks.py", line 3033, in watch_book
    raise e 
  File "/home/bragar/python/capital/trading/tasks.py", line 2928, in watch_book
    ob = await client.watch_order_book(symbol)  
  File "/home/bragar/python/envdev/lib/python3.6/site-packages/ccxtpro/binance.py", line 144, in watch_order_book
    orderbook = await self.watch(url, messageHash, message, messageHash, subscription)
ccxt.base.errors.NetworkError: 1006
Traceback (most recent call last):
  File "/home/bragar/python/capital/trading/tasks.py", line 2928, in watch_book
    ob = await client.watch_order_book(symbol) 
  File "/home/bragar/python/envdev/lib/python3.6/site-packages/ccxtpro/binance.py", line 144, in watch_order_book
    orderbook = await self.watch(url, messageHash, message, messageHash, subscription)
  File "/home/bragar/python/capital/trading/tasks.py", line 3033, in watch_book
    raise e 
  File "/home/bragar/python/capital/trading/tasks.py", line 2928, in watch_book
    ob = await client.watch_order_book(symbol)  
  File "/home/bragar/python/envdev/lib/python3.6/site-packages/ccxtpro/binance.py", line 144, in watch_order_book
    orderbook = await self.watch(url, messageHash, message, messageHash, subscription)
  File "/home/bragar/python/capital/trading/tasks.py", line 3033, in watch_book
    raise e 
  File "/home/bragar/python/capital/trading/tasks.py", line 2928, in watch_book
    ob = await client.watch_order_book(symbol)  
  File "/home/bragar/python/envdev/lib/python3.6/site-packages/ccxtpro/binance.py", line 144, in watch_order_book
    orderbook = await self.watch(url, messageHash, message, messageHash, subscription)
ccxt.base.errors.NetworkError: 1006
Traceback (most recent call last):
  File "/home/bragar/python/capital/trading/tasks.py", line 2928, in watch_book
    ob = await client.watch_order_book(symbol)  
  File "/home/bragar/python/envdev/lib/python3.6/site-packages/ccxtpro/binance.py", line 144, in watch_order_book
    orderbook = await self.watch(url, messageHash, message, messageHash, subscription)
  File "/home/bragar/python/capital/trading/tasks.py", line 3033, in watch_book
    raise e 
  File "/home/bragar/python/capital/trading/tasks.py", line 2928, in watch_book
    ob = await client.watch_order_book(symbol)  
  File "/home/bragar/python/envdev/lib/python3.6/site-packages/ccxtpro/binance.py", line 144, in watch_order_book
    orderbook = await self.watch(url, messageHash, message, messageHash, subscription)
  File "/home/bragar/python/capital/trading/tasks.py", line 3033, in watch_book
    raise e  
  File "/home/bragar/python/capital/trading/tasks.py", line 2928, in watch_book
    ob = await client.watch_order_book(symbol) 
  File "/home/bragar/python/envdev/lib/python3.6/site-packages/ccxtpro/binance.py", line 144, in watch_order_book
    orderbook = await self.watch(url, messageHash, message, messageHash, subscription)
  File "/home/bragar/python/capital/trading/tasks.py", line 3033, in watch_book
    raise e 
  File "/home/bragar/python/capital/trading/tasks.py", line 2928, in watch_book
    ob = await client.watch_order_book(symbol)  
  File "/home/bragar/python/envdev/lib/python3.6/site-packages/ccxtpro/binance.py", line 144, in watch_order_book
    orderbook = await self.watch(url, messageHash, message, messageHash, subscription)
ccxt.base.errors.NetworkError: 1006
Kinzowa commented 3 years ago

Just to mention that my code is executed periodically but loops are closed 1 or 2 minutes after execution. In a previous execution I noticed that the NetworkError: 1006 message was absent but the ping-pong keepalive missing was trigged 2 times (maybe the 2 markets from the delivery API).

Please let me know if you need further informations

frosty00 commented 3 years ago

@Kinzowa can you remove the raise e and replace it with print(str(e)), this should stop your code from throwing, and ccxt will automatically reconnect on disconnects

Kinzowa commented 3 years ago

Thank you frosty I will do this.

Kinzowa commented 3 years ago

@Kinzowa can you remove the raise e and replace it with print(str(e)), this should stop your code from throwing, and ccxt will automatically reconnect on disconnects

Unfortunnately it seems that ccxt doesn't reconnect on disconnect because my program stop receiving the streams. Do you think it could be due to do_stuff() in my code that is blocking the loop ?

   while True:
        try:
            ob = await client.watch_order_book(symbol)  
            if ob:
                do_stuff()

            await client.sleep(1000)

        except Exception as e:
            traceback.print_exc()
            raise e
Kinzowa commented 3 years ago

I close this issue because it get better now

Matthias-Thul commented 3 years ago

@Kinzowa I am curious what you changed that improved this? I am occasionally running into this too.

Kinzowa commented 3 years ago

@Kinzowa I am curious what you changed that improved this? I am occasionally running into this too.

Not sure what is the reason errors are mostly gone, altough are still some. As suggested by Frosty I silent them and ccxt reconnects. Also I changed the loop design of my code to make it less blocking (just one over tens loop is blocking). But to be honnest I have no idea if it has something to do with the error. Good luck