Need a callback to recover from WebSocketConnectionClosedException

[6ameDev]

In binance/websocket/binance_socket_manager.py, when WebSocketException is caught and identified as WebSocketConnectionClosedException, it logs that websocket connection is lost.

However in that case, I haven't been able to identify a flow where a on_close or on_error callback is triggered, so that it can be handled in the application code, in order to re-establish the websocket connection.

Any help regarding this would be highly appreciated. Thanks

[2pd]

What exactly error log did you receive?

[6ameDev]

@2pd Thanks for the response, I'm sharing the error log at the end of this message.

For some additional context, I have wrapped client.subscribe(stream=streams) with try block attempting to handle both WebSocketConnectionClosedException and general Exception, but its never caught there, reading the Binance client code, I believe its because the BinanceSockerManager spawns a different thread, and eventually when WebSocketConnectionClosedException is raised it happens in that separate thread.

Let me know if there's anything else I can provide in order to help.

Error Log:

ERROR:binance.websocket.websocket_client:Lost websocket connection
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
    self.run()
  File "/usr/local/lib/python3.10/dist-packages/binance/websocket/binance_socket_manager.py", line 47, in run
    self.read_data()
  File "/usr/local/lib/python3.10/dist-packages/binance/websocket/binance_socket_manager.py", line 66, in read_data
    raise e
  File "/usr/local/lib/python3.10/dist-packages/binance/websocket/binance_socket_manager.py", line 60, in read_data
    op_code, frame = self.ws.recv_data_frame(True)
  File "/usr/local/lib/python3.10/dist-packages/websocket/_core.py", line 406, in recv_data_frame
    frame = self.recv_frame()
  File "/usr/local/lib/python3.10/dist-packages/websocket/_core.py", line 445, in recv_frame
    return self.frame_buffer.recv_frame()
  File "/usr/local/lib/python3.10/dist-packages/websocket/_abnf.py", line 338, in recv_frame
    self.recv_header()
  File "/usr/local/lib/python3.10/dist-packages/websocket/_abnf.py", line 294, in recv_header
    header = self.recv_strict(2)
  File "/usr/local/lib/python3.10/dist-packages/websocket/_abnf.py", line 373, in recv_strict
    bytes_ = self.recv(min(16384, shortage))
  File "/usr/local/lib/python3.10/dist-packages/websocket/_core.py", line 529, in _recv
    return recv(self.sock, bufsize)
  File "/usr/local/lib/python3.10/dist-packages/websocket/_socket.py", line 122, in recv
    raise WebSocketConnectionClosedException(
websocket._exceptions.WebSocketConnectionClosedException: Connection to remote host was lost.

[SoundProvider]

I'm having the same issue

[jerrylee2013]

yeah, I got the same issue. Need a way to catch that remote host was lost exception.

[jerrylee2013]

Could we update like this? See my comment in the code below:

while True:
            try:
                op_code, frame = self.ws.recv_data_frame(True)
            except WebSocketException as e:
                if isinstance(e, WebSocketConnectionClosedException):
                    # I think the on_close callback should be called here not just raise the exception
                    self.logger.error("Lost websocket connection")
                else:
                    self.logger.error("Websocket exception: {}".format(e))
                raise e
            except Exception as e:
                self.logger.error("Exception in read_data: {}".format(e))
                raise e

[erolasan]

Any update on this?

[imanf94]

@6ameDev Yeah, testing the on_error argument I realized only errors from 1st level of API got triggered by this. Like when you have an error in the on_message function. So I think you are right about the thread issue!

As a temporary solution, I think you can define an infinite loop after you create your WS connection and check the ws.connected, and if the connection is lost, create it again. Something like:

self.ws_client = None

   def self.start_ws():

      self.ws_client = UMFuturesWebsocketClient(on_close=self.close_handler, on_message=self.message_handler, 
                                                on_error=self.error_handler, is_combined=True)
      self.ws_client.subscribe(
         stream=streams,
      )

self.start_ws()

 while True:
   time.sleep(10)
      try:
         print('WS Status:', self.ws_client.socket_manager.ws.connected)
         if(self.ws_client.socket_manager.ws.connected==False):
            self.start_ws()
      except:
         print('WS status check failed, recreating the WS connection.')
         self.start_ws()

You should also change the connected property of WS to False when there is an internal exception there. So in BinanceSocketManager add self.ws.connected = False when any exception is raised.

Let me know if this helps so I can create a PR for it.

[lovedancer075]

I met the same issue for several month, my solution is .... i write a script to check the log file, if this lost word appears, then kill the process and reboot my program: loading histroy data and create the connection, this is a ugly method, I hope anyone give a graceful method

[garbusbeach]

Guys, check this: https://github.com/binance/binance-futures-connector-python/pull/158. I've done monkey patch in my own project, but I'd suggest using version from this PR.

[MFRealG]

Hey guys, Thanks a lot to @6ameDev, @jerrylee2013, @imanf94 and @daboooooo #158

What I did:

           except WebSocketException as e:
                if isinstance(e, WebSocketConnectionClosedException):
                    self.logger.error("Lost websocket connection")
                else:
                    self.logger.error("Websocket exception: {}".format(e))
                if self.ws.connected:
                    self.ws.send_close()
                self.ws.connected = False
                self._callback(self.on_error, e)
                break
            except Exception as e:
                self.logger.error("Exception in read_data: {}".format(e))
                if self.ws.connected:
                   self.ws.send_close()
                self.ws.connected = False
                self._callback(self.on_error, e)
                break