RuntimeError `error: 7` - error_msg: maximum recursion depth exceeded

[veejar]

Version of this library.

1.46.1

Solution to Issue cannot be found in the documentation or other Issues and also occurs in the latest version of this library.

• [X] I checked the documentation and other Issues. I am using the latest version of this library.

Hardware?

VPS or other cloud hosting

Operating System?

Linux

Python version?

Python3.10

Installed packages

No response

Logging output

No response

Processing method?

stream_buffer

Used endpoint?

concerns all

Issue

Python: 3.11

Error: BinanceWebSocketApiManager._create_stream_thread() stream_id=e1f977f05884-bb5d-6cc0-3ba8-476bf926 - RuntimeError error: 7 - error_msg: maximum recursion depth exceeded

To start websocker connection was used code:

class BinanceWebsocket:
    def __init__(self, api_key: str, api_secret: str, testnet: bool = False):
        self.api_key = api_key
        self.api_secret = api_secret
        self.testnet = testnet

    def process_stream_buffer(self, websocket_manager):
        while True:
            if websocket_manager.is_manager_stopping():
                exit(0)
            stream_data = websocket_manager.pop_stream_data_from_stream_buffer()
            if stream_data is False:
                time.sleep(0.01)

    def start_user_stream(self):
        wsm = BinanceWebSocketApiManager(exchange=self.get_exchange())
        wsm.create_stream(
            "arr", "!userData", api_key=self.api_key, api_secret=self.api_secret
        )
        worker_thread = threading.Thread(target=self.process_stream_buffer, args=(wsm,))
        worker_thread.start()

        # monitor the stream
        while True:
            # wsm.print_stream_info(stream_id)
            time.sleep(1)

    def get_exchange(self):
        return "binance.com-testnet" if self.testnet else "binance.com"

   # using class above
   ....
   bws = BinanceWebsocket(
        api_key=creds["api_key"],
        api_secret=creds["api_secret"],
        testnet=Settings.binance_testnet,
    )
    bws.start_user_stream(handle_binance_user_ws)

Critical problem: websocket script process was running, but new websocket messages were not received until restart of linux process (restart python script)

[oliver-zehentleitner]

I need a log file please!

[veejar]

That's all, what I have for now. Python script is running in Docker. Error was reported to Sentry. I don't see log files re: that.

[oliver-zehentleitner]

I never have seen this problem.... I need a log file to understand what is going on in your setting: https://github.com/LUCIT-Systems-and-Development/unicorn-binance-websocket-api/wiki/Logging

[Mozilla88]

hi, oliver,

I have the same problem with veejar, when the error occurred: "maximum recursion depth exceeded". version is 1.46.1

Here is our full error log, maybe it will be help for you. Thans a lot for your great effort! error.log

[lr2bmail]

+1 but not when i use 1.42

#celery worker logs
main.log | grep "maximum recursion depth exceeded"

[2023-04-15 21:51:41,064:  raised unexpected: RecursionError('maximum recursion depth exceeded')RecursionError: maximum recursion depth exceeded 
[2023-04-15 22:22:56,743:  raised unexpected: RecursionError('maximum recursion depth exceeded')RecursionError: maximum recursion depth exceeded
[2023-04-20 13:54:21,076:  raised unexpected: RecursionError('maximum recursion depth exceeded')RecursionError: maximum recursion depth exceeded
[2023-04-20 17:44:08,136:  raised unexpected: RecursionError('maximum recursion depth exceeded')RecursionError: maximum recursion depth exceeded
[2023-04-22 12:51:26,981:  raised unexpected: RecursionError('maximum recursion depth exceeded')RecursionError: maximum recursion depth exceeded
[2023-04-24 14:19:18,471:  raised unexpected: RecursionError('maximum recursion depth exceeded')RecursionError: maximum recursion depth exceeded
[2023-04-24 15:02:45,434:  raised unexpected: RecursionError('maximum recursion depth exceeded')RecursionError: maximum recursion depth exceeded
[2023-05-02 10:12:49,755:  raised unexpected: RecursionError('maximum recursion depth exceeded')RecursionError: maximum recursion depth exceeded
[2023-05-02 13:12:14,468:  raised unexpected: RecursionError('maximum recursion depth exceeded')RecursionError: maximum recursion depth exceeded

[Mozilla88]

One way to overcome this problem, is using Process to create the BinanceWebSocketApiManager in a sub process:

    ws_process = Process(target=ws_reader_processor, args=process_args)
    ws_process.start()

then in the sub process catch the error "maximum recursion depth exceeded", using a handler of logging. logging.basicConfig(handlers=[InterceptHandler(error_func)], level=logging.ERROR, force=True)

when this error occurred, terminate the sub process using a flag:

    while True:
        if critical_error is not None:
            logger.error(f"Critical error occurred, websocket process exit!")
            os._exit(1)
        time.sleep(1)

in the main process, test if the sub processs is alive, if not alive, restart sub process.

      while True:
          if not ws_process.is_alive():
              logger.error(f"Websocket process is not alive, restart it!")
              start_ws_process()

[oliver-zehentleitner]

hi, oliver,

I have the same problem with veejar, when the error occurred: "maximum recursion depth exceeded". version is 1.46.1

Here is our full error log, maybe it will be help for you. Thans a lot for your great effort! error.log

Thank you very much, but logging doesn't help me much in this way. Please use the following code for logging:

logging.getLogger("unicorn_binance_websocket_api")
logging.basicConfig(level=logging.DEBUG,
                    filename=os.path.basename(__file__) + '.log',
                    format="{asctime} [{levelname:8}] {process} {thread} {module}: {message}",
                    style="{")

The error message comes from Colorama, a third-party library. You could deactivate it:

BinanceWebSocketApiManager(disable_colorama=True)

I looked at your code again and noticed that you may create a new UBWA instance frequently. Just create one instance and use it over and over again, then Colorama will only be instantiated once.

Do this once in __init__() instead of doing it everytime in start_user_stream():

        wsm = BinanceWebSocketApiManager(exchange=self.get_exchange())

Here is a "best practice" snippet, try it - i think the problems will be gone: https://gist.github.com/oliver-zehentleitner/560c102d7867a2743aeba13720e8083b#file-best_practice_ubwa_full-py

[oliver-zehentleitner]

I looked through the log file again and found the error message from _create_stream_thread(). i'll have a look.

[oliver-zehentleitner]

I have just compared the old codebase (https://github.com/LUCIT-Systems-and-Development/unicorn-binance-websocket-api/blob/301c473f689ab220247dacd570f606bdd4798aa2/unicorn_binance_websocket_api/manager.py) with the current one. I have improved the loop management a lot since 1.46.1. The restart is always done from the _keepalive_streams() thread and is NOT recursive! Please update and test with the best practice templates: https://medium.lucit.tech/how-to-obtain-and-use-a-unicorn-binance-suite-license-key-and-run-the-ubs-module-according-to-best-87b0088124a8#417c

I am also available in the chat to analyse the problems in detail together: https://www.lucit.tech/get-support.html

If this problem still exists in the latest version (>=2.2.0), please post the code and log file here.

[oliver-zehentleitner]

In 2.3.0, the restart procedure from another thread no longer exists. The error, if caused by UBWA, can no longer occur.

https://pypi.org/project/unicorn-binance-websocket-api/2.3.0/