search

LUCIT-Systems-and-Development / unicorn-binance-websocket-api

A Python SDK by LUCIT to use the Binance Websocket API`s (com+testnet, com-margin+testnet, com-isolated_margin+testnet, com-futures+testnet, com-coin_futures, us, tr, dex/chain+testnet) in a simple, fast, flexible, robust and fully-featured way.
https://unicorn-binance-websocket-api.docs.lucit.tech/
Other
685 stars 165 forks source link

Process fails to exit if a user stream is open #137

Closed dpugachev77 closed 3 years ago

dpugachev77 commented 3 years ago

Check this or we will delete your issue. (fill in the checkbox with an X like so: [x])

Select one:

Environment

What kind of internet connection do you have?

Average system load (CPU)

Hardware specification

Operating System? (include version)

Options

Python Version Requirement

Exact Python Version?

Python 3.7.3

Pip Version?

pip 18.1 from /usr/lib/python3/dist-packages/pip (python 3.7)

Dependencies

Run pip list > pip_list.txt and upload the file.

UNICORN Binance WebSocket API Version?

Did you upgrade to the latest release version with `pip install unicorn-binance-websocket-api --upgrade`?

Please control what version you are using with this script and post the output: https://github.com/oliver-zehentleitner/unicorn-binance-websocket-api/blob/master/tools/get_used_module_version.py

Description of your issue

If a user data stream is open, the process fails to exit normally (hangs up indefinitely).  No other streams are open.  If any other stream is open (but not a user data stream), the process exits as expected.  Example:

    client = Client(api_key, api_secret)
    bsm_f = BinanceWebSocketApiManager(process_stream_data=callback_f, exchange="binance.com-futures", output_default="dict")
    user_stream = bsm_f.create_stream(["arr"], ["!userData"], api_key=api_key, api_secret=api_secret)
    sleep(5)
    bsm_f.stop_manager_with_all_streams()
    sleep(5)
    sys.exit(0)

Last messages in the log:

2020-12-05 00:03:29,135 [DEBUG   ] 1040 140439536498432 unicorn_binance_websocket_api_manager: BinanceWebSocketApiManager.set_heartbeat(d7d397c0-f227-46a4-8dd7-56b333451a2e)
2020-12-05 00:03:29,135 [DEBUG   ] 1040 140439536498432 unicorn_binance_websocket_api_manager: BinanceWebSocketApiManager.is_stop_request(d7d397c0-f227-46a4-8dd7-56b333451a2e)
2020-12-05 00:03:29,135 [DEBUG   ] 1040 140439536498432 unicorn_binance_websocket_api_manager: BinanceWebSocketApiManager.is_stop_as_crash_request(d7d397c0-f227-46a4-8dd7-56b333451a2e)
2020-12-05 00:03:29,135 [DEBUG   ] 1040 140439536498432 unicorn_binance_websocket_api_manager: BinanceWebSocketApiManager.set_heartbeat(d7d397c0-f227-46a4-8dd7-56b333451a2e)
2020-12-05 00:03:33,744 [INFO    ] 1040 140439584999232 unicorn_binance_websocket_api_manager: BinanceWebSocketApiManager.stop_manager_with_all_streams() - Stopping unicorn_binance_websocket_api_manager 1.27.0 ...
2020-12-05 00:03:33,745 [INFO    ] 1040 140439584999232 unicorn_binance_websocket_api_manager: BinanceWebSocketApiManager.stop_stream(d7d397c0-f227-46a4-8dd7-56b333451a2e)
2020-12-05 00:03:49,156 [DEBUG   ] 1040 140439536498432 protocol: client > Frame(fin=True, opcode=9, data=b'T\x0b\xf0\x01', rsv1=False, rsv2=False, rsv3=False)
2020-12-05 00:03:49,160 [DEBUG   ] 1040 140439536498432 protocol: client - event = data_received(<6 bytes>)
2020-12-05 00:03:49,161 [DEBUG   ] 1040 140439536498432 protocol: client < Frame(fin=True, opcode=10, data=b'T\x0b\xf0\x01', rsv1=False, rsv2=False, rsv3=False)
2020-12-05 00:03:49,161 [DEBUG   ] 1040 140439536498432 protocol: client - received solicited pong: 540bf001

If the user stream line is commented out, the process exits as expected.
oliver-zehentleitner commented 3 years ago

does stop_stream() work with the user_data stream?

maybe the loop needs an action like send, receive or ping to execute the shutdown. if stop_stream() doenst work - maybe it works if a payload gets send to the endpoint, just use get_stream_subscriptions() after stop_stream() to test this out - please tell me if that helps!

dpugachev77 commented 3 years ago

Unfortunately, adding stop_stream() and get_stream_subscriptions() does not change anything... Here is the new code:

client = Client(api_key, api_secret)
bsm_f = BinanceWebSocketApiManager(exchange="binance.com-futures", output_default="dict")
user_stream = bsm_f.create_stream(["arr"], ["!userData"], api_key=api_key, api_secret=api_secret)
print("After create_stream (stream_id = " + str(user_stream) + ")")
sleep(5)
print("Before stop_stream")
bsm_f.stop_stream(user_stream)
print("After stop_stream")
print("Before get_stream_subscriptions")
request_id = bsm_f.get_stream_subscriptions(user_stream)
print("After get_stream_subscriptions (request_id = " + str(request_id) + ")")
print("Before get_result_by_request_id")
result = bsm_f.get_result_by_request_id(request_id)
print("After get_result_by_request_id (result = " + str(result) + ")")
print("Before stop_manager_with_all_streams")
bsm_f.stop_manager_with_all_streams()
print("After stop_manager_with_all_streams")
sys_exit(0)

Result:

After create_stream (stream_id = a50130b1-a201-487b-8305-ccd40eabf928) Before stop_stream After stop_stream Before get_stream_subscriptions After get_stream_subscriptions (request_id = 1) Before get_result_by_request_id After get_result_by_request_id (result = False) Before stop_manager_with_all_streams After stop_manager_with_all_streams [the process hangs here, no further output and no exit]

Last messages in the log:

2020-12-06 18:15:25,594 [DEBUG ] 1932 139936851040000 client: client < HTTP/1.1 101 Switching Protocols 2020-12-06 18:15:25,594 [DEBUG ] 1932 139936851040000 client: client < Headers([('Date', 'Sun, 06 Dec 2020 18:15:25 GMT'), ('Connection', 'upgrade'), ('upgrade', 'websocket'), ('sec-websocket-accept', '5OSiMxfdrPsB9GDvPoq3Bbvej7g='), ('sec-websocket-extensions', 'permessage-deflate')]) 2020-12-06 18:15:25,594 [DEBUG ] 1932 139936851040000 protocol: client - state = OPEN 2020-12-06 18:15:25,594 [DEBUG ] 1932 139936851040000 unicorn_binance_websocket_api_manager: BinanceWebSocketApiManager.set_heartbeat(a50130b1-a201-487b-8305-ccd40eabf928) 2020-12-06 18:15:25,594 [DEBUG ] 1932 139936851040000 unicorn_binance_websocket_api_manager: BinanceWebSocketApiManager.is_stop_request(a50130b1-a201-487b-8305-ccd40eabf928) 2020-12-06 18:15:25,594 [DEBUG ] 1932 139936851040000 unicorn_binance_websocket_api_manager: BinanceWebSocketApiManager.is_stop_as_crash_request(a50130b1-a201-487b-8305-ccd40eabf928) 2020-12-06 18:15:25,594 [DEBUG ] 1932 139936851040000 unicorn_binance_websocket_api_manager: BinanceWebSocketApiManager.set_heartbeat(a50130b1-a201-487b-8305-ccd40eabf928) 2020-12-06 18:15:30,165 [INFO ] 1932 139936899540800 unicorn_binance_websocket_api_manager: BinanceWebSocketApiManager.stop_stream(a50130b1-a201-487b-8305-ccd40eabf928) 2020-12-06 18:15:30,165 [INFO ] 1932 139936899540800 unicorn_binance_websocket_api_manager: BinanceWebSocketApiManager.get_stream_subscriptions(a50130b1-a201-487b-8305-ccd40eabf928, 1) payload added! 2020-12-06 18:15:40,166 [INFO ] 1932 139936899540800 unicorn_binance_websocket_api_manager: BinanceWebSocketApiManager.stop_manager_with_all_streams() - Stopping unicorn_binance_websocket_api_manager 1.27.0 ... 2020-12-06 18:15:40,166 [INFO ] 1932 139936899540800 unicorn_binance_websocket_api_manager: BinanceWebSocketApiManager.stop_stream(a50130b1-a201-487b-8305-ccd40eabf928) 2020-12-06 18:15:45,595 [DEBUG ] 1932 139936851040000 protocol: client > Frame(fin=True, opcode=9, data=b'\xf1KX:', rsv1=False, rsv2=False, rsv3=False) 2020-12-06 18:15:45,602 [DEBUG ] 1932 139936851040000 protocol: client - event = data_received(<6 bytes>) 2020-12-06 18:15:45,602 [DEBUG ] 1932 139936851040000 protocol: client < Frame(fin=True, opcode=10, data=b'\xf1KX:', rsv1=False, rsv2=False, rsv3=False) 2020-12-06 18:15:45,602 [DEBUG ] 1932 139936851040000 protocol: client - received solicited pong: f14b583a 2020-12-06 18:16:05,623 [DEBUG ] 1932 139936851040000 protocol: client > Frame(fin=True, opcode=9, data=b'\xf4j\x8f\x1f', rsv1=False, rsv2=False, rsv3=False) 2020-12-06 18:16:05,631 [DEBUG ] 1932 139936851040000 protocol: client - event = data_received(<6 bytes>) 2020-12-06 18:16:05,631 [DEBUG ] 1932 139936851040000 protocol: client < Frame(fin=True, opcode=10, data=b'\xf4j\x8f\x1f', rsv1=False, rsv2=False, rsv3=False) 2020-12-06 18:16:05,631 [DEBUG ] 1932 139936851040000 protocol: client - received solicited pong: f46a8f1f ...

oliver-zehentleitner commented 3 years ago

thanks for testing and the good explanation. till now we are not awaiting a ping/pong, its managed by the lib websockets. maybe it could help to await it and add code for stopping the stream. If you want play around and fix it, a PR is appreciated.

moritzgun commented 3 years ago

Are there any news regarding this issue? My current workaround is sending a limit order and immediatly canceling it right after the disconnection happend.. I dont really like that approach though :D

oliver-zehentleitner commented 3 years ago

Are there any news regarding this issue? My current workaround is sending a limit order and immediatly canceling it right after the disconnection happend.. I dont really like that approach though :D

haha :D

oliver-zehentleitner commented 3 years ago

I close this since this is error is a subset of this: https://github.com/oliver-zehentleitner/unicorn-binance-websocket-api/issues/161

zenoftrading commented 3 years ago

A have the same issue. Sorry, but I don`t understand how I can stop all streams? I updated unicorn-binance-websocket-api to v1.30.0 After stop_manager_with_all_streams() like an @dpugachev77 [the process hangs here, no further output and no exit]