xilopaint
commented
2 years ago i thought you meant in the post before it works on mac too...
Yes, it works on macOS.
Closed jon4hz closed 2 years ago
xilopaint
commented
2 years ago i thought you meant in the post before it works on mac too...
Yes, it works on macOS.
oliver-zehentleitner
commented
2 years ago Thanks, i tested it on macOS with your script: https://github.com/LUCIT-Systems-and-Development/unicorn-binance-websocket-api/blob/master/dev_test_issue_131_xilopaint.py
And it seems to work fine now :)
xilopaint
commented
2 years ago Thanks, i tested it on macOS with your script: https://github.com/LUCIT-Systems-and-Development/unicorn-binance-websocket-api/blob/master/dev_test_issue_131_xilopaint.py
And it seems to work fine now :)
So we won't have a solution on the lib side? In this case, maybe we should have something about this workaround on README.
oliver-zehentleitner
commented
2 years ago I think we are talking past each other :)
I started this file on my mac: https://github.com/LUCIT-Systems-and-Development/unicorn-binance-websocket-api/blob/master/dev_test_issue_131_xilopaint.py
and it works with the current ubwa version with python3.10.2
xilopaint
commented
2 years ago I started this file on my mac: https://github.com/LUCIT-Systems-and-Development/unicorn-binance-websocket-api/blob/master/dev_test_issue_131_xilopaint.py
and it works with the current ubwa version with python3.10.2
Really? It doesn't work on my Mac. But my Python version is 3.9.10 installed with Homebrew.
oliver-zehentleitner
commented
2 years ago The one bug in this thead has been fixed, the other one on macOS remains. I need more information please, on my macOS the snippet of xilopaint works fine unfortunately :/
Anyone affected by the bug please update to the latest version of UBWA and post DEBUG logs. Thank you!
oliver-zehentleitner
commented
2 years ago Both issues are fixed now!
Thanks @xilopaint for the hints and testing!!
https://github.com/LUCIT-Systems-and-Development/unicorn-binance-websocket-api/releases/tag/1.38.1
Czx00000
commented
1 year ago 
dima-dmytruk23
commented
1 year ago @oliver-zehentleitner @xilopaint @jon4hz
I also have this problem.
websockets==10.2
unicorn-binance-websocket-api==1.40.7
python v3.9.16
def execute(self):
self.stream_id = self.manager.create_stream(
channels=['kline_1h'],
markets=self.symbols,
process_stream_data=self.handle_socket_message,
)
monitoring_thread = Thread(target=self.monitoring_streams)
monitoring_thread.start()
# ... another logic ...
def monitoring_streams(self):
while True:
try:
logger.info(self.manager.get_monitoring_status_plain())
logger.info(self.manager.get_stream_list())
logger.info(self.manager.print_summary())
except Exception as e:
logger.exception(e)
time.sleep(15)@dima-dmytruk23
I think the problem can be handled only with complete code.
Maybe this approach helps you:
from unicorn_binance_websocket_api import BinanceWebSocketApiManager
import asyncio
import logging
import os
logging.getLogger("unicorn_binance_websocket_api")
logging.basicConfig(level=logging.DEBUG,
filename=os.path.basename(__file__) + '.log',
format="{asctime} [{levelname:8}] {process} {thread} {module}: {message}",
style="{")
class Test:
def __init__(self, apikey, apisecret):
self.apikey = apikey
self.apisecret = apisecret
self.ubwa = None
async def start(self):
def handle_socket_message(data):
print(f"msg: {data}")
self.ubwa = BinanceWebSocketApiManager(exchange='binance.com-margin', output_default='dict')
self.ubwa.create_stream(["arr"], ["!userData"], process_stream_data=handle_socket_message,
api_key=self.apikey, api_secret=self.apisecret)
print('Started')
while True:
await asyncio.sleep(1)
if __name__ == "__main__":
test = Test("key", "sec")
try:
asyncio.run(test.start())
except KeyboardInterrupt:
print("Gracefully stopping the websocket manager...")
test.ubwa.stop_manager_with_all_streams()
cxfshelter
commented
3 months ago I have some findings on this topic. On my bot project I got the error at the beginning of the execution and my script couldn't run at all. This is a minimal way for reproducing the issue:
from unicorn_binance_websocket_api.unicorn_binance_websocket_api_manager import BinanceWebSocketApiManager API_KEY = '*' API_SECRET = '*' binance_websocket_api_manager = BinanceWebSocketApiManager( exchange='binance.com-futures', ) binance_websocket_api_manager.create_stream( channels=['!userData'], markets=['arr'], api_key=API_KEY, api_secret=API_SECRET )$ python3 sample.py CRITICAL:root:BinanceWebSocketApiManager._create_stream_thread() stream_id=db732ee8-01bc-492f-98e4-caa0e43654ea - RuntimeError error_msg: - cannot schedule new futures after interpreter shutdown - stopping and shutting down - read https://github.com/oliver-zehentleitner/unicorn-binance-websocket-api/issues/131 for further information!What I found out is that a simple
whileloop at the end of the script is enough to workaround the problem:from unicorn_binance_websocket_api.unicorn_binance_websocket_api_manager import BinanceWebSocketApiManager import time API_KEY = '*' API_SECRET = '*' binance_websocket_api_manager = BinanceWebSocketApiManager( exchange='binance.com-futures', ) binance_websocket_api_manager.create_stream( channels=['!userData'], markets=['arr'], api_key=API_KEY, api_secret=API_SECRET ) while True: time.sleep(1)I can't be sure why it works but the issue was fixed on my project once I added a
whileloop at the end of the script for thebinance_websocket_api_manager.print_summary()method so I could consume the websocket data normally. Maybe the loop prevents the interpreter from shutting down at the beginning of the execution so the python process keeps running.Another thing I found out is that the issue seems to be related with the use of
threading.Thread()method. If I replacethreading.Thread()with awhileloop to consume the websocket everything works fine. However, even usingthreading.Thread()you can workaround the issue if you use an emptywhileloop at the end.
thank you sincerely, helped me solve the problem that had been bothering me for two days
oliver-zehentleitner
commented
3 months ago I have published best practice examples for all modules of our suite here: https://medium.lucit.tech/how-to-obtain-and-use-a-unicorn-binance-suite-license-key-and-run-the-ubs-module-according-to-best-87b0088124a8#417c (Will be continuously revised and adapted)
I work a lot with the libraries and recommend this approach as it is the most resource-efficient, fastest and most robust - here is the example for UBWA: https://gist.github.com/oliver-zehentleitner/560c102d7867a2743aeba13720e8083b#file-best_practice_ubwa_full-py
from unicorn_binance_websocket_api import BinanceWebSocketApiManager
import asyncio
import logging
import os
logging.getLogger("unicorn_binance_websocket_api")
logging.basicConfig(level=logging.INFO,
filename=os.path.basename(__file__) + '.log',
format="{asctime} [{levelname:8}] {process} {thread} {module}: {message}",
style="{")
class BinanceDataProcessor:
def __init__(self, print_new_data=False, start_multiplex=True, start_userdata_a=True):
self.api_key = "YOUR_BINANCE_API_KEY"
self.api_secret = "YOUR_BINANCE_API_SECRET"
self.example_database = []
self.print_new_data = print_new_data
self.start_multiplex = start_multiplex
self.start_userdata_a = start_userdata_a
self.ubwa = BinanceWebSocketApiManager(exchange='binance.com',
auto_data_cleanup_stopped_streams=True,
enable_stream_signal_buffer=True,
output_default='UnicornFy',
process_stream_signals=self.receive_stream_signal)
async def main(self):
if self.start_multiplex is True:
self.ubwa.create_stream(channels=['trade', 'kline_1m', 'depth5'],
markets=['btcusdt', 'ethusdt', 'bnbusdt'],
process_asyncio_queue=self.processing_of_new_data,
stream_label="multiplex")
if self.start_userdata_a is True:
self.ubwa.create_stream(api_key=self.api_key, api_secret=self.api_secret,
channels=["arr"], markets=["!userData"],
process_asyncio_queue=self.processing_of_new_data,
stream_label="userData_A")
while self.ubwa.is_manager_stopping() is False:
await asyncio.sleep(1)
stream_info = \
{'multiplex': self.ubwa.get_stream_info(stream_id=self.ubwa.get_stream_id_by_label('multiplex')),
'userData_A': self.ubwa.get_stream_info(stream_id=self.ubwa.get_stream_id_by_label('userData_A'))}
status_text = ""
if self.start_multiplex is True:
status_text += (f"\tStream 'multiplex' is {stream_info['multiplex']['status']} "
f"(last_stream_signal={stream_info['multiplex']['last_stream_signal']})\r\n")
if self.start_userdata_a is True:
status_text += (f"\tStream 'userData_A' is {stream_info['userData_A']['status']} "
f"(last_stream_signal={stream_info['userData_A']['last_stream_signal']})\r\n")
print(f"Status:\r\n\tStored {len(self.example_database)} data records in `self.example_database`\r\n"
f"{status_text}")
async def processing_of_new_data(self, stream_id=None):
print(f"Saving the data from webstream {self.ubwa.get_stream_label(stream_id=stream_id)} to the database ...")
while self.ubwa.is_stop_request(stream_id=stream_id) is False:
data = await self.ubwa.get_stream_data_from_asyncio_queue(stream_id)
self.example_database.append(data)
if self.print_new_data is True:
print(f"Data record received and added to the database: {data}")
self.ubwa.asyncio_queue_task_done(stream_id)
def receive_stream_signal(self, signal_type=None, stream_id=None, data_record=None, error_msg=None):
# More info about `stream_signals`:
# https://github.com/LUCIT-Systems-and-Development/unicorn-binance-websocket-api/wiki/%60stream_signals%60
print(f"Received stream_signal for stream '{self.ubwa.get_stream_label(stream_id=stream_id)}': "
f"{signal_type} - {stream_id} - {data_record} - {error_msg}")
if __name__ == "__main__":
bdp = BinanceDataProcessor(print_new_data=False, start_multiplex=True, start_userdata_a=True)
try:
asyncio.run(bdp.main())
except KeyboardInterrupt:
print("\r\nGracefully stopping ...")
except Exception as e:
print(f"\r\nERROR: {e}")
print("Gracefully stopping ...")
bdp.ubwa.stop_manager()
To explain what is happening here: A new thread is started in the main thread and shortly afterwards your main thread ends (even before the new one has finished starting). This leads to problems.
Even a time.sleep(1) afterwards would solve the problem. Actually, there is already a fix implemented in newer versions that fixes the problem regularly, but it only works if high_performance=False! This prevents create_stream() from exiting before the thread is started and thus the main thread remains long enough.
Check this or we will delete your issue. (fill in the checkbox with an X like so: [x])
Select one:
Environment
What kind of internet connection do you have?
~1-2 GB/s
Average system load (CPU)
45%
Hardware specification
VPS, Vultr
Operating System? (include version)
Options
Python Version Requirement
Exact Python Version?
Pip Version?
Dependencies
Run
pip list > pip_list.txtand upload the file.UNICORN Binance WebSocket API Version?
Description of your issue