UNICORN BINANCE CRASHES

[chomparoja]

Hi" Im using the latest version of unicorn binance websocket api, and it crashes every 2 minutes. CRITICAL ubwa, BinaceWebSocketApiManager.stream_is_crashing... I'm pointing to Binace futures.

CRITICAL:unicorn_binance_websocket_api.unicorn_binance_websocket_api_socket:BinanceWebSocketApiSocket.start_socket(3391e91a92f0-ef59-492b-5f88-4cf06487, ['kline_1m'], ['axsusdt', 'alphausdt']) - Exception ConnectionClosed - error_msg: sent 1011 (unexpected error) keepalive ping timeout; no close frame received CRITICAL:unicorn_binance_websocket_api.unicorn_binance_websocket_api_manager:BinanceWebSocketApiManager.stream_is_crashing(3391e91a92f0-ef59-492b-5f88-4cf06487) not updated symbols: 0

[oliver-zehentleitner]

To less infos to help you!

We catch the exception and restart, but if the ping timeouts i suggest your network is weak or your system overloaded. Is this possible?

[chomparoja]

Hello! I have upgrade to the latest version but is working even worst. Right now I’m uninstalling it and getting back to previous version 1.34.2, which is working much better with less crashes.

Got a very good broadband, 100 Mbps, and an effective 11 Mbps at the trading server.

One question, python-binance is asking for websockets 9.1 and unicorn api for 10.0… understand that the dependency is with 10.0?

My system is not overload, got a 4 core cpu, working at max with 42% with all the processes and threads.

Will report the results of the test during overnight.

Best regards,

Santiago

From: Oliver Zehentleitner @.> Sent: 04 January 2022 18:58 To: LUCIT-Systems-and-Development/unicorn-binance-websocket-api @.> Cc: chomparoja @.>; Author @.> Subject: Re: [LUCIT-Systems-and-Development/unicorn-binance-websocket-api] UNICORN BINANCE CRASHES (Issue #228)

To less infos to help you!

We catch the exception and restart, but if the ping timeouts i suggest your network is weak or your system overloaded. Is this possible?

— Reply to this email directly, view it on GitHub https://github.com/LUCIT-Systems-and-Development/unicorn-binance-websocket-api/issues/228#issuecomment-1005200258 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AWZIE362BC6BBXXRISMIBZDUUNUNTANCNFSM5LINNVXA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . You are receiving this because you authored the thread. https://github.com/notifications/beacon/AWZIE36JZB4JYR276WLEKKTUUNUNTA5CNFSM5LINNVXKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOHPVCHAQ.gif Message ID: @. @.> >

[oliver-zehentleitner]

ubwa 1.34.2 is using websockets10.0 ubwa 1.35.0 is using websockets10.1

if you have 4 cores than python is only able to use 25% for your processes, except you use subprocesses... as mentioned here:

This lib is using multi-threading which is not really parallel in python - through the GIL all threads are executed sequential in a cycle. Divide 100 with your cpu cores and you have the max % of cpu power that your program (and this lib) can use. If your streams are needing more power than they can get, they restart after a while!

[chomparoja]

Hi Oliver! Thank you for your feedback.

During the last 12 hours using v.1.24.2 situation was much bette but got various crashes, 3 types, as per below details:

Exception ConnectionClosed - error_msg: code = 1006 (connection closed abnormally [internal]), no reason

CRITICAL:root:BinanceWebSocketApiManager.stream_is_crashing(ab829989-fa93-4733-b589-e80e9ff44042)

socket_id=f3257af8-096a-413a-bbc2-3e453b9bf451 recent_socket_id=f3257af8-096a-413a-bbc2-3e453b9bf451 - Sending payload - exit because its not the recent socket id! stream_id=ab829989-fa93-4733-b589-e80e9ff44042, recent_socket_id=3b326e31-05e4-4bbb-9592-cb2ce4c163b3

No internet connection? - error_msg: [Errno 11001] getaddrinfo failed

CRITICAL:root:BinanceWebSocketApiManager.stream_is_crashing(ab829989-fa93-4733-b589-e80e9ff44042)

Is there any problem as when installing websockets 10.0 appears this message:

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.

python-binance 1.0.15 requires websockets==9.1, but you have websockets 10.0 which is incompatible.

Successfully installed websockets-10.0

Best regards,

Santiago

From: Oliver Zehentleitner @.> Sent: 05 January 2022 06:11 To: LUCIT-Systems-and-Development/unicorn-binance-websocket-api @.> Cc: chomparoja @.>; Author @.> Subject: Re: [LUCIT-Systems-and-Development/unicorn-binance-websocket-api] UNICORN BINANCE CRASHES (Issue #228)

ubwa 1.34.2 is using websockets10.0 ubwa 1.35.0 is using websockets10.1

if you have 4 cores than python is only able to use 25% for your processes, except you use subprocesses... as mentioned here https://github.com/LUCIT-Systems-and-Development/unicorn-binance-websocket-api/issues/113 :

This lib is using multi-threading which is not really parallel in python - through the GIL all threads are executed sequential in a cycle. Divide 100 with your cpu cores and you have the max % of cpu power that your program (and this lib) can use. If your streams are needing more power than they can get, they restart after a while!

— Reply to this email directly, view it on GitHub https://github.com/LUCIT-Systems-and-Development/unicorn-binance-websocket-api/issues/228#issuecomment-1005505682 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AWZIE3574OP2GBW2GY4Z3ULUUQDK7ANCNFSM5LINNVXA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . You are receiving this because you authored the thread. https://github.com/notifications/beacon/AWZIE32GDTN4PWCU46IGYSDUUQDK7A5CNFSM5LINNVXKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOHPXMZEQ.gif Message ID: @. @.> >

[oliver-zehentleitner]

I am pretty sure you have a weak internet/server, not the bandwidth but its not stable enough!

just try the same script on a 10 EUR server at digital ocean, google or aws... or with less subscriptions.

Please read https://github.com/LUCIT-Systems-and-Development/unicorn-binance-websocket-api/issues/113 to get an understanding for your problem.

To avoid the problem with python-binance you can use this fork: https://github.com/LUCIT-Systems-and-Development/unicorn-binance-rest-api

It does not support asyncio atm, but it works perfectly together with UBWA.

[chomparoja]

Thank you Oliver for your reply.

Have downgraded to 1.34.2 and I get much less crashed than with the 35 version.

For sure will try with my account on aws, which I have to upgrade, as with 1 cpu is not working, as always is at 100% cpu!

Guess that with 4 cores will be ok.

Still wondering why the ubwa 1.35 is not working…can you send me the dependencies check up file..as something must be wrong in my pc.

Best regards from Argentina!

Santiago

From: Oliver Zehentleitner @.> Sent: 05 January 2022 09:51 To: LUCIT-Systems-and-Development/unicorn-binance-websocket-api @.> Cc: chomparoja @.>; Author @.> Subject: Re: [LUCIT-Systems-and-Development/unicorn-binance-websocket-api] UNICORN BINANCE CRASHES (Issue #228)

I am pretty sure you have a weak internet/server, not the bandwidth but its not stable enough!

just try the same script on a 10 EUR server at digital ocean, google or aws... or with less subscriptions.

Please read #113 https://github.com/LUCIT-Systems-and-Development/unicorn-binance-websocket-api/issues/113 to get an understanding for your problem.

To avoid the problem with python-binance you can use this fork: https://github.com/LUCIT-Systems-and-Development/unicorn-binance-rest-api

It does not support asyncio atm, but it works perfectly together with UBWA.

— Reply to this email directly, view it on GitHub https://github.com/LUCIT-Systems-and-Development/unicorn-binance-websocket-api/issues/228#issuecomment-1005659165 , or unsubscribe https://github.com/notifications/unsubscribe-auth/AWZIE3ZKXHIZ2YA46CXP4MTUUQ5CPANCNFSM5LINNVXA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . You are receiving this because you authored the thread. https://github.com/notifications/beacon/AWZIE3YGARJEK4LCLZY4X2LUUQ5CPA5CNFSM5LINNVXKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOHPYSIHI.gif Message ID: @. @.> >

[oliver-zehentleitner]

the error msg you posted like "No internet connection? - error_msg: [Errno 11001] getaddrinfo failed" means that there was no network available.... i know this error! :)

as i can read in your answer you did not understand what i wrote in https://github.com/LUCIT-Systems-and-Development/unicorn-binance-websocket-api/issues/113 and is also described here https://medium.com/python-features/pythons-gil-a-hurdle-to-multithreaded-program-d04ad9c1a63

then you understand why 4 cores is not much better than 1 core if you work with just one python process.

i close this as i think you problem is clear: internet loss and i also think an overloaded python process

[chomparoja]

Hi Oliver!

I have a new installation on a vps, running python 3.10, and the latest version of unicorn binance api.

After many installations, still receiving the same error

(base) C:\Users\Administrator\Documents\BinanceBot_Live>python binance_data1.py

Traceback (most recent call last):

File "C:\Users\Administrator\Documents\BinanceBot_Live\binance_data1.py", line 12, in

from unicorn_binance_websocket_api.unicorn_binance_websocket_api_manager import BinanceWebSocketApiManager

ModuleNotFoundError: No module named 'unicorn_binance_websocket_api.unicorn_binance_websocket_api_manager'

Thank you in advance for your feedback.

Best regards,

Santiago

Package Version

---

aiohttp 3.8.1 aiosignal 1.2.0 alabaster 0.7.12 anaconda-client 1.9.0 anaconda-navigator 2.1.1 anaconda-project 0.10.1 aniso8601 9.0.1 anyio 2.2.0 appdirs 1.4.4 argh 0.26.2 argon2-cffi 20.1.0 arrow 0.13.1 asn1crypto 1.4.0 astroid 2.6.6 astropy 4.3.1 async-generator 1.10 async-timeout 4.0.2 atomicwrites 1.4.0 attrs 21.2.0 autopep8 1.5.7 Babel 2.9.1 backcall 0.2.0 backports.functools-lru-cache 1.6.4 backports.shutil-get-terminal-size 1.0.0 backports.tempfile 1.0 backports.weakref 1.0.post1 bcrypt 3.2.0 beautifulsoup4 4.10.0 binaryornot 0.4.4 bitarray 2.3.0 bkcharts 0.2 black 19.10b0 bleach 4.0.0 bokeh 2.4.1 boto 2.49.0 Bottleneck 1.3.2 brotlipy 0.7.0 cached-property 1.5.2 certifi 2021.10.8 cffi 1.14.6 chardet 4.0.0 charset-normalizer 2.0.4 cheroot 8.6.0 click 8.0.3 cloudpickle 2.0.0 clyent 1.2.2 colorama 0.4.4 comtypes 1.1.10 conda 4.10.3 conda-build 3.21.6 conda-content-trust 0+unknown conda-pack 0.6.0 conda-package-handling 1.7.3 conda-repo-cli 1.0.4 conda-token 0.3.0 conda-verify 3.4.2 contextlib2 0.6.0.post1 cookiecutter 1.7.2 cryptography 3.4.8 cycler 0.10.0 Cython 0.29.24 cytoolz 0.11.0 daal4py 2021.3.0 dask 2021.10.0 dateparser 1.1.0 debugpy 1.4.1 decorator 5.1.0 defusedxml 0.7.1 diff-match-patch 20200713 distributed 2021.10.0 docutils 0.17.1 entrypoints 0.3 et-xmlfile 1.1.0 fastcache 1.1.0 filelock 3.3.1 flake8 3.9.2 Flask 1.1.2 Flask-RESTful 0.3.9 fonttools 4.25.0 frozenlist 1.3.0 fsspec 2021.10.1 future 0.18.2 gevent 21.8.0 glob2 0.7 greenlet 1.1.1 h5py 3.2.1 HeapDict 1.0.1 html5lib 1.1 idna 3.2 imagecodecs 2021.8.26 imageio 2.9.0 imagesize 1.2.0 importlib-metadata 4.8.1 inflection 0.5.1 iniconfig 1.1.1 intervaltree 3.1.0 ipykernel 6.4.1 ipython 7.29.0 ipython-genutils 0.2.0 ipywidgets 7.6.5 isort 5.9.3 itsdangerous 2.0.1 jaraco.functools 3.5.0 jdcal 1.4.1 jedi 0.18.0 Jinja2 2.11.3 jinja2-time 0.2.0 joblib 1.1.0 json5 0.9.6 jsonschema 3.2.0 jupyter 1.0.0 jupyter-client 6.1.12 jupyter-console 6.4.0 jupyter-core 4.8.1 jupyter-server 1.4.1 jupyterlab 3.2.1 jupyterlab-pygments 0.1.2 jupyterlab-server 2.8.2 jupyterlab-widgets 1.0.0 keyring 23.1.0 kiwisolver 1.3.1 lazy-object-proxy 1.6.0 libarchive-c 2.9 llvmlite 0.37.0 locket 0.2.1 lxml 4.6.3 MarkupSafe 1.1.1 matplotlib 3.4.3 matplotlib-inline 0.1.2 mccabe 0.6.1 menuinst 1.4.18 mistune 0.8.4 mkl-fft 1.3.1 mkl-random 1.2.2 mkl-service 2.4.0 mock 4.0.3 more-itertools 8.10.0 mpmath 1.2.1 msgpack 1.0.2 multidict 6.0.2 multipledispatch 0.6.0 munkres 1.1.4 mypy-extensions 0.4.3 navigator-updater 0.2.1 nbclassic 0.2.6 nbclient 0.5.3 nbconvert 6.1.0 nbformat 5.1.3 nest-asyncio 1.5.1 networkx 2.6.3 nltk 3.6.5 nose 1.3.7 notebook 6.4.5 numba 0.54.1 numexpr 2.7.3 numpy 1.20.3 numpydoc 1.1.0 olefile 0.46 openpyxl 3.0.9 packaging 21.0 pandas 1.3.4 pandocfilters 1.4.3 paramiko 2.7.2 parso 0.8.2 partd 1.2.0 path 16.0.0 pathlib2 2.3.6 pathspec 0.7.0 patsy 0.5.2 pep8 1.7.1 pexpect 4.8.0 pickleshare 0.7.5 Pillow 8.4.0 pip 21.2.4 pkginfo 1.7.1 pluggy 0.13.1 ply 3.11 poyo 0.5.0 prometheus-client 0.11.0 prompt-toolkit 3.0.20 psutil 5.8.0 ptyprocess 0.7.0 py 1.10.0 pyasn1 0.4.8 pyasn1-modules 0.2.8 pycodestyle 2.7.0 pycosat 0.6.3 pycparser 2.20 pycurl 7.44.1 pydocstyle 6.1.1 pyerfa 2.0.0 pyflakes 2.3.1 Pygments 2.10.0 PyJWT 2.1.0 pylint 2.9.6 pyls-spyder 0.4.0 PyNaCl 1.4.0 pyodbc 4.0.0-unsupported pyOpenSSL 21.0.0 pyparsing 3.0.4 pyreadline 2.1 pyrsistent 0.18.0 PySocks 1.7.1 pytest 6.2.4 python-binance 1.0.15 python-dateutil 2.8.2 python-lsp-black 1.0.0 python-lsp-jsonrpc 1.0.0 python-lsp-server 1.2.4 python-slugify 5.0.2 pytz 2021.3 pytz-deprecation-shim 0.1.0.post0 PyWavelets 1.1.1 pywin32 228 pywin32-ctypes 0.2.0 pywinpty 0.5.7 PyYAML 6.0 pyzmq 22.2.1 QDarkStyle 3.0.2 qstylizer 0.1.10 QtAwesome 1.0.2 qtconsole 5.1.1 QtPy 1.10.0 regex 2021.8.3 requests 2.26.0 rope 0.19.0 Rtree 0.9.7 ruamel-yaml-conda 0.15.100 scikit-image 0.18.3 scikit-learn 0.24.2 scikit-learn-intelex 2021.20210714.120553 scipy 1.7.1 seaborn 0.11.2 Send2Trash 1.8.0 service-identity 21.1.0 setuptools 58.0.4 simplegeneric 0.8.1 singledispatch 3.7.0 sip 4.19.13 six 1.16.0 sniffio 1.2.0 snowballstemmer 2.1.0 sortedcollections 2.1.0 sortedcontainers 2.4.0 soupsieve 2.2.1 Sphinx 4.2.0 sphinxcontrib-applehelp 1.0.2 sphinxcontrib-devhelp 1.0.2 sphinxcontrib-htmlhelp 2.0.0 sphinxcontrib-jsmath 1.0.1 sphinxcontrib-qthelp 1.0.3 sphinxcontrib-serializinghtml 1.1.5 sphinxcontrib-websupport 1.2.4 spyder 5.1.5 spyder-kernels 2.1.3 SQLAlchemy 1.4.22 statsmodels 0.12.2 sympy 1.9 ta 0.9.0 tables 3.6.1 TBB 0.2 tblib 1.7.0 terminado 0.9.4 testpath 0.5.0 text-unidecode 1.3 textdistance 4.2.1 threadpoolctl 2.2.0 three-merge 0.1.1 tifffile 2021.7.2 tinycss 0.4 toml 0.10.2 toolz 0.11.1 tornado 6.1 tqdm 4.62.3 traitlets 5.1.0 typed-ast 1.4.3 typing-extensions 3.10.0.2 tzdata 2021.5 tzlocal 4.1 ujson 4.0.2 unicodecsv 0.14.1 unicorn-binance-rest-api 1.3.0 unicorn-binance-suite 0.0.0 unicorn-binance-websocket-api 1.36.0 unicorn-fy 0.11.0 Unidecode 1.2.0 urllib3 1.26.7 watchdog 2.1.3 wcwidth 0.2.5 webencodings 0.5.1 websocket-client 1.2.3 websockets 10.1 Werkzeug 2.0.2 wheel 0.37.0 whichcraft 0.6.1 widgetsnbextension 3.5.1 win-inet-pton 1.1.0 win-unicode-console 0.5 wincertstore 0.2 wrapt 1.12.1 xlrd 2.0.1 XlsxWriter 3.0.1 xlwings 0.24.9 xlwt 1.3.0 xmltodict 0.12.0 yapf 0.31.0 yarl 1.7.2 zict 2.0.0 zipp 3.6.0 zope.event 4.5.0 zope.interface 5.4.0

[oliver-zehentleitner]

@chomparoja this is the abolute wrong place for a new issue :)

your issue is related to: https://github.com/LUCIT-Systems-and-Development/unicorn-binance-websocket-api/discussions/231#discussioncomment-2046677

In the future please open new issues and dont use cold closed issues for different topics