code = 1006 when using too many kline streams

[Flowelfox]

Check this or we will delete your issue. (fill in the checkbox with an X like so: [x])

• [x] I have searched for other issues with the same problem or similar feature requests.

Select one:

• [x] Bug
• [ ] Feature Request
• [ ] Technical Help
• [ ] Other

Environment

• [x] Are you using the module on a VPS or other Cloud hosting?
• [x] If so, have you tried running the code on your local machine?
• [ ] Are you using the module on a Raspberry Pi?

Operating System? (include version)

• [ ] macOS
• [ ] Windows
• [x] Linux (include flavour)

Python Version Requirement

• [x] I am using Python 3.6.1 or above

Exact Python Version?

Include here the response of 'python --version' or 'python3 --version'
Python 3.6.9

Pip Version?

Include here the response of 'pip --version' or 'pip3 --version'
pip 19.0.3

Dependencies

Run pip list > pip_list.txt and upload the file.

Package                       Version     Location                                   
----------------------------- ----------- -------------------------------------------
alembic                       1.4.2       
aniso8601                     8.0.0       
APScheduler                   3.6.3       
attrs                         19.3.0      
autobahn                      20.6.1      
Automat                       20.2.0      
certifi                       2020.6.20   
cffi                          1.14.0      
chardet                       3.0.4       
cheroot                       8.4.4       
click                         7.1.2       
colorama                      0.4.3       
constantly                    15.1.0      
cryptography                  2.9.2       
dateparser                    0.7.6       
decorator                     4.4.2       
Flask                         1.1.2       
Flask-RESTful                 0.3.8       
flask-restx                   0.2.0       
Flask-SQLAlchemy              2.4.3       
hyperlink                     19.0.0      
idna                          2.10        
importlib-metadata            1.6.1       
incremental                   17.5.0      
inprofit                      0.1         
inprofitapi                   0.1         /home/flowelcat/PycharmProjects/inprofitapi
itsdangerous                  1.1.0       
jaraco.functools              3.0.1       
Jinja2                        2.11.2      
jsonschema                    3.2.0       
Mako                          1.1.3       
MarkupSafe                    1.1.1       
more-itertools                8.4.0       
pathlib                       1.0.1       
pip                           19.0.3      
psutil                        5.7.2       
psycopg2-binary               2.8.5       
public                        2019.4.13   
pyasn1                        0.4.8       
pyasn1-modules                0.2.8       
pycparser                     2.20        
PyHamcrest                    2.0.2       
pyOpenSSL                     19.1.0      
pyrsistent                    0.16.0      
python-binance                0.7.5       
python-dateutil               2.8.1       
python-editor                 1.0.4       
pytz                          2020.1      
regex                         2020.6.8    
requests                      2.24.0      
self                          2019.4.13   
service-identity              18.1.0      
setuptools                    47.3.1      
six                           1.15.0      
SQLAlchemy                    1.3.18      
Twisted                       20.3.0      
txaio                         20.4.1      
tzlocal                       2.1         
ujson                         3.1.0       
unicorn-binance-websocket-api 1.16.7.dev0 
unicorn-fy                    0.4.1       
urllib3                       1.25.10     
websocket-client              0.57.0      
websockets                    8.1         
Werkzeug                      1.0.1       
zipp                          3.1.0       
zope.interface                5.1.0  

UNICORN Binance WebSocket API Version?

Version 1.16.7.dev0 from my fork, with fixed "error 2" bug

Description of your issue

I am using 24 pairs and 12 kline stream for each pair. This all is starting in one BinanceWebSocketApiManager. After start i constantly start getting critical errors "code = 1006 (connection closed abnormally [internal]), no reason" ~every second.

If i start 3 BinanceWebSocketApiManager with pairs grouped as 10, 10 and 4 i am getting same error but much less often, one in like 5-10 minutes.

This started on 02.09 morning from version 1.16.4dev and continues now. Also i noticed that after crash only one or two specific streams starting, not all of them.

SYMBOLS = [
    "BTCUSDT",
    "ETHUSDT",
    "BCHUSDT",
    "XRPUSDT",
    "EOSUSDT",
    "LTCUSDT",
    "TRXUSDT",
    "ETCUSDT",
    "LINKUSDT",
    "XLMUSDT",
    "ADAUSDT",
    "XMRUSDT",
    "DASHUSDT",
    "ZECUSDT",
    "XTZUSDT",
    "BNBUSDT",
    "ATOMUSDT",
    "ONTUSDT",
    "IOTAUSDT",
    "BATUSDT",
    "VETUSDT",
    "NEOUSDT",
    "QTUMUSDT",
    "IOSTUSDT"
]
streams = ["kline_1m", 'kline_3m', 'kline_5m', 'kline_15m', 'kline_30m', "kline_1h", "kline_2h", "kline_4h", "kline_6h", "kline_8h", "kline_12h", "kline_1d"]

Logs without groups, 24 pairs, 1 manager:

2020-09-03 16:01:57,536 - root - INFO - BinanceWebSocketApiSocket->start_socket(6d20d67a-1639-43d0-a733-d255f6bcdcba) Received result message: {"id":1,"result":null}
2020-09-03 16:01:59,044 - root - CRITICAL - BinanceWebSocketApiSocket->start_socket(42e2c52c-3337-4c2a-a8a0-51a6c997db5b, ['kline_1m', 'kline_3m', 'kline_5m', 'kline_15m', 'kline_30m', 'kline_1h', 'kline_2h', 'kline_4h', 'kline_6h', 'kline_8h', 'kline_12h', 'kline_1d'], ['btcusdt', 'ethusdt', 'bchusdt', 'xrpusdt', 'eosusdt', 'ltcusdt', 'trxusdt', 'etcusdt', 'linkusdt', 'xlmusdt', 'adausdt', 'xmrusdt', 'dashusdt', 'zecusdt', 'xtzusdt', 'bnbusdt', 'atomusdt', 'ontusdt', 'iotausdt', 'batusdt', 'vetusdt', 'neousdt', 'qtumusdt', 'iostusdt']) Exception ConnectionClosed Info: code = 1006 (connection closed abnormally [internal]), no reason
2020-09-03 16:01:59,878 - root - INFO - BinanceWebSocketApiSocket->start_socket(42e2c52c-3337-4c2a-a8a0-51a6c997db5b, ['kline_6h', 'kline_2h', 'kline_12h', 'kline_8h', 'kline_5m', 'kline_1m', 'kline_3m', 'kline_1d', 'kline_30m', 'kline_1h', 'kline_15m', 'kline_4h'], ['xlmusdt', 'btcusdt', 'ontusdt', 'neousdt', 'iotausdt', 'bchusdt', 'eosusdt', 'qtumusdt', 'adausdt', 'zecusdt', 'ethusdt', 'trxusdt', 'xtzusdt', 'vetusdt', 'bnbusdt', 'xmrusdt', 'linkusdt', 'xrpusdt', 'atomusdt', 'iostusdt', 'batusdt', 'dashusdt', 'etcusdt', 'ltcusdt'])
2020-09-03 16:01:59,879 - root - INFO - Created websocket URI for stream_id=42e2c52c-3337-4c2a-a8a0-51a6c997db5b is wss://fstream.binance.com/stream?streams=ltcusdt@kline_4h
2020-09-03 16:02:01,115 - root - CRITICAL - BinanceWebSocketApiSocket->start_socket(42e2c52c-3337-4c2a-a8a0-51a6c997db5b, ['kline_6h', 'kline_2h', 'kline_12h', 'kline_8h', 'kline_5m', 'kline_1m', 'kline_3m', 'kline_1d', 'kline_30m', 'kline_1h', 'kline_15m', 'kline_4h'], ['xlmusdt', 'btcusdt', 'ontusdt', 'neousdt', 'iotausdt', 'bchusdt', 'eosusdt', 'qtumusdt', 'adausdt', 'zecusdt', 'ethusdt', 'trxusdt', 'xtzusdt', 'vetusdt', 'bnbusdt', 'xmrusdt', 'linkusdt', 'xrpusdt', 'atomusdt', 'iostusdt', 'batusdt', 'dashusdt', 'etcusdt', 'ltcusdt']) Exception ConnectionClosed Info: code = 1006 (connection closed abnormally [internal]), no reason
2020-09-03 16:02:01,882 - root - INFO - BinanceWebSocketApiSocket->start_socket(42e2c52c-3337-4c2a-a8a0-51a6c997db5b, ['kline_6h', 'kline_2h', 'kline_12h', 'kline_8h', 'kline_5m', 'kline_1m', 'kline_3m', 'kline_1d', 'kline_30m', 'kline_1h', 'kline_15m', 'kline_4h'], ['xlmusdt', 'btcusdt', 'ontusdt', 'neousdt', 'iotausdt', 'bchusdt', 'eosusdt', 'qtumusdt', 'adausdt', 'zecusdt', 'ethusdt', 'trxusdt', 'xtzusdt', 'vetusdt', 'bnbusdt', 'xmrusdt', 'linkusdt', 'xrpusdt', 'atomusdt', 'iostusdt', 'batusdt', 'dashusdt', 'etcusdt', 'ltcusdt'])
2020-09-03 16:02:01,885 - root - INFO - Created websocket URI for stream_id=42e2c52c-3337-4c2a-a8a0-51a6c997db5b is wss://fstream.binance.com/stream?streams=ltcusdt@kline_4h
2020-09-03 16:02:03,473 - root - CRITICAL - BinanceWebSocketApiSocket->start_socket(42e2c52c-3337-4c2a-a8a0-51a6c997db5b, ['kline_6h', 'kline_2h', 'kline_12h', 'kline_8h', 'kline_5m', 'kline_1m', 'kline_3m', 'kline_1d', 'kline_30m', 'kline_1h', 'kline_15m', 'kline_4h'], ['xlmusdt', 'btcusdt', 'ontusdt', 'neousdt', 'iotausdt', 'bchusdt', 'eosusdt', 'qtumusdt', 'adausdt', 'zecusdt', 'ethusdt', 'trxusdt', 'xtzusdt', 'vetusdt', 'bnbusdt', 'xmrusdt', 'linkusdt', 'xrpusdt', 'atomusdt', 'iostusdt', 'batusdt', 'dashusdt', 'etcusdt', 'ltcusdt']) Exception ConnectionClosed Info: code = 1006 (connection closed abnormally [internal]), no reason
2020-09-03 16:02:03,884 - root - INFO - BinanceWebSocketApiSocket->start_socket(42e2c52c-3337-4c2a-a8a0-51a6c997db5b, ['kline_6h', 'kline_2h', 'kline_12h', 'kline_8h', 'kline_5m', 'kline_1m', 'kline_3m', 'kline_1d', 'kline_30m', 'kline_1h', 'kline_15m', 'kline_4h'], ['xlmusdt', 'btcusdt', 'ontusdt', 'neousdt', 'iotausdt', 'bchusdt', 'eosusdt', 'qtumusdt', 'adausdt', 'zecusdt', 'ethusdt', 'trxusdt', 'xtzusdt', 'vetusdt', 'bnbusdt', 'xmrusdt', 'linkusdt', 'xrpusdt', 'atomusdt', 'iostusdt', 'batusdt', 'dashusdt', 'etcusdt', 'ltcusdt'])
2020-09-03 16:02:03,886 - root - INFO - Created websocket URI for stream_id=42e2c52c-3337-4c2a-a8a0-51a6c997db5b is wss://fstream.binance.com/stream?streams=ltcusdt@kline_4h
2020-09-03 16:02:06,171 - root - CRITICAL - BinanceWebSocketApiSocket->start_socket(42e2c52c-3337-4c2a-a8a0-51a6c997db5b, ['kline_6h', 'kline_2h', 'kline_12h', 'kline_8h', 'kline_5m', 'kline_1m', 'kline_3m', 'kline_1d', 'kline_30m', 'kline_1h', 'kline_15m', 'kline_4h'], ['xlmusdt', 'btcusdt', 'ontusdt', 'neousdt', 'iotausdt', 'bchusdt', 'eosusdt', 'qtumusdt', 'adausdt', 'zecusdt', 'ethusdt', 'trxusdt', 'xtzusdt', 'vetusdt', 'bnbusdt', 'xmrusdt', 'linkusdt', 'xrpusdt', 'atomusdt', 'iostusdt', 'batusdt', 'dashusdt', 'etcusdt', 'ltcusdt']) Exception ConnectionClosed Info: code = 1006 (connection closed abnormally [internal]), no reason
2020-09-03 16:02:06,889 - root - INFO - BinanceWebSocketApiSocket->start_socket(42e2c52c-3337-4c2a-a8a0-51a6c997db5b, ['kline_6h', 'kline_2h', 'kline_12h', 'kline_8h', 'kline_5m', 'kline_1m', 'kline_3m', 'kline_1d', 'kline_30m', 'kline_1h', 'kline_15m', 'kline_4h'], ['xlmusdt', 'btcusdt', 'ontusdt', 'neousdt', 'iotausdt', 'bchusdt', 'eosusdt', 'qtumusdt', 'adausdt', 'zecusdt', 'ethusdt', 'trxusdt', 'xtzusdt', 'vetusdt', 'bnbusdt', 'xmrusdt', 'linkusdt', 'xrpusdt', 'atomusdt', 'iostusdt', 'batusdt', 'dashusdt', 'etcusdt', 'ltcusdt'])
2020-09-03 16:02:06,891 - root - INFO - Created websocket URI for stream_id=42e2c52c-3337-4c2a-a8a0-51a6c997db5b is wss://fstream.binance.com/stream?streams=ltcusdt@kline_4h
2020-09-03 16:02:09,195 - root - CRITICAL - BinanceWebSocketApiSocket->start_socket(42e2c52c-3337-4c2a-a8a0-51a6c997db5b, ['kline_6h', 'kline_2h', 'kline_12h', 'kline_8h', 'kline_5m', 'kline_1m', 'kline_3m', 'kline_1d', 'kline_30m', 'kline_1h', 'kline_15m', 'kline_4h'], ['xlmusdt', 'btcusdt', 'ontusdt', 'neousdt', 'iotausdt', 'bchusdt', 'eosusdt', 'qtumusdt', 'adausdt', 'zecusdt', 'ethusdt', 'trxusdt', 'xtzusdt', 'vetusdt', 'bnbusdt', 'xmrusdt', 'linkusdt', 'xrpusdt', 'atomusdt', 'iostusdt', 'batusdt', 'dashusdt', 'etcusdt', 'ltcusdt']) Exception ConnectionClosed Info: code = 1006 (connection closed abnormally [internal]), no reason
2020-09-03 16:02:09,894 - root - INFO - BinanceWebSocketApiSocket->start_socket(42e2c52c-3337-4c2a-a8a0-51a6c997db5b, ['kline_6h', 'kline_2h', 'kline_12h', 'kline_8h', 'kline_5m', 'kline_1m', 'kline_3m', 'kline_1d', 'kline_30m', 'kline_1h', 'kline_15m', 'kline_4h'], ['xlmusdt', 'btcusdt', 'ontusdt', 'neousdt', 'iotausdt', 'bchusdt', 'eosusdt', 'qtumusdt', 'adausdt', 'zecusdt', 'ethusdt', 'trxusdt', 'xtzusdt', 'vetusdt', 'bnbusdt', 'xmrusdt', 'linkusdt', 'xrpusdt', 'atomusdt', 'iostusdt', 'batusdt', 'dashusdt', 'etcusdt', 'ltcusdt'])
2020-09-03 16:02:09,896 - root - INFO - Created websocket URI for stream_id=42e2c52c-3337-4c2a-a8a0-51a6c997db5b is wss://fstream.binance.com/stream?streams=ltcusdt@kline_4h
2020-09-03 16:02:11,876 - root - CRITICAL - BinanceWebSocketApiSocket->start_socket(42e2c52c-3337-4c2a-a8a0-51a6c997db5b, ['kline_6h', 'kline_2h', 'kline_12h', 'kline_8h', 'kline_5m', 'kline_1m', 'kline_3m', 'kline_1d', 'kline_30m', 'kline_1h', 'kline_15m', 'kline_4h'], ['xlmusdt', 'btcusdt', 'ontusdt', 'neousdt', 'iotausdt', 'bchusdt', 'eosusdt', 'qtumusdt', 'adausdt', 'zecusdt', 'ethusdt', 'trxusdt', 'xtzusdt', 'vetusdt', 'bnbusdt', 'xmrusdt', 'linkusdt', 'xrpusdt', 'atomusdt', 'iostusdt', 'batusdt', 'dashusdt', 'etcusdt', 'ltcusdt']) Exception ConnectionClosed Info: code = 1006 (connection closed abnormally [internal]), no reason
2020-09-03 16:02:11,900 - root - INFO - BinanceWebSocketApiSocket->start_socket(42e2c52c-3337-4c2a-a8a0-51a6c997db5b, ['kline_6h', 'kline_2h', 'kline_12h', 'kline_8h', 'kline_5m', 'kline_1m', 'kline_3m', 'kline_1d', 'kline_30m', 'kline_1h', 'kline_15m', 'kline_4h'], ['xlmusdt', 'btcusdt', 'ontusdt', 'neousdt', 'iotausdt', 'bchusdt', 'eosusdt', 'qtumusdt', 'adausdt', 'zecusdt', 'ethusdt', 'trxusdt', 'xtzusdt', 'vetusdt', 'bnbusdt', 'xmrusdt', 'linkusdt', 'xrpusdt', 'atomusdt', 'iostusdt', 'batusdt', 'dashusdt', 'etcusdt', 'ltcusdt'])
2020-09-03 16:02:11,902 - root - INFO - Created websocket URI for stream_id=42e2c52c-3337-4c2a-a8a0-51a6c997db5b is wss://fstream.binance.com/stream?streams=ltcusdt@kline_4h
2020-09-03 16:02:13,668 - root - CRITICAL - BinanceWebSocketApiSocket->start_socket(42e2c52c-3337-4c2a-a8a0-51a6c997db5b, ['kline_6h', 'kline_2h', 'kline_12h', 'kline_8h', 'kline_5m', 'kline_1m', 'kline_3m', 'kline_1d', 'kline_30m', 'kline_1h', 'kline_15m', 'kline_4h'], ['xlmusdt', 'btcusdt', 'ontusdt', 'neousdt', 'iotausdt', 'bchusdt', 'eosusdt', 'qtumusdt', 'adausdt', 'zecusdt', 'ethusdt', 'trxusdt', 'xtzusdt', 'vetusdt', 'bnbusdt', 'xmrusdt', 'linkusdt', 'xrpusdt', 'atomusdt', 'iostusdt', 'batusdt', 'dashusdt', 'etcusdt', 'ltcusdt']) Exception ConnectionClosed Info: code = 1006 (connection closed abnormally [internal]), no reason
2020-09-03 16:02:13,905 - root - INFO - BinanceWebSocketApiSocket->start_socket(42e2c52c-3337-4c2a-a8a0-51a6c997db5b, ['kline_6h', 'kline_2h', 'kline_12h', 'kline_8h', 'kline_5m', 'kline_1m', 'kline_3m', 'kline_1d', 'kline_30m', 'kline_1h', 'kline_15m', 'kline_4h'], ['xlmusdt', 'btcusdt', 'ontusdt', 'neousdt', 'iotausdt', 'bchusdt', 'eosusdt', 'qtumusdt', 'adausdt', 'zecusdt', 'ethusdt', 'trxusdt', 'xtzusdt', 'vetusdt', 'bnbusdt', 'xmrusdt', 'linkusdt', 'xrpusdt', 'atomusdt', 'iostusdt', 'batusdt', 'dashusdt', 'etcusdt', 'ltcusdt'])
2020-09-03 16:02:13,907 - root - INFO - Created websocket URI for stream_id=42e2c52c-3337-4c2a-a8a0-51a6c997db5b is wss://fstream.binance.com/stream?streams=ltcusdt@kline_4h
2020-09-03 16:02:16,594 - root - CRITICAL - BinanceWebSocketApiSocket->start_socket(42e2c52c-3337-4c2a-a8a0-51a6c997db5b, ['kline_6h', 'kline_2h', 'kline_12h', 'kline_8h', 'kline_5m', 'kline_1m', 'kline_3m', 'kline_1d', 'kline_30m', 'kline_1h', 'kline_15m', 'kline_4h'], ['xlmusdt', 'btcusdt', 'ontusdt', 'neousdt', 'iotausdt', 'bchusdt', 'eosusdt', 'qtumusdt', 'adausdt', 'zecusdt', 'ethusdt', 'trxusdt', 'xtzusdt', 'vetusdt', 'bnbusdt', 'xmrusdt', 'linkusdt', 'xrpusdt', 'atomusdt', 'iostusdt', 'batusdt', 'dashusdt', 'etcusdt', 'ltcusdt']) Exception ConnectionClosed Info: code = 1006 (connection closed abnormally [internal]), no reason
2020-09-03 16:02:16,911 - root - INFO - BinanceWebSocketApiSocket->start_socket(42e2c52c-3337-4c2a-a8a0-51a6c997db5b, ['kline_6h', 'kline_2h', 'kline_12h', 'kline_8h', 'kline_5m', 'kline_1m', 'kline_3m', 'kline_1d', 'kline_30m', 'kline_1h', 'kline_15m', 'kline_4h'], ['xlmusdt', 'btcusdt', 'ontusdt', 'neousdt', 'iotausdt', 'bchusdt', 'eosusdt', 'qtumusdt', 'adausdt', 'zecusdt', 'ethusdt', 'trxusdt', 'xtzusdt', 'vetusdt', 'bnbusdt', 'xmrusdt', 'linkusdt', 'xrpusdt', 'atomusdt', 'iostusdt', 'batusdt', 'dashusdt', 'etcusdt', 'ltcusdt'])
2020-09-03 16:02:16,913 - root - INFO - Created websocket URI for stream_id=42e2c52c-3337-4c2a-a8a0-51a6c997db5b is wss://fstream.binance.com/stream?streams=ltcusdt@kline_4h
2020-09-03 16:02:19,522 - root - CRITICAL - BinanceWebSocketApiSocket->start_socket(42e2c52c-3337-4c2a-a8a0-51a6c997db5b, ['kline_6h', 'kline_2h', 'kline_12h', 'kline_8h', 'kline_5m', 'kline_1m', 'kline_3m', 'kline_1d', 'kline_30m', 'kline_1h', 'kline_15m', 'kline_4h'], ['xlmusdt', 'btcusdt', 'ontusdt', 'neousdt', 'iotausdt', 'bchusdt', 'eosusdt', 'qtumusdt', 'adausdt', 'zecusdt', 'ethusdt', 'trxusdt', 'xtzusdt', 'vetusdt', 'bnbusdt', 'xmrusdt', 'linkusdt', 'xrpusdt', 'atomusdt', 'iostusdt', 'batusdt', 'dashusdt', 'etcusdt', 'ltcusdt']) Exception ConnectionClosed Info: code = 1006 (connection closed abnormally [internal]), no reason
2020-09-03 16:02:19,915 - root - INFO - BinanceWebSocketApiSocket->start_socket(42e2c52c-3337-4c2a-a8a0-51a6c997db5b, ['kline_6h', 'kline_2h', 'kline_12h', 'kline_8h', 'kline_5m', 'kline_1m', 'kline_3m', 'kline_1d', 'kline_30m', 'kline_1h', 'kline_15m', 'kline_4h'], ['xlmusdt', 'btcusdt', 'ontusdt', 'neousdt', 'iotausdt', 'bchusdt', 'eosusdt', 'qtumusdt', 'adausdt', 'zecusdt', 'ethusdt', 'trxusdt', 'xtzusdt', 'vetusdt', 'bnbusdt', 'xmrusdt', 'linkusdt', 'xrpusdt', 'atomusdt', 'iostusdt', 'batusdt', 'dashusdt', 'etcusdt', 'ltcusdt'])
2020-09-03 16:02:19,916 - root - INFO - Created websocket URI for stream_id=42e2c52c-3337-4c2a-a8a0-51a6c997db5b is wss://fstream.binance.com/stream?streams=ltcusdt@kline_4h

Logs with grouped symbols, 6 managers, 5 per manager:

2020-09-03 15:42:48,827 - root - CRITICAL - BinanceWebSocketApiSocket->start_socket(ae98456d-4179-42e6-af39-54db84c98524, ['kline_1m', 'kline_3m', 'kline_5m', 'kline_15m', 'kline_30m', 'kline_1h', 'kline_2h', 'kline_4h', 'kline_6h', 'kline_8h', 'kline_12h', 'kline_1d'], ['btcusdt', 'ethusdt', 'bchusdt', 'xrpusdt', 'eosusdt']) Exception ConnectionClosed Info: code = 1006 (connection closed abnormally [internal]), no reason
2020-09-03 15:42:49,188 - root - INFO - BinanceWebSocketApiSocket->start_socket(ae98456d-4179-42e6-af39-54db84c98524, ['kline_5m', 'kline_3m', 'kline_30m', 'kline_1d', 'kline_15m', 'kline_12h', 'kline_8h', 'kline_1h', 'kline_4h', 'kline_2h', 'kline_1m', 'kline_6h'], ['bchusdt', 'eosusdt', 'ethusdt', 'xrpusdt', 'btcusdt'])
2020-09-03 15:42:49,189 - root - INFO - Created websocket URI for stream_id=ae98456d-4179-42e6-af39-54db84c98524 is wss://fstream.binance.com/stream?streams=btcusdt@kline_6h
2020-09-03 15:42:51,265 - root - INFO - BinanceWebSocketApiSocket->start_socket(ae98456d-4179-42e6-af39-54db84c98524) Received result message: {"id":2,"result":null}
2020-09-03 15:46:00,504 - root - CRITICAL - BinanceWebSocketApiSocket->start_socket(ce276798-a1cb-4c41-85c6-103694ef9d9c, ['kline_1m', 'kline_3m', 'kline_5m', 'kline_15m', 'kline_30m', 'kline_1h', 'kline_2h', 'kline_4h', 'kline_6h', 'kline_8h', 'kline_12h', 'kline_1d'], ['vetusdt', 'neousdt', 'qtumusdt', 'iostusdt']) Exception ConnectionClosed Info: code = 1006 (connection closed abnormally [internal]), no reason
2020-09-03 15:46:01,238 - root - INFO - BinanceWebSocketApiSocket->start_socket(ce276798-a1cb-4c41-85c6-103694ef9d9c, ['kline_5m', 'kline_3m', 'kline_30m', 'kline_1d', 'kline_15m', 'kline_12h', 'kline_8h', 'kline_1h', 'kline_4h', 'kline_2h', 'kline_1m', 'kline_6h'], ['vetusdt', 'qtumusdt', 'neousdt', 'iostusdt'])
2020-09-03 15:46:01,240 - root - INFO - Created websocket URI for stream_id=ce276798-a1cb-4c41-85c6-103694ef9d9c is wss://fstream.binance.com/stream?streams=iostusdt@kline_6h
2020-09-03 15:46:04,050 - root - INFO - BinanceWebSocketApiSocket->start_socket(ce276798-a1cb-4c41-85c6-103694ef9d9c) Received result message: {"id":2,"result":null}
2020-09-03 15:46:30,817 - root - CRITICAL - BinanceWebSocketApiSocket->start_socket(2d684ec2-ff76-4f3c-924b-36340fb442a6, ['kline_5m', 'kline_3m', 'kline_30m', 'kline_1d', 'kline_15m', 'kline_12h', 'kline_8h', 'kline_1h', 'kline_4h', 'kline_2h', 'kline_1m', 'kline_6h'], ['dashusdt', 'xmrusdt', 'zecusdt', 'xtzusdt', 'adausdt']) Exception ConnectionClosed Info: code = 1006 (connection closed abnormally [internal]), no reason
2020-09-03 15:46:30,883 - root - INFO - BinanceWebSocketApiSocket->start_socket(2d684ec2-ff76-4f3c-924b-36340fb442a6, ['kline_5m', 'kline_3m', 'kline_30m', 'kline_1d', 'kline_15m', 'kline_12h', 'kline_8h', 'kline_1h', 'kline_4h', 'kline_2h', 'kline_1m', 'kline_6h'], ['dashusdt', 'xmrusdt', 'zecusdt', 'xtzusdt', 'adausdt'])
2020-09-03 15:46:30,884 - root - INFO - Created websocket URI for stream_id=2d684ec2-ff76-4f3c-924b-36340fb442a6 is wss://fstream.binance.com/stream?streams=adausdt@kline_6h
2020-09-03 15:46:32,142 - root - INFO - BinanceWebSocketApiSocket->start_socket(2d684ec2-ff76-4f3c-924b-36340fb442a6) Received result message: {"id":3,"result":null}
2020-09-03 15:52:14,880 - root - CRITICAL - BinanceWebSocketApiSocket->start_socket(310031cf-b6df-4749-b8a6-3b9a9453805d, ['arr'], ['!userData']) Exception ConnectionClosed Info: code = 1006 (connection closed abnormally [internal]), no reason
2020-09-03 15:52:15,403 - root - INFO - BinanceWebSocketApiSocket->start_socket(310031cf-b6df-4749-b8a6-3b9a9453805d, ['arr'], ['!userData'])

[oliver-zehentleitner]

My anwser is not cool: I know, there is a bug inside, but I cant trace it till now. I dont know what and why it happens, but in my tests i often end up with active streams which are eceiving nothing. If i set a restart request the original thread does not get closed, but a new extra one is created. so there are zombi threads after a while and the lib cant handle it atm - i think i would need a couple of days for testing and diving into the code, but i am too busy atm to get that time.

code = 1006 is nothing i can really control. I think, its something happening on the network or there are to less local resources (cpu/ram) and also sometimes binance seems to be very unstable. the same code runs 2 weeks with zero restart and then i have days with a lot of restarts per day.

I want to crate something that can proof if a thread and its stream is really working, if not, kill and restart.

Maybe logging needs to get a clever update too, because atm the logs dont help me to find the reason for dead streams.

[oliver-zehentleitner]

how much cpu and ram does you plattform have? how much % of the system are you using in average?

[Flowelfox]

My local machine has 32GB or ram and 6 core, 12 threads CPU. I am using only 10-20% of my cpu.

On VPS there is only 1 core, 1 thread and 2GB of ram. Full application + web site using 50% of CPU in avarege and ~600MB of memory.

[oliver-zehentleitner]

Did you read the link i posted about 1006 in the other thread?

1006 is something thats happening outside of the lib. the lib is getting this signal from the OS, the reason is that the connection is not established anymore... for any reason. This can happen through physical line probs, problems on binance servers, problems with your local resources, what ever...

Its really hard for me to find the specific reason and I think we have to deal with that. If you have a better idea lets talk about it.

First I loved the idea of one mega multiplex connection all pairs and all stream types, but then i recognized that such channels are very unstable. the less channel types I mixed, the more stable they were. I dont know the reason.

Also if I had to less CPU on my server I had a lot of side effects...

Please take a look at https://github.com/oliver-zehentleitner/unicorn-binance-websocket-api/blob/master/example_stream_everything.py - here i create all kind of kline streams with all available pairs - in separate streams. the most stable way i found out. sometimes 14 without a single restart on my VPS.

What does happen if you run that script on your server where you have the problems?

[oliver-zehentleitner]

This script is running on my demo server: https://ubwa-demo.lucit.co/print_summary.php

[oliver-zehentleitner]

Can we close this?

[Flowelfox]

Yes, but issue still exist

[oliver-zehentleitner]

Did you read the link i posted about 1006 in the other thread?

1006 is something thats happening outside of the lib. the lib is getting this signal from the OS, the reason is that the connection is not established anymore... for any reason. This can happen through physical line probs, problems on binance servers, problems with your local resources, what ever...

Its really hard for me to find the specific reason and I think we have to deal with that. If you have a better idea lets talk about it.

First I loved the idea of one mega multiplex connection all pairs and all stream types, but then i recognized that such channels are very unstable. the less channel types I mixed, the more stable they were. I dont know the reason.

Also if I had to less CPU on my server I had a lot of side effects...

Please take a look at https://github.com/oliver-zehentleitner/unicorn-binance-websocket-api/blob/master/example_stream_everything.py - here i create all kind of kline streams with all available pairs - in separate streams. the most stable way i found out. sometimes 14 without a single restart on my VPS.

What does happen if you run that script on your server where you have the problems?

What are your thoughts about that?

Please try an other setup: Only one channel like kline_1m and then all symbolpairs (but less than 1024 per stream)

create_stream('kline_1m', ['btcusdt', 'ethusdt', 'bchusdt', 'xrpusdt', 'eosusdt', 'dashusdt', 'xmrusdt', 'zecusdt', 'xtzusdt', 'adausdt'])
create_stream('kline_3m', ['btcusdt', 'ethusdt', 'bchusdt', 'xrpusdt', 'eosusdt', 'dashusdt', 'xmrusdt', 'zecusdt', 'xtzusdt', 'adausdt'])
create_stream('kline_5m', ['btcusdt', 'ethusdt', 'bchusdt', 'xrpusdt', 'eosusdt', 'dashusdt', 'xmrusdt', 'zecusdt', 'xtzusdt', 'adausdt'])
create_stream('kline_1h', ['btcusdt', 'ethusdt', 'bchusdt', 'xrpusdt', 'eosusdt', 'dashusdt', 'xmrusdt', 'zecusdt', 'xtzusdt', 'adausdt'])
create_stream('kline_1d', ['btcusdt', 'ethusdt', 'bchusdt', 'xrpusdt', 'eosusdt', 'dashusdt', 'xmrusdt', 'zecusdt', 'xtzusdt', 'adausdt'])

[oliver-zehentleitner]

https://github.com/oliver-zehentleitner/unicorn-binance-websocket-api/issues/113

[oliver-zehentleitner]

I tested with a few different VPS and settings.

1006 after too many kline streams... in my tests the problem is always the CPU power. when i scale it up, then the streams are getting stable....