Closed Flowelfox closed 3 years ago
I found that in Python 3.8.0 and above InvalidStateError does not exists:
(.venv3.8) flowelcat@Flowelcat-PC:~/PycharmProjects/inprofitapi$ python test.py
Traceback (most recent call last):
File "test.py", line 1, in <module>
from asyncio.futures import InvalidStateError
ImportError: cannot import name 'InvalidStateError' from 'asyncio.futures' (/usr/lib/python3.8/asyncio/futures.py)
(.venv3.8) flowelcat@Flowelcat-PC:~/PycharmProjects/inprofitapi$ python --version
Python 3.8.3
Thanks for the good report!
I added a restart handling for this error, please can you test it?
pip install https://github.com/oliver-zehentleitner/unicorn-binance-websocket-api/tarball/master --upgrade
Please proof that you are on 1.16.1.dev!
I am not shure if the code I added is really good, maybe we have to filter out some other attribute errors... lets test it :)
Best regards, Oliver
Hi, updated library and restarted application recently.
inprofitapi | 2020-06-09 20:04:38,817 - root - INFO - Initializing flask application...
inprofitapi | 2020-06-09 20:04:38,891 - root - INFO - Setting up kline subscription...
inprofitapi | 2020-06-09 20:04:38,892 - root - INFO - New instance of unicorn_binance_websocket_api_manager 1.16.1 started ...
inprofitapi | 2020-06-09 20:04:39,031 - root - INFO - Starting missing candles loader...
inprofitapi | 2020-06-09 20:04:39,036 - root - INFO - Setting up mark price subscription...
inprofitapi | 2020-06-09 20:04:39,038 - root - INFO - New instance of unicorn_binance_websocket_api_manager 1.16.1 started ...
inprofitapi | 2020-06-09 20:04:39,183 - root - INFO - Setting up mini ticker subscription...
inprofitapi | 2020-06-09 20:04:39,184 - root - INFO - New instance of unicorn_binance_websocket_api_manager 1.16.1 started ...
inprofitapi | 2020-06-09 20:04:39,284 - root - INFO - Setting up user data subscription...
inprofitapi | 2020-06-09 20:04:39,300 - root - INFO - New instance of unicorn_binance_websocket_api_manager 1.16.1 started ...
inprofitapi | 2020-06-09 20:04:39,431 - root - INFO - Starting flask application...
And
flowelcat@vps729526:~/inprofit$ docker container exec -it inprofitapi python
Python 3.8.3 (default, Jun 3 2020, 19:49:40)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from unicorn_binance_websocket_api.unicorn_binance_websocket_api_manager import BinanceWebSocketApiManager
>>>
>>> binance_websocket_api_manager = BinanceWebSocketApiManager()
>>> print(binance_websocket_api_manager.get_version())
1.16.1
>>>
A will monitor logs and candles from stream and notify here if something goes wrong.
1.16.1 is WRONG! It must be 1.16.1.dev!
Try to deinstall all unicorn installations via pip and then run the installation of the dev stage again!
Ok, changed version to 1.16.1.dev.
inprofitapi | 2020-06-10 09:51:42,228 - root - INFO - Initializing flask application...
inprofitapi | 2020-06-10 09:51:42,313 - root - INFO - Setting up kline subscription...
inprofitapi | 2020-06-10 09:51:42,315 - root - INFO - New instance of unicorn_binance_websocket_api_manager 1.16.1.dev started ...
inprofitapi | 2020-06-10 09:51:42,593 - root - INFO - Starting missing candles loader...
inprofitapi | 2020-06-10 09:51:42,598 - root - INFO - Setting up mark price subscription...
inprofitapi | 2020-06-10 09:51:42,599 - root - INFO - New instance of unicorn_binance_websocket_api_manager 1.16.1.dev started ...
inprofitapi | 2020-06-10 09:51:42,738 - root - INFO - Setting up mini ticker subscription...
inprofitapi | 2020-06-10 09:51:42,740 - root - INFO - New instance of unicorn_binance_websocket_api_manager 1.16.1.dev started ...
inprofitapi | 2020-06-10 09:51:42,858 - root - INFO - Setting up user data subscription...
inprofitapi | 2020-06-10 09:51:42,886 - root - INFO - New instance of unicorn_binance_websocket_api_manager 1.16.1.dev started ...
inprofitapi | 2020-06-10 09:51:43,047 - root - INFO - Starting flask application...
This fix is included now in 1.16.2 release!
Can we close this issue? I think its successfully fixed.
I think yes, no problems so far. Thank you for quick fix)
Thanks for the good report, I think this fix made the lib stable, I was hunting this problem since a while :D (https://github.com/oliver-zehentleitner/unicorn-binance-websocket-api/issues/45)
Its so weired. 8 days no issue, then I write its fixed and then it happens again:
4 streams stopped. I investigate it asap.
I have same issue. A stream is crashing time after time:
ERROR:root:BinanceWebSocketApiSocket->start_socket(d1f4bcc1-092d-4e63-bbda-2003ec246d48, ['arr'], ['!userData']) Exception AttributeError Info: module 'asyncio.base_futures' has no attribute 'InvalidStateError' CRITICAL:root:BinanceWebSocketApiManager->stream_is_crashing(d1f4bcc1-092d-4e63-bbda-2003ec246d48)
Is it possible to fix it?
Python 3.8.5 unicorn-binance-websocket-api 1.17.0 Windows 10
same issue here. using 1.16.9 with python 3.8.1 on windows 10
I am not sure, but it seems the InvalidSateError happens if you have to less system powser (CPU).
Can you test it by making less streams or use a more powerful system?
I am running this program on a server and the cpu useage is typically low. It won't go beyong 50% no matter what happens. I have only two streams per user and these are the userData streams for SPOT and FUTURES At the moment there are 5 users that my program is keeping track of though. So, there are 5 different instances of websocket running in parallel each containing 2 streams. I don't think that I can personally test it. I mean, the program is under load and trading in binance and I can't take it down, at least not in a few days
i am running different test systems of this lib since one year. on my systems i recognized even 50% on average is very much. sometimes are peaks that are too much then.
Running on a raspberry pi doenst work in general, dont know why.
You just need one instance per endpoint (spot/futures/...), doesnt matter how much users you have.
So your best setup should be two instances, one for spot and one for futures.
then you can create 5 spot userData streams within one instance and 5 futures userData streams.
you can split the stream_buffer, look at this example https://github.com/oliver-zehentleitner/unicorn-binance-websocket-api/blob/master/example_multiple_userdata_streams.py so you know which stream has received the data.
Till now you have had to use set_private_api_config()
but its not thread safe and i added the possibility to add the key and secret to create_stream()
see here: https://oliver-zehentleitner.github.io/unicorn-binance-websocket-api/unicorn_binance_websocket_api.html?highlight=create_stream#unicorn_binance_websocket_api.unicorn_binance_websocket_api_manager.BinanceWebSocketApiManager.create_stream
BUT: As you can see here https://github.com/oliver-zehentleitner/unicorn-binance-websocket-api/issues/109 the new way of providing key and secret doenst work now, i try to fix it this weekend.
Its fixed providing key and secret via create_stream()!
Just update to 1.17.1
Still facing same issue. A stream is crashing and reconnecting time after time:
ERROR:root:BinanceWebSocketApiSocket->start_socket(d1f4bcc1-092d-4e63-bbda-2003ec246d48, ['arr'], ['!userData']) Exception AttributeError Info: module 'asyncio.base_futures' has no attribute 'InvalidStateError' CRITICAL:root:BinanceWebSocketApiManager->stream_is_crashing(d1f4bcc1-092d-4e63-bbda-2003ec246d48)
Is it possible to fix it?
Python 3.8.5 unicorn-binance-websocket-api 1.17.1 Windows 2019 Server, 6 GB RAM, 4x 3.4ГГц CPU
I will remove the InvalidStateError code part - it was a fix for an other problem. Lets make one step back.
@oliver-zehentleitner The odd thing is that this process is somewhat random. I mean, there are times that it happens like once every few hours but other times, its not that frequent. Since the stream is crashing each time, I think the websocket gets disconnected and thus any data sent in between the reconnect is lost (I have had inconsistent data only once in like 2 month but, I only use userData so I don't expect that much frequent data). So I think this is a somewhat important bug. One more thing. You added the parameters apiKey and apiSecret to create_stream but you have not added them to replace_stream(). I didn't check other places that it might have been useful.
get_stream_info(stream_id) returns 'reconnects' = 1 after any qty of 'InvalidStateError' reconnects. Each time 'reconnects' = 1.
'logged_reconnects' = [..last reconnect timestamp..] - has only one element in the array of the last reconnection. May be it should be a list of timestamps of all reconnects?
Python 3.8.5 unicorn-binance-websocket-api 1.17.1 Windows 2019 Server
@tahaEntesari Thanks for the hint about replace_stream() - i add it to the todo list. About disconnects in general: https://github.com/oliver-zehentleitner/unicorn-binance-websocket-api/issues/42 But I am with you, we want it as stable as possible :)
@alekkhalex i recogniced that too, this bugs is new since 1.17.0 - also added to todo list.
Its much more handy if there is one issue for one topic. Please think about that in the future! And open new one if needed!
@tahaEntesar @alekkhalex
Both fixes are released now in 1.17.4
Now all that reconnects have another message:
CRITICAL:root:BinanceWebSocketApiSocket->start_socket(09655d52-c65d-4ea3-9df7-5e242dc2577d, ['arr'], ['!userData']) Exception ConnectionClosed Info: code = 1006 (connection closed abnormally [internal]), no reason
From Binance documentation: -1006 UNEXPECTED_RESP An unexpected response was received from the message bus. Execution status unknown.
Python 3.8.5 unicorn-binance-websocket-api 1.17.4 Windows 2019 Server
Check this or we will delete your issue. (fill in the checkbox with an X like so: [x])
Select one:
Environment
Operating System? (include version)
Python Version Requirement
Exact Python Version?
Pip Version?
Dependencies
UNICORN Binance WebSocket API Version?
Description of your issue
Debug log when streams crashed: