Closed rightx2 closed 6 years ago
slazarov
commented
6 years ago Hi, yes that error has been bothering everyone. It might be happening because we are overloading the bittrex socket. I put some exceptions in the on_channels to see if the error can be handled from there. Unfortunately, I’ve done that on some experimental branch and they got discarded. I will try to put them back in v0.0.6.3 again and you can start testing to see if you will capture them.
rightx2
commented
6 years ago @slazarov I think the last commit was 8 days ago, which means do I have to wait a new version?
slazarov
commented
6 years ago Yes, you will be notified. Watch the repo.
slazarov
commented
6 years ago Put some traps to capture the nasty websocket._exceptions.WebSocketConnectionClosedException: Connection is already closed?
Let's see if they will work. For anyone wishing to help, please install:
pip install git+https://github.com/slazarov/python-bittrex-websocket.git@0.0.6.3
slazarov
commented
6 years ago Ok, I have found a way to capture the exception and I am working on a solution.
rightx2
commented
6 years ago @slazarov What a good news! After some experiments, I found out that exception occured only for a few tickers, not for ALL tickers, which means, remain tickers are still getting data. So, I periodically start/quit the socket now.... If I can capture the exception, I can make my code more efficient and clean
rightx2
commented
6 years ago @slazarov Also, I know the someone who tried to make a program using javascript and snip the socket data packet on bittrex stie page(Chrome extension app). He said he should refresh the page periodically because socket closed unexpectedly.. Might it be a little help to you?
slazarov
commented
6 years ago Hi @rightx2, the problem is that there is a tradeoff between how many tickers you assign per connection. Namely, if you assign one connection per ticker, you can easily monitor it to check if data is coming through. However, what happens if you are subscribing to 100+ tickers? That means 100+ connections and threads, which is not efficient and Bittrex might block you.
Now...if we start monitoring a connection with 20 tickers (that's the default I have set), we can easily asses if the connection has dropped. This could be determined in 2 ways:
_on_debug, i.e the connection has droppedWith respect to the more certain way, that's easily handled. Note that I haven't incorporated it yet. With respect to the more uncertain way, I haven't observed whether the connection can partially drop for specific tickers and stay active for others. I highly welcome testers!
About your last message. The library refreshes the cookie every 5400 seconds (i.e 1h30) and restarts the connection. You can check def _init_connection(self, conn_obj):
I will soon release, what I assume should handle these exceptions and you are free to build on them or entirely use them to make your improvements. Looking forward to that!
slazarov
commented
6 years ago @rightx2 update to 0.0.7.0 either from pip or github and start testing :)
rightx2
commented
6 years ago @slazarov Hey I found out sort of bug(or inconvenience) in your new commit (0.0.7.0). Your new exception catcher, which print out "Caught {} in gevent. Don\'t worry.", catch my application-side errors too !!, which is not a gevent error...
slazarov
commented
6 years ago It’s not a gevent error but caught in gevent. You have to provide more info about what you have done in your code. Code, logs etc, I cant do guesswork.
On Feb 27, 2018 at 10:44 am, <JeongWoo Choi (mailto:notifications@github.com)> wrote:
@slazarov (https://github.com/slazarov) Hey I found out sort of bug(or inconvenience) in your new commit (0.0.7.0). Your new exception catcher, which print out "Caught {} in gevent. Don't worry.", catch my application-side errors too !!, which is not a gevent error...
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub (https://github.com/slazarov/python-bittrex-websocket/issues/35#issuecomment-368789233), or mute the thread (https://github.com/notifications/unsubscribe-auth/AbVUGyjaLdTKAOkSkNo01nLV7evD93QVks5tY8BugaJpZM4SBk8L).
emestee
commented
6 years ago @slazarov just upgraded to 0.0.7, will let you know if anything interesting shows up
rightx2
commented
6 years ago @slazarov I used this with django framework, and the error was like There is no such field name in your Symbol model, which is surely not related with gevent. This happened because I tried to store the data in DB but I misspelled the field name of my Symbol model. In 0.0.7, I could not recognize the error detail so I downgrade the version back to check what those errors are.
slazarov
commented
6 years ago It’s not related to gevent but within the websocket library, however errors in gevent are not propagated to the main script so they are handled differently.
On Feb 27, 2018 at 1:21 pm, <JeongWoo Choi (mailto:notifications@github.com)> wrote:
@slazarov (https://github.com/slazarov) I used this with django framework, and the error was like There is no such field name in your Symbol model, which is surely not related with gevent. This happened because I tried to store the data in DB but I misspelled the field name of my Symbol model. In 0.0.7, I could not recognize the error detail so I downgrade the version back to check what those errors are.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub (https://github.com/slazarov/python-bittrex-websocket/issues/35#issuecomment-368842237), or mute the thread (https://github.com/notifications/unsubscribe-auth/AbVUG5fQJC0JyDKByAQW3McO9lINeUokks5tY-VTgaJpZM4SBk8L).
rightx2
commented
6 years ago After several tries, It(0.0.7.0) turned out to be not working either after reconnection in 90mins... By the way, It's ok to run a program for getting tick, order book and order quote data simultaneously, isn't it?.... I'm getting those data of 30 coins simultaneously and wonder whether it is ok or not
slazarov
commented
6 years ago Hi, mate. I’ve been really busy the past few days and haven’t yet had the chance to troubleshoot it further. Does the error happens after reconnection or it occurs from the first run?
On the second questions - it’s ok.
On Mar 1, 2018 at 6:51 am, <JeongWoo Choi (mailto:notifications@github.com)> wrote:
After several tries, It turned out to be not working after reconnection in 90mins... By the way, It's ok to run a program for getting tick, order book and order quote data simultaneously, isn't it?.... I'm getting those data of 30 coins simultaneously and wonder whether it is ok or not
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub (https://github.com/slazarov/python-bittrex-websocket/issues/35#issuecomment-369473336), or mute the thread (https://github.com/notifications/unsubscribe-auth/AbVUG7zmojxmV6VhPoPQZsjfk3qg9oY8ks5tZ35HgaJpZM4SBk8L).
rightx2
commented
6 years ago @slazarov only after reconnection
rightx2
commented
6 years ago @slazarov Is there any update?
slazarov
commented
6 years ago Hi, I am working on a new SignalR client but it’s not clear what error you are getting now.
rightx2
commented
6 years ago Still got an Errors 2018-04-13 12:58:33 - bittrex_websocket.websocket_client - ERROR - [Connection][6f76ac56aa34899b66a1701eb59c36d]:Caught <class 'websocket._exceptions.WebSocketConnectionClosedException'> in gevent. Don't worry after 20mins run.. version is 0.0.7.3.
slazarov
commented
6 years ago Hi, the only when to simulate the error is by blocking the connection to Bittrex. This is what I am getting as an error message:
2018-04-13 07:22:36 - bittrex_websocket.websocket_client - INFO - [Subscription][OrderBook][['BTC-2GIVE']]: Initiating reconnection procedure. 2018-04-13 07:22:36 - bittrex_websocket._auxiliary - INFO - [Subscription][OrderBook][BTC-2GIVE]: Order book depth set to 10. 2018-04-13 07:22:36 - bittrex_websocket._auxiliary - INFO - [Subscription][OrderBook][BTC-2GIVE]: Snapshot nounce, state and internal queue are reset. 2018-04-13 07:22:36 - bittrex_websocket._auxiliary - INFO - [Subscription][OrderBook][BTC-2GIVE]: Disabled.
As you can see, the reconnection is initiated. What happens in your case?
slazarov
commented
6 years ago Closing due to inactivity.
sahfd4eiugf
commented
5 years ago Please re-open this issue as I just started to get this error consistently starting today. I am running bittrex-websocket==1.0.6.3
bittrex_websocket.websocket_client : ERROR - Exception = class 'websocket._exceptions.WebSocketConnectionClosedException'>, Message = Connection is already closed.
mrjingles41
commented
5 years ago seconded, I am also keeping a local copy of the orderbook which is falling out of sync after about 24 hours.
The last time this happened was about 08:30 am today. The log doesn`t tell me anything out of the ordinary though:
2019-06-18 06:01:52 - bittrex_websocket.websocket_client - INFO - Establishing connection to Bittrex through https://socket.bittrex.com/signalr. 2019-06-18 06:01:53 - bittrex_websocket.order_book - INFO - Successfully subscribed to [SubscribeToExchangeDeltas] for [BTC-PIVX]. 2019-06-18 06:02:10 - bittrex_websocket.order_book - INFO - Successfully subscribed to [QueryExchangeState] for [BTC-PIVX]. 2019-06-19 09:13:00 - bittrex_websocket.websocket_client - ERROR - Exception = class 'websocket._exceptions.WebSocketConnectionClosedException', Message = Connection is already closed.. 2019-06-19 09:13:00 - bittrex_websocket.websocket_client - DEBUG - Initiating reconnection procedure. 2019-06-19 09:13:00 - bittrex_websocket.websocket_client - DEBUG - Previous reconnection failed. Retrying in 5 seconds. Reconnection attempt 1. 2019-06-19 09:13:00 - bittrex_websocket.websocket_client - DEBUG - Last reconnection was more than 60 seconds ago. Resetting retry counter. 2019-06-19 09:13:00 - bittrex_websocket.websocket_client - INFO - Establishing connection to Bittrex through https://socket.bittrex.com/signalr. 2019-06-19 09:13:01 - bittrex_websocket.order_book - INFO - Successfully subscribed to [SubscribeToExchangeDeltas] for [BTC-PIVX]. 2019-06-19 11:04:49 - bittrex_websocket.websocket_client - ERROR - Exception = class 'websocket._exceptions.WebSocketConnectionClosedException', Message = Connection is already closed.. 2019-06-19 11:04:49 - bittrex_websocket.websocket_client - DEBUG - Initiating reconnection procedure. 2019-06-19 11:04:49 - bittrex_websocket.websocket_client - DEBUG - Previous reconnection failed. Retrying in 5 seconds. Reconnection attempt 1. 2019-06-19 11:04:49 - bittrex_websocket.websocket_client - DEBUG - Last reconnection was more than 60 seconds ago. Resetting retry counter. 2019-06-19 11:04:49 - bittrex_websocket.websocket_client - INFO - Establishing connection to Bittrex through https://socket.bittrex.com/signalr. 2019-06-19 11:04:50 - bittrex_websocket.order_book - INFO - Successfully subscribed to [SubscribeToExchangeDeltas] for [BTC-PIVX]. 2019-06-19 11:41:50 - bittrex_websocket.websocket_client - ERROR - Exception = class 'websocket._exceptions.WebSocketConnectionClosedException', Message = Connection is already closed.. 2019-06-19 11:41:50 - bittrex_websocket.websocket_client - DEBUG - Initiating reconnection procedure. 2019-06-19 11:41:50 - bittrex_websocket.websocket_client - DEBUG - Previous reconnection failed. Retrying in 5 seconds. Reconnection attempt 1. 2019-06-19 11:41:50 - bittrex_websocket.websocket_client - DEBUG - Last reconnection was more than 60 seconds ago. Resetting retry counter. 2019-06-19 11:41:50 - bittrex_websocket.websocket_client - INFO - Establishing connection to Bittrex through https://socket.bittrex.com/signalr. 2019-06-19 11:41:51 - bittrex_websocket.order_book - INFO - Successfully subscribed to [SubscribeToExchangeDeltas] for [BTC-PIVX].
So the connection is closed at 09:13:00 and successfully re-established at 09:13:01. However the values in the orderbook are out of sync. When monitoring the first bid and ask rates, I get the following picture:
(The other two tickers that keep moving after 09:13:00 are from a different exchange, the bittrex ones are the stalling ones)
So it seems like the orderbooks is not synced at all anymore. Do you have any suggestions on how to solve this?
I am also using version 1.0.6.3
Thanks in advance!
sahfd4eiugf
commented
5 years ago Yeah it actually gets even worst than that for me since the bid strangely overtakes the ask, which is obviously impossible in a normal market.
I run
bittrex-websocket==0.0.6.3version on my Linux Ubuntu 16.04 desktop.I run a code which gets about 40 symbol's order book and tick.
It works well at first time but after 1hour 30mins later, it raise exception like below :
. .
As you can see the log, there is nothing wrong with my code(all logs are related with related library, not my code).
And also
def on_error(self, error):doesn't called, even once.I don't know why this happend.
If you need a more infos, please tell me.
Thanks