[Errno 104] Connection reset by peer - data: None

[dorinand]

Hello,

$ pip list | grep cbpro
cbpro (1.1.4)

Today, my websocket stopped working because of Error 104: [Errno 104] Connection reset by peer - data: None

I change source code little bit, but I do not think it could cause the error, so just to be sure:

class ImprovedCoinbaseWebsocketClient(cbpro.WebsocketClient):

    def __init__(self, logger, url="wss://ws-feed.pro.coinbase.com", products=None, message_type="subscribe", mongo_collection=None,
                 psql_connection=None, should_print=True, auth=False, api_key="", api_secret="", api_passphrase="", channels=None):
        self.url = url
        self.products = products
        self.channels = channels
        self.type = message_type
        self.stop = True
        self.error = None
        self.ws = None
        self.thread = None
        self.auth = auth
        self.api_key = api_key
        self.api_secret = api_secret
        self.api_passphrase = api_passphrase
        self.should_print = should_print
        self.mongo_collection = mongo_collection
        self.psql_connection = psql_connection
        self.logger = logger

    def on_message(self, msg):
        """Process websocket message.
        """
        if self.should_print and msg['type'] == "match":
            print(msg)
        if self.mongo_collection:  # dump JSON to given mongo collection
            self.mongo_collection.insert_one(msg)
        if self.psql_connection and msg['type'] == "match":
            try:
                cur = self.psql_connection.cursor()
                cur.execute(my_query)
                self.psql_connection.commit()
            except Exception as e:
                self.logger.error('Unable to write to database.', exc_info=1)

So basically, I just added postgres connection and filter only match messages.

According to me, it should reconnect automatically.

[livingsilver94]

Happening to me too on three different machines (A Linux desktop, an armbian installation ad macOS). This is the simple testing script I used:

#!/usr/bin/env python3
import cbpro
class Client(cbpro.WebsocketClient):
        def on_open(self):
                self.count = 0
        def on_message(self, msg):
                self.count += 1
                print(self.count)
ws = Client(channels=['full'])
ws.start()

It closed after about.... 6 hours.

[noah222]

Happening to me too on three different machines (A Linux desktop, an armbian installation ad macOS). This is the simple testing script I used:

#!/usr/bin/env python3
import cbpro
class Client(cbpro.WebsocketClient):
        def on_open(self):
                self.count = 0
        def on_message(self, msg):
                self.count += 1
                print(self.count)
ws = Client(channels=['full'])
ws.start()](url)

It closed after about.... 6 hours.

Sounds about right, you just need to enclose this in another loop that reconnects. Personally I reconnect every minute. It seems redundant but helps keep your money. Based on the frequency of dropped connections, I suspected that coinbase was doing this to help them make money. Reconnecting often is key if you are conducting many trades per day.

yesterday my feed cut right before a large drop on ETH, they want to trap you then you will do a panic market sell and they get some fee earnings. But it reconnects automatically so it's no big deal. What's worse is when they cut my feed AND stall my last request at the same time.

I treat this stuff like war, and you can never be too prepared.

The solution that worked out the best for me was to run the feed in a separate python script, and it outputs the current feed information to a text file. Then my main trading script runs in another window, reading the textfile and that is undisturbed by feed issues. So the current state of my trading (that can be complex with many entrance points) is preserved and does not error out when the feed has an issue. I looked into just using different threads but this way gives you much more stability and control.

[moyphilip]

Out of curiosity, does your cbpro version have the commit that fixed the websocket dropped connections? https://github.com/danpaquin/coinbasepro-python/commit/e4ba196c3416e062ff5497a7d6e148ef874c422f

The reason I ask is because I was getting the same msg but i enclosed it in a loop to reconnect as well. But, when I noticed there was commit that fixed it pip install --upgrade cbpro did not pull in the latest commits.

To get the latest changes I ran: pip uninstall cbpro pip install git+git://github.com/danpaquin/coinbasepro-python.git

[edmondja]

Out of curiosity, does your cbpro version have the commit that fixed the websocket dropped connections? e4ba196

The reason I ask is because I was getting the same msg but i enclosed it in a loop to reconnect as well. But, when I noticed there was commit that fixed it pip install --upgrade cbpro did not pull in the latest commits.

To get the latest changes I ran: pip uninstall cbpro pip install git+git://github.com/danpaquin/coinbasepro-python.git

Hi, this didn't solve anything for me, I'm afraid spamming reconnections is the solution.. (EDIT: it doesn't solve anything) read : https://github.com/danpaquin/coinbasepro-python/issues/211

[jacobfrank250]

I downloaded cbpro package using pip install git+git://github.com/danpaquin/coinbasepro-python.git and I still run into this issue. It occurs frequently when subscribed to the full channel and sometimes (every 48 hours or so) with the user channel.

Spamming reconnections does not seem like the best solution because we will miss messages while reconnecting.

I also think this might be happening when we reach the rate limit of over 100 messages per second. This is from the coinbase pro api docs: "messages sent by client on each connection are rate-limited to 100 per second per IP"

Is anyone else experiencing this issue or have found another solution?

[noah222]

So the solution is to force your own connection to time out every 10 seconds and reconnect automatically when you have not received a message. If done fast enough there will be no dropped messages or very rarely a dropped message and a very stable connection over the long term.

[gabdim]

So the solution is to force your own connection to time out every 10 seconds and reconnect automatically when you have not received a message. If done fast enough there will be no dropped messages or very rarely a dropped message and a very stable connection over the long term.

I am faced with the same disconnection problem. Some times it takes place after 29 hours, other times after 1 hour. Errors are either host forcibly closed the connection or other errors. How the websocket can be disconnected and reconnected from time to time using this python wrapper? It would be nice to pre-define a time interval when the code will disconnect and then the reconnection takes place after a few seconds, this all in perpetuity (no manual intervention). Would it be that possible? I don't mind if I Ioose some messages (I need this to get bid and ask quotations in order to place buy orders when the price goes below some level). Using get_product_ticker() in perpetuity is discouraged by the API (according to this)

[jacobfrank250]

I close and restart the WebSocket connection whenever I run into an error, instead of reconnecting at a predefined interval.

The code essentially is:

wsClient = cbpro.WebsocketClient(url="wss://ws-feed.pro.coinbase.com",
                                products=["BTC-USD", "ETH-USD"],
                                channels=["ticker"])
wsClient.start()
while true:
        if wsClient.error:
                wsClient.close()
                wsClient.error = None
                wsClient.start()
        time.sleep(1)

I've had this running 24/7 with no intervention for a few months now.

[gabdim]

I close and restart the WebSocket connection whenever I run into an error, instead of reconnecting at a predefined interval.

The code essentially is:

wsClient = cbpro.WebsocketClient(url="wss://ws-feed.pro.coinbase.com",
                                products=["BTC-USD", "ETH-USD"],
                                channels=["ticker"])
wsClient.start()
while true:
        if wsClient.error:
                wsClient.close()
                wsClient.error = None
                wsClient.start()
        time.sleep(1)

I've had this running 24/7 with no intervention for a few months now.

Started yesterday to run but today I got the same error: [WinError 10054] An existing connection was forcibly closed by the remote host It did not restart as the stream stopped out.

I am on WIndows Server 2012 R2 Standard Python version 3.7.0 (in a virtual environment - to isolate from other version of Python I have installed on that machine) I am going to repeat the running for another couple of days in order to try reproduce the error, will update later.

Update (18 July 2021): no more errors as it seems I inlcluded in my code the line time.sleep(1) within the if (i.e. wsClient.error:) so I guess this is why when I corrected the code the API took a breath and signaled no more errors. Interruptions were 3-4 per day (as mentioned later on by robertbasham - see below), but the code acted well and reconnected without any problem.

[robertbasham]

It seems that Coinbase randomly disconnects the public web socket feed for each feed 2 - 3 times each day. It seems to happen more when the markets are busy, and occurs at at a different time for each pair code, when each has its own feed. I assume it's their way of limiting connections. So your code needs to allow for that, and just reconnect when it occurs, as in the sample code above. If it is happening more than x3 a day, then it's due to some problem in your code.

If you can't tolerate any data feed dropouts, you could run two concurrent feeds. I used to do that and found they would sometimes get dropped within a fraction of a second of one another, if they were started at the same time, so you should start them up at different times.

[jepessen]

I've the same problem. Sometime the websocket disconnects:

[Errno 104] Connection reset by peer - data: None

I'm using a derived class that call start when on_close is invoked:

class CBProFeed(cbpro.WebsocketClient):

  def __init__(self):
    pass

  def on_open(self):
    self.url = "wss://ws-feed.pro.coinbase.com/"
    self.channels=["ticker"]

  def on_message(self, msg):
    # My business logic

  def on_close(self):
    logging.error("Connection closed. Trying to start again..")
    self.start();

  def run_forever(self):
    logging.info("Feed is starting...")
    self.start()
    schedule.every().minute.at(':00').do(self._handle_candle_callback)
    while True:
      schedule.run_pending()
      time.sleep(.1)
    self.close()

The only thing that I don't understand is that every time that the websocket closes, it open another thread:

2022-02-03 17:59:40,727 [Thread-1    ] [ERROR]  Connection closed. Trying to start again..
...
2022-02-03 18:47:18,565 [Thread-2    ] [ERROR]  Connection closed. Trying to start again..
...
2022-02-03 19:01:36,261 [Thread-3    ] [ERROR]  Connection closed. Trying to start again..

I hope that the thread numbering does not call for problems if the connection is closed too many times...