polkascan / py-substrate-interface

Python Substrate Interface
https://polkascan.github.io/py-substrate-interface/
Apache License 2.0
239 stars 111 forks source link

Websockets for long-lived-connections #356

Closed MathCryptoDoc closed 9 months ago

MathCryptoDoc commented 9 months ago

Hi,

Is there a way to use websockets in a thread like here so that reconnections are handled better?

For example, with a subscription the connection is not made again. Also, if a the wss server is down for a few seconds, I think the current reconnection logic does not retry once it failed.

arjanz commented 9 months ago

What you can do is catch the exceptions that occur when the Substrate node disconnects for some reason (or any other connectivity issues). Something like:

try:
    self.substrate = SubstrateInterface(...)

    # main loop

except (WebSocketConnectionClosedException, ConnectionRefusedError,
                        WebSocketBadStatusException, BrokenPipeError, SubstrateRequestException) as e:
                    # re-establish connection
                    self.log("⛔ Connection lost: '{}' Reconnecting ...".format(e))
                    try:
                        self.substrate.connect_websocket()
                        # recover and continue
                    except (ConnectionRefusedError, WebSocketBadStatusException, BrokenPipeError,
                            SubstrateRequestException) as e:
                        self.log("⛔ Reconnect failed, retrying in 30 seconds ".format(e))
                        sleep(27)

This example is taken from the harvester application

I have considered automatic reconnects in the library itself, but came to the conclusion this is really up to the application, to give more control and anticipate in a specific situation.

Also, some public Substrate endpoints can be quitte unreliable, depending on your use-case you might want to use your own Substrate node.

MathCryptoDoc commented 9 months ago

Yes, I understand this kind of logic and maybe it is the best way forward to keep it flexible. With a subscription you need to re-issue the query anyway if the substrate node went down or resets.

Thanks for the reference to harvester. Good example code as well there in general.