pendulum-chain / spacewalk

Apache License 2.0
34 stars 7 forks source link

Use reconnecting RPC client #521

Open ebma opened 4 months ago

ebma commented 4 months ago

Context

Sometimes, it happens that the underlying RPC client used by Subxt disconnects (e.g. due to network issues). In this case, the client throws an error that is not easy to detect in subxt, see https://github.com/paritytech/subxt/issues/1190.

This makes our client stuck in an infinite retry loop.

[2024-05-07T13:56:14Z INFO  runner::runner] Error fetching executable: SubxtError: Rpc error: RPC error: The background task been terminated because: Networking or low-level protocol error: WebSocket connection error: connection closed; restart required. Retrying...
[2024-05-07T13:56:15Z INFO  runner::runner] Error fetching executable: SubxtError: Rpc error: RPC error: The background task been terminated because: Networking or low-level protocol error: WebSocket connection error: connection closed; restart required. Retrying...

TODO

This PR adds a new experimental implementation of an RPC client that automatically reconnects. This implementation is only available in subxt v0.35 or later. I tried bumping the subxt dependencies we use in Spacewalk to that version but I encountered conflicts because our Polkadot dependencies are too outdated.

prayagd commented 2 weeks ago

@ebma is this still relevant? should we get it estimated?

ebma commented 1 week ago

This unfortunately is blocked by yet another upgrade of our polkadot dependencies. We can probably only really work on this once we bump our dependencies to Polkadot v1.6.0, ie the next one after the upgrade to v1.1.0. @prayagd do we already have a ticket for the v1.6.0 upgrade? If so, let's indicate as a Zenhub dependency that this ticket is blocked by that.