[TASK] Optmize Substrate Connections

[shekohex]

Overview

Recently, we found a bug that spams (opens a lot of) connections to the tangle archival nodes, this may be due to a lot of clients that gets created instead of cached. we found it easily identifiable from the logging here: https://github.com/webb-tools/relayer/blob/f42ec552697faca1f6bd3b5749ce26c11d702d66/crates/event-watcher-traits/src/substrate/event_watcher.rs#L159-L171 which was introduced through this PR #522

Every call to ctx.substrate_provider actually creates a new connection (even for the same chain id) which is a waste of resources, what we should do instead is to cache the RPC clients in a hashmap so that we can reuse them across different clients. That is exactly we did way back for EVM Providers.

Task Checklist

• [x] Create a shared RPC Client and store it in Arc<HashMap<types::U256, Arc<RpcClient>>>
• [x] Construct the RPC Clients during the RelayerContext::new call (notice that creating the clients does not actually connect)
• [x] Whenever we call RelayerContext::substrate_provider method, we create a new OnlineClient from the shared Arc<RpcClient> we cached.
• [x] if this failed, calling this again with a same client will reuse the connection.
• [x] Can we use HttpClient as RPC Client? if yes, maybe that is a better option as they are stateless.
• [ ] (optional) While constructing the RpcClient, please add User-Agent header to webb-relayer/<version> <build-commit> so that we can see these in the Graphana logs/UI.

References

1. Creating a custom RPC Client: https://github.com/paritytech/subxt/blob/c8462defabad10a2c09f945737731e7259f809dd/subxt/src/backend/rpc/rpc_client.rs#L240-L253
2. Using the created RPC Client to create an OnlineClient: https://github.com/paritytech/subxt/blob/059723e4313d91e8ca0bcd84b0dd7dd66686ca50/subxt/src/client/online_client.rs#L86

[salman01zp]

Since we use subscriptions, I don't think we can use HttpClient.

[salman01zp]

Creating RpcClient wrapper for something that impl RpcClientT is not possible in the current version of subxt, since its new method is private. This will be available in the next release https://github.com/paritytech/subxt/blob/c8462defabad10a2c09f945737731e7259f809dd/subxt/src/backend/rpc/rpc_client.rs#L23-#L35

For now creating WebbRpcClient type which we can use to cache and then create online Client from it

pub struct WebbRpcClient(pub Arc<jsonrpsee::async_client::Client>);

impl WebbRpcClient {
    pub async fn new(url: impl Into<String> ) -> webb_relayer_utils::Result<Self> {
        let url: http::Uri = url.into().parse().map_err(|_| {
            webb_relayer_utils::Error::Generic("RPC url is invalid")
        })?;
        let (sender, receiver) =
                jsonrpsee::client_transport::ws::WsTransportClientBuilder::default()
                .build(url)
                .await
                .map_err(|_| {
                    webb_relayer_utils::Error::Generic("RPC failed to connect")
                })?;

        let client = jsonrpsee::async_client::ClientBuilder::default()
                .max_notifs_per_subscription(4096)
                .build_with_tokio(sender, receiver);

       Ok(Self(Arc::new(client)))
    }
}

impl RpcClientT for WebbRpcClient {
    fn request_raw<'a>(
        &'a self,
        method: &'a str,
        params: Option<Box<jsonrpsee::core::JsonRawValue>>,
    ) -> subxt::rpc::RpcFuture<'a, Box<jsonrpsee::core::JsonRawValue>> {
        self.0.request_raw(method, params)
    }

    fn subscribe_raw<'a>(
        &'a self,
        sub: &'a str,
        params: Option<Box<jsonrpsee::core::JsonRawValue>>,
        unsub: &'a str,
    ) -> subxt::rpc::RpcFuture<'a, subxt::rpc::RpcSubscription> {
        self.0.subscribe_raw(sub, params, unsub)
    }
}