meta: middleware roadmap

[niklasad1]

Currently jsonrpsee hasn't any specific JSON-RPC middleware interface rather than to just a logger to easily provide grafana metrics and similar usecase.

Currently jsonrpsee has:

• HTTP middleware (more specifically tower middleware)
• JSON RPC logger

We have a bunch of issues where folks want middleware to inspect JSON-RPC calls including HTTP specific details such as the URI, HTTP Headers and such things...

The HTTP (tower) middleware works great for pure HTTP related things but as soon as one wants to parse a JSON-RPC request, it not as nice to work with. Then in context of WebSocket connections it's only useful for the WebSocket upgrade handshake and after that point it's not possible to understand what's going, restrict certain calls or to disconnect peer that is misbehaving doing plenty of expensive calls.

The path forward is to replace the JSON-RPC logger with a more flexible JSON-RPC specific middleware which we have pending implementation for it but that alone won't solve disconnecting peers just deny calls.

To deal with disconnecting peers I could think of the following suggestions:

1. Provide some kind of ConnectionHandler/RpcMiddleware trait that does something similar to:

trait ConnectionHandler {
   /// Run on every connection attempt
   ///
   /// Return on `Some` if you want to deny a request from a certain peer or headers etc.
   async fn on_connect(&self, req: http::Request, conn_id: usize, remote_addr: SockAddr) -> Option<http::Response>;

   /// Run on every JSON-RPC call
   /// 
   /// Similar to https://github.com/paritytech/jsonrpsee/pull/1215
   async fn on_call(&self, req: JsonRpcRequest) -> MethodResponse;

   /// Runs when each connection is terminated.
   ///
   async fn on_disconnect(&self, id: Id); 

   /// Disconnect a peer
   async fn disconnect_peer(&self, id: Id);
}

This suggestion is really tricky to implement with all the state that needs to be synchronized somehow and jsonrpsee may become really complicated. It's not really clear how to couple calls with disconnect_peer.

2. Expose the jsonrpsee as a service such that users can control the state themselves via the hyper::service::fn then HTTP middleware interface can be removed but JSONRPC specific middleware is needed.

async fn run_server() {
    use hyper::service::{make_service_fn, service_fn};

    // Construct our SocketAddr to listen on...
    let addr = SocketAddr::from(([127, 0, 0, 1], 9944));

    // State
        let stop_handle = StopHandle::new(...);
    let conn_guard = ConnectionGuard::new(...);
    let service_cfg = jsonrpsee::server::Server::builder().to_service(().into_rpc());
    let conn_id = Arc::new(AtomicU32::new(0));
    let blacklisted_peers = Arc::new(Mutex::new(HashSet::new()));

    let make_service = make_service_fn(|conn: &AddrStream| {
        // Connection state.
        let conn_id = conn_id.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
        let remote_addr = conn.remote_addr();
        let stop_handle = stop_handle.clone();
        let conn_permit = Arc::new(conn_guard.try_acquire().unwrap());
        let service_cfg = service_cfg.clone();
        let blacklisted_peers = blacklisted_peers.clone();

        async move {
            let stop_handle = stop_handle.clone();
            let conn_permit = conn_permit.clone();
            let service_cfg = service_cfg.clone();
            let stop_handle = stop_handle.clone();
            let blacklisted_peers = blacklisted_peers.clone();

            Ok::<_, Infallible>(service_fn(move |req| {
                                // other application dependent logic....

                if peer_blacklisted(remote_addr.ip()) {
                    return reject(req).await;
                }

                if is_websocket {
                    let service_cfg = service_cfg.clone();
                    let stop_handle = stop_handle.clone();
                    let conn_permit = conn_permit.clone();
                    let blacklisted_peers = blacklisted_peers.clone();

                    let (tx, rx_conn) = mpsc::channel(1);
                    // Just an example to put a mpsc channel in the middleware to trigger a connection shutdown 
                                        // when it misbehaves.
                                        let rpc_service = RpcServiceBuilder::new().layer_fn(move |service| DummyRateLimit {
                        service,
                        count: Arc::new(AtomicUsize::new(0)),
                        state: tx.clone(),
                    });
                    tokio::select! {
                                           _ = websocket_task(....) => (),
                                           _ = rx_conn.recv() => {
                                             // RPC middleware triggered an limit
                                             // disconnect the peer and may blacklist it..
                                           }
                                      }
                } else {
                    http_request(...).await;
                }
            }))
        }
    });

    // Then bind and serve...
    let server = hyper::Server::bind(&addr).serve(make_service);

    server.await.unwrap();
}

The downside with this approach is that users needs to write plenty of code themselves but should be really flexible.

Any other suggestions?

//cc @jsdw @lexnv

[niklasad1]

Personally I prefer option 2) but open to suggestions

[jsdw]

I like the general idea of 2, in that we can allow for middleware which is capable of tracking whatever details matter, and then leave the disconnection outside of middleware (after all; disconnecting could happen at any point between when calls are being made etc, whereas middleware only runs when a call is made).

Would removing HTTP middleware make some basic use cases more difficult? I guess the idea is that with jsonrpsee as a Service, you can add middleware easily on top of it and don't need to have any interface/code in jsonrpsee for setting it, which sounds reasonable!

The current service I guess would handle the HTTP/WS stuff, and I think the example above for (2) is manually writing that code in the service (or some of it) in order to add whatever custom disconnect logic into it?

I think it'd be worth experimenting with anyway! If things are a bit too tricky there are probably things we can do to make life easier too

[niklasad1]

Would removing HTTP middleware make some basic use cases more difficult? I guess the idea is that with jsonrpsee as a Service, you can add middleware easily on top of it and don't need to have any interface/code in jsonrpsee for setting it, which sounds reasonable!

I guess we should keep the HTTP middleware API because we have a bunch of nice stuff such as the CORS, host filtering stuff which is much easier to use to provide your own service handler

The current service I guess would handle the HTTP/WS stuff, and I think the example above for (2) is manually writing that code in the service (or some of it) in order to add whatever custom disconnect logic into it?

Yes and I think it's much easier to write a hyper::service_fn than to write custom tower middleware but the jsonpsee::TowerService isn't really required when one wants custom disconnect logic just that one can re-use some configurations from the builder.

Maybe it's possible to expose another "jsonrpsee::DisconnectableTowerService" instead having a such low-level API.

[jsdw]

I guess we should keep the HTTP middleware API because we have a bunch of nice stuff such as the CORS, host filtering stuff which is much easier to use to provide your own service handler

I guess I'd keep it for now and see what comes out of experimenting with how to add disconnect logic etc!

Maybe it's possible to expose another "jsonrpsee::DisconnectableTowerService" instead having a such low-level API.

I'd have a go at the "low level" approach you had in mind anyway and just see how it works out; when there's some real code it'll be easier to see if there are any obvious things we can do to make it simpler or encapsulate some logic away or something :)

[ababo]

@niklasad1 Am I right assuming that currently there's no way to access values produced by HTTP middleware in JSON-RPC methods (e.g. to get user id retrieved by authentication middleware)?

[niklasad1]

@niklasad1 Am I right assuming that currently there's no way to access values produced by HTTP middleware in JSON-RPC methods (e.g. to get user id retrieved by authentication middleware)?

Yes, for latest jsonrpsee release you are correct but you could write specific HTTP middleware to take care of that but that works only properly for HTTP JSON-RPC calls and the API is very low-level but we recently merged a specific JSON-RPC middleware to deal with that.

The intention forward is that folks can do authentication with the new middleware that we recently merged #1215 but https://github.com/paritytech/jsonrpsee/pull/1224 is needed because we didn't want to expose the entire HTTP request in the middleware itself because it's already known when HTTP connection is made.

Instead folks have to launch a hyper::service_fn and wrap whatever details one would need from HTTP context instead that we pass it in for every RPC call, you can have a look this example that does dummy HTTP authentication.

This way it's more flexible and "jsonrpsee" doesn't have clone and bunch data that may not be used anyway.

However, other suggestions are also welcome of course.

[niklasad1]

Option 2) is now merged and the API jsonrpsee will provide