This is the best effort to get the current gadgets to work on our new design without going through my approach of using JSON RPC for communication between the Shell and the running gadgets.
Approach for Shell <> Gadget
┌───────────────────Shell─────────────────────────────┐
│ │
│ │
│ Spwans all the gadgets that are required │
│ for the current validator, regardless if │
│ there any jobs for them or not. │
│ │
│ │
│ │
└──────────────────────────┬──────────────────────────┘
│
│
│
┌─────────────Gadget──────▼─────────────────────────┐
│ │
│ │
│ ┌───────────────────────────────────────────┐ │
│ │ │ │
│ │ libp2p networking layer │ │
│ │ │ │
│ └───────────────────────────────────────────┘ │
│ │
│ │
│ ┌───────────────────┐ ┌───────────────────┐ │
│ │ │ │ │ │
│ │ Substrate │ │ │ │
│ │ RPC Client │ │ Keystore │ │
│ │ │ │ Secure Enclave │ │
│ │ │ │ │ │
│ │ │ │ │ │
│ └───────────────────┘ └───────────────────┘ │
│ │
│ │
│ ┌──────────────────────────────────────────────┐ │
│ │ │ │
│ │ Gadget Specific Code │ │
│ │ │ │
│ └──────────────────────────────────────────────┘ │
│ │
└───────────────────────────────────────────────────┘
This likely the current architecture we have today, the shell starts, gets all the blueprints that the current validator registered on, downloads all of them, and then starts all of them. Each gadget now has to manage all of its libp2p connections (if needed) and have access to the Keystore, and of course and RPC client, that needs to listen for jobs.
To go with this flow, we will need an SDK that would essentially let me write the following:
#[tokio::main]
async fn main() -> anyhow::Result<()> {
// This should read the following from the ENV
// 1. RPC_URL: the endpoint for the RPC node.
// 2. KEYSTORE_PATH: to have access to the keystore
// 3. LIBP2P_ID: libp2p specific idenity file
// 4. BLUEPRINT_ID: the id of this blueprint
//
// then, this would create a context, that holds all these needed things
let ctx = sdk::init().await?;
// init the libp2p networking
let libp2p_networking = sdk::start_libp2p_networking(ctx.clone()).await?;
// then we will need access to all the services that are already created from the current blueprint
// and a stream of the future services that will be created from that blueprint
let services_stream = sdk::start_blueprint_services_stream(ctx.clone()).await?;
// then for each service, start a new background task that would handle a single job for each service.
...
}
we should start working on the SDK, and the blueprint template git repo that has this boilerplate code ready to work with.
This is the best effort to get the current gadgets to work on our new design without going through my approach of using JSON RPC for communication between the Shell and the running gadgets.
Approach for Shell <> Gadget
This likely the current architecture we have today, the shell starts, gets all the blueprints that the current validator registered on, downloads all of them, and then starts all of them. Each gadget now has to manage all of its libp2p connections (if needed) and have access to the Keystore, and of course and RPC client, that needs to listen for jobs.
To go with this flow, we will need an SDK that would essentially let me write the following:
we should start working on the SDK, and the blueprint template git repo that has this boilerplate code ready to work with.