Make client parameters available in runtime environment

[alexpArtos]

In Substrate, we can use Offchain-Workers to do some things that cannot be done inside the runtime, for example during a consensus round. These are suitable for asynchronous tasks, like consulting a remote server or signing some message.

Offchain-Workers are restricted to a kind of sandbox though, and can’t fully access the file system. They can access the key store, but only for keys described in a specific format. As far as we know, we cannot freely read from the filesystem inside an OCW.

Conversely, we can pass any configuration parameters to a client node at start up which could be useful for the logic in the offchain worker (for example, a changeable server address). We would prefer to read these from a config file or a CLI parameter, instead of hard-coding them in the client and the runtime configuration, especially if these data shoult be private to this particular client and cannot be stored in public chain storage.

We would like to have a method of easy transfer of private data from the client to the runtime. Since we may want to copy private data, writing to Storage is not a good alternative. Use of the node’s local Db could be acceptable especially if we could write to it from outside the runtime.

[bkchr]

There are RPC methods for setting offchain storage: https://github.com/paritytech/substrate/blob/master/client/rpc-api/src/offchain/mod.rs#L30-L39

You can also follow the implementation to support reading from a config file.

[alexpArtos]

Thanks Basti. If I understand you correctly, you're saying we can contact the client from the OCW via a JSON-RPC Api. And also that we can access the local db from the OCW.

Yes, we know about that, but it is not a convenient method. We would rather not have to make a JSON-RPC call to obtain data from the client. They are asynchronous, for example. And we'd have to customize the client API for every piece of info we want to read to the OCW. I was rather hoping we could have a short cut method to obtain, say, named CLI parameters passed to the Client in the OCW.

Alternatively, since the local db is a file on disk, is there any external tool that we can use (say, a node.js program) to insert data into this local db to make it accessible to the OCW?

[bkchr]

Maybe we could provide some simple json file that would be loaded into the OCW database.

CC @tomusdrw

[tomusdrw]

Yeah, sounds good. I imagined a separate sub-command for the CLI to provide read/writes to Offchain Database. @alexpArtos you would do below to get what you need:

./substrate offchain-db --dev write 0xkey 0xvalue
./substrate offchain-db --dev read 0xkey # check the value is present
./substrate offchain-db --dev write-file ./test.json # What Basti proposed (probably a single JSON object with key-values)
./substrate --dev # Run the node

Is that sufficient?

I prefer separate sub-command over a flag to the regular run to make sure you don't accidentally overwrite some entries on every restart.

[alexpArtos]

Hi Thomas, yes, that would be great, thanks. So, the json file would allow us to set multiple key value pairs? I prefer the sub-command too, it gives us more control.

[takahser]

@bkchr what's the current state of this issue? I think having the ability to access client parameters (e.g. through a config file or environment variables) would be especially useful for accessing protected APIs through OCWs.