bluealloy / revm

Rust implementation of the Ethereum Virtual Machine.
https://bluealloy.github.io/revm/
MIT License
1.58k stars 518 forks source link

revm can not be used in restricted async environments such as WASM environments #554

Open oblique opened 1 year ago

oblique commented 1 year ago

We want to use this crate in a WASM environment that do not have threads. To be more specific we want to use it in an Internet Computer canister, but the same restriction exists in browsers too.

The main reason is because those environments are strictly single threaded and concurrency is only done via async. In our case we want to create a light node that does RPC requests to a full node, so all Database methods need to be async, because communication is async.

We notice that you already have something similar in EthersDb and you solve the issue by starting a new Tokio runtime and then use block_on, however this is not an option in our case.

We tried just to change Dadabase::basic to async and it looks like a lot more changes need to be done to convert everything to async.

We still didn't decide how to tackle the issue, but if we decide to convert everything to async, are you interested for the changes?

rakita commented 1 year ago

I apologise for the delayed response, a lot of things happened :)

revm is currently not wasm ready as ruint (dependent lib) is not there.

The reason why i didn't use async, is that database calls are sync calls, so the assumption is you could always bridge it in some way inside a database (maybe with queue) as it is user defined struct. I am open to discussing it.

oblique commented 1 year ago

revm is currently not wasm ready as ruint (dependent lib) is not there.

Actually we are already running revm in WASM environment and works.

The reason why i didn't use async, is that database calls are sync calls, so the assumption is you could always bridge it in some way inside a database (maybe with queue) as it is user defined struct. I am open to discussing it.

Imagine this: You are running an async function and inside you call revm::Database::basic, then the user defined struct may not have the result already cached, so it needs to fetch it over the network. That means the revm::Database::basic will block, so executor will block.

Of course if you are using Tokio there is an easy solution: you use spawn_blocking or block_in_place and call it a day. However in a more restricted environments were only one thread exists, this can not be done, because any solution involves spawning a new worker thread. One example of such environment is WASM.

Another solution would be to have async fn fetch_basic method in your user defined struct and you call it before you call revm::Database::basic. However the problem here is that you can not use any higher level logic that is bound to Database trait. You can not even use anything else from revm, for example revm::EVM::transact.

I don't see any solution other than going full async.

rakita commented 1 year ago

revm is currently not wasm ready as ruint (dependent lib) is not there.

Actually we are already running revm in WASM environment and works.

This is nice, didn't follow but ruint seems to got the update: https://github.com/recmo/uint/pull/274

The reason why i didn't use async, is that database calls are sync calls, so the assumption is you could always bridge it in some way inside a database (maybe with queue) as it is user defined struct. I am open to discussing it.

Imagine this: You are running an async function and inside you call revm::Database::basic, then the user defined struct may not have the result already cached, so it needs to fetch it over the network. That means the revm::Database::basic will block, so executor will block.

Of course if you are using Tokio there is an easy solution: you use spawn_blocking or block_in_place and call it a day. However in a more restricted environments were only one thread exists, this can not be done, because any solution involves spawning a new worker thread. One example of such environment is WASM.

How does the fetching over the network happen is it from wasm or outside, if you have a small example that would be great for me to see. I mean you still need an executor to consume async/await features.

Another solution would be to have async fn fetch_basic method in your user defined struct and you call it before you call revm::Database::basic. However the problem here is that you can not use any higher level logic that is bound to Database trait. You can not even use anything else from revm, for example revm::EVM::transact.

Yeah, this is not good solution.

oblique commented 1 year ago

How does the fetching over the network happen is it from wasm or outside, if you have a small example that would be great for me to see. I mean you still need an executor to consume async/await features.

It is within WASM, which has it's own async executor. If an address is missing we basically use JSON RPC and request from a full node the information. It is very similar to what you do in ethersdb. The only difference is that we can not use block_on or anything similar because it requires another worker thread (on a higher level). Most WASM environments are single-threaded by design and concurrency is done with async.