Closed grod220 closed 5 days ago
if we query for blocks directly from Rust using a wasm-compatible grpc client
This is possible in theory, but the rabbit hole may be too deep for now (?). We need the rpc feature enabled to get the query clients needed for tonic-web-wasm-client
. We also need the proto-compiler's tonic-build dep to exclude the tonic transport impls and re-compiled. Something like:
tonic-build = { version = "0.10.0", default-features = false, features = ["cleanup-markdown"] }
# then run ./deployments/scripts/protobuf-codegen
And then finally, we need to have penumbra-proto
package be wasm compatible:
cargo build --lib --release --target wasm32-unknown-unknown
At the moment, we are greeted by an assortment of compile issues relating to required "js" features and exclusion of randomness libs.
can we close this as not planned?
the underlying motivation is raw performance, but the primary goal has been identified as parallelization. in fact the compute-heavy tasks of the block processor are already performed in wasm.
we've experimented with the serde techniques in the issue and found them to be unsuitable. profiling has provided new insight. at best this issue needs refinement, but i think ultimately it does not represent current understanding of the objectives.
given the present consensus and planning effort, to be issued in the next few days, i'm closing this issue.
Context
Attempts to serialize through binary have not added any performance gains
An idea arose to move more decryption logic into typescript to put less load on serialization requirements. However, it did not have any performance gains.
To do
Given wasm is quite performant, the idea is to move block processing into wasm entirely. This prevents the back-and-forth serialization overhead. It may even simply logic if it only exists in one domain.
== Consideration == We are querying from typescript HERE. By default,
connectRPC
is taking the bytes from the query response and parses them. So our current flow actually looks like this:We should attempt to simplify this to:
This would mean copying over parts of code that allow us to not parse, but pass the UnitArray on. Suggestions: https://bufbuild.slack.com/archives/CRZ680FUH/p1711072213138009
From bufbuild team:
Though, we may be able to skip typescript fetching entirely and simplify this to:
Which would only be possible if we query for blocks directly from Rust using a wasm-compatible grpc client like tonic-web-wasm-client.
Tasks: