The measurements are very noisy due to differing failures between EDR and EJS when replaying RPC calls from scenarios. The increased number of failures with EJS favors EJS as it runs fewer transactions/calls. For this reason I don’t think we should add the baseline to our dashboard.
I can say with relative confidence that we’re ~2x faster on smaller scenarios
Variance in EJS mode and EDR + Rust is a lot lower than in EDR + JS which suggests that the garbage collection of NAPI objects is significantly slower
neptune-mutualstands out which is slower in EDR mode for some reason in the benchmark while it was faster when running the test suite (https://github.com/NomicFoundation/edr/issues/33)