near / nearcore

Reference client for NEAR Protocol
https://near.org
GNU General Public License v3.0
2.31k stars 618 forks source link

Proper benchmarking for near-vm-runner #1289

Closed MaksymZavershynskyi closed 4 years ago

MaksymZavershynskyi commented 5 years ago

We need to know when our changes cause smart contract to run slower or faster.

Currently we have some benchmarks but they use bencher, which is problematic because it is too simplistic: https://github.com/nearprotocol/nearcore/blob/deee97ac4c29d61095770d04373bc5619a490926/runtime/near-vm-runner/benches/bench.rs

We need to switch to criterion for the following reasons:

We can follow the following code to start using criterion: https://github.com/nearprotocol/borsh/blob/master/borsh-rs/benchmarks/benches/bench.rs

MaksymZavershynskyi commented 4 years ago

Fully implemented by runtime-params-estimator without criterion.