pallet benchmarking: Bump default steps and repeat

paritytech / substrate

Substrate: The platform for blockchain innovators

Apache License 2.0

8.39k stars 2.65k forks source link

pallet benchmarking: Bump default steps and repeat #14408

Closed ggwpez closed 1 year ago

ggwpez commented 1 year ago

Even experienced devs are unaware that these need to be set to something higher than the default to produce meaningful values.
Not obvious; I admit. It came up on SE.

Changes:

~~Adds a test to the integrity_test of the scheduler and whitelist pallet to ensure that the weight depends on the components.~~ Not done in favour of https://github.com/paritytech/polkadot-sdk/issues/285.
Uses Substrate default values for steps and repeat

cc @AurevoirXavier

shawntabrizi commented 1 year ago

Really there should be some kind of feedback loop where we keep repeating a benchmark until we converge to some value with low error. 50 / 20 was something that I came to after testing and looking at raw results, but might not make sense for every scenario.

ggwpez commented 1 year ago

bot merge

ggwpez commented 1 year ago

Really there should be some kind of feedback loop where we keep repeating a benchmark until we converge to some value with low error. 50 / 20 was something that I came to after testing and looking at raw results, but might not make sense for every scenario.

Yea there is this issued https://github.com/paritytech/polkadot-sdk/issues/379, but no time to work in it (yet).