paritytech / polkadot-sdk

The Parity Polkadot Blockchain SDK
https://polkadot.network/
1.8k stars 652 forks source link

[benchmarking] Reset to genesis storage after each run #5655

Closed ggwpez closed 6 days ago

ggwpez commented 2 weeks ago

The genesis state is currently partially provided via OverlayedChanges, but these changes are reset by the runtime after the first repetition, causing the second repitition to use an invalid genesis state.

Changes:

skunert commented 1 week ago

@ggwpez Anything speaks against merging this? I have a follow up that could share some code.

bkontur commented 6 days ago

This fix is working finally for short-benchmarking with omni-bencher for bridge-hubs/collectives: https://github.com/paritytech/polkadot-sdk/actions/runs/10983379680/job/30492700127?pr=5706 https://github.com/paritytech/polkadot-sdk/actions/runs/10983379680/job/30492700240?pr=5706 https://github.com/paritytech/polkadot-sdk/actions/runs/10983379680/job/30492700384?pr=5706 so merging this

paritytech-cmd-bot-polkadot-sdk[bot] commented 6 days ago

Created backport PR for stable2407:

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin backport-5655-to-stable2407
git worktree add --checkout .worktree/backport-5655-to-stable2407 backport-5655-to-stable2407
cd .worktree/backport-5655-to-stable2407
git reset --hard HEAD^
git cherry-pick -x 2e4e5bf2fd0ae19fa38951c7e5f495dd1453b2bb
git push --force-with-lease
paritytech-cmd-bot-polkadot-sdk[bot] commented 6 days ago

Successfully created backport PR for stable2409: