Runtime upgrade migrations leaves some keys in storage

[goncer]

Hey! I'm running a runtime upgrade, that triggers a migration. Somehow it leaves some storage keys in the chainstate, while when running the nodes with parachain-launch, migrations look okay.

How to reproduce:

You can either create two versions of any parachain, with a migration ( hopefully gets reproduced )

Or

You can run the test on the mangata-e2e repo. This test basically spawns two parachains, performs a runtime upgrade, and runs some xcms.

• Checkout this PR - code branch: https://github.com/mangata-finance/mangata-e2e/pull/328/files
• nvm use 18.16.1
• yarn
• yarn test-xcm-ru-statemine

Ping me if any q! thanks!

[xlc]

Likely some key handling needs an improvement.

[ermalkaleci]

@goncer can't reproduce this

[goncer]

hey @ermalkaleci , Sorry, long weekend :)

How are you running the test? I tried again, and works for me :o Steps:

1. git checkout feature/improve-ru-xcm-tests
2. nvm use 18.16
3. rm -rf node_modules && rm yarn.lock && yarn
4. yarn test-xcm-ru-statemine It should take some time until the runtime applies, but after ~100s it will print some AssetRegistry info, and you will probably check on polkadotJs 3000 or 3001 port.

:crossed_fingers:

[ermalkaleci]

@goncer I am following your steps and I still get the same error:

[ermalkaleci]

I believe the problem is at this line https://github.com/mangata-finance/mangata-e2e/pull/328/files#diff-61c0e1b3905edb6ef2907983f7581a0f791baf288d90de1ba24fc920e1022d7dR145

[ermalkaleci]

most likely the account doesn't have balance to pay fee

[ermalkaleci]

got it running

but I can't find any key ["25"] nor null value

[goncer]

omg, we run the runtime upgrade on Oct23, so that's the reason, the upgrade is already applied on Kusama, let me try to find the block before the runtime upgrade and see if we can reproduce it :facepalm: Sorry for the late answer :(

[goncer]

@ermalkaleci Pushed a script change that sets the block number way before the runtime upgrade.

• "test-xcm-ru-statemine": "export MANGATA_BLOCK_NUMBER=3469500 ` plz, get the latest and greatest. The issue must be reproducible now. Thanks for following this up! :clap:

[ermalkaleci]

ok reproduced

[ermalkaleci]

@goncer what's the upgrade block number?

[goncer]

hey @ermalkaleci , 3469557! https://mangatax.subscan.io/block/3469557?tab=event

[goncer]

:tada: