search

paritytech / polkadot-sdk

The Parity Polkadot Blockchain SDK
https://polkadot.com/
1.89k stars 691 forks source link

Parachain ValidationFunctionApplied different relay block than Relay #962

Open crystalin opened 3 years ago

crystalin commented 3 years ago

Moonriver (para id 2023) did a runtime upgrade, Friday around 2:45am UTC The runtime was planned to be applied 1 hour later.

When the time came, the runtime upgrade was applied on the parachain. The collators started to produce blocks with the new runtime successfully (no error in the logs neither), but the relay validator refused them (Digest item must match that calculated.). The parachain became stalled at that point with no possibility of recovery.

After looking at the paras state, we observed that the relay paras.futureCodeUpgrades was planned for block 8,263,579 (which was already past when looking at it) image

but the parachain parachainSystem.ValidationFunctionStored reported the execution would happen on relay block 8,263,577 https://polkadot.js.org/apps/?rpc=wss%3A%2F%2Fwss.moonriver.moonbeam.network#/explorer/query/0x62e6661e7adc691420fdc125be711be59b91aaa7fc4e6976e3cd3e408464df84 image

And the parachainSystem.ValidationFunctionApplied reported the execution happened on block 8,263,578 (https://polkadot.js.org/apps/?rpc=wss%3A%2F%2Fwss.moonriver.moonbeam.network#/explorer/query/0xef9af479d1bf580fa9289d7bdd331f7d11056e398448c2e888a9eb69224d5c27) image

According to @bkchr, this is really really bad :(

(edit) The code hashes:

paras.futureCodeHash: 0x0180ee63bf0c715462bfcdd8de7072ff462af9ce99226a06b79bb1a581e71449 (spec_version 155)
paras.currentCodeHash: 0x08f6d9aa9cc1f326cf38e3385fe56de65700bc42a3d84350a2224f0d798e18d4 (spec_version 53)

runtime-53 (moonriver) can be found here: https://github.com/PureStake/moonbeam/releases/tag/v0.8.4 runtime-155 (moonriver) can be found here: https://github.com/PureStake/moonbeam/releases/tag/v0.9.2

xlc commented 3 years ago

https://moonriver.subscan.io/block/67640

The parachainSystem.setValidationData sets the relaychain block number to 8262979.

This runtime upgrade is performed using system.setCode via democracy. This means it is called in on_initialize hook, that is executed before inherents. So it is reading the old relaychain block number instead of the new one.

We really need https://github.com/paritytech/polkadot-sdk/issues/312 to address this.

Meanwhile, parachain team should be able to use parachainSystem.authorizeUpgrade for democracy and use parachainSystem.enactAuthorizedUpgrade to actually perform the upgrade.