search

paritytech / substrate

Substrate: The platform for blockchain innovators
Apache License 2.0
8.39k stars 2.65k forks source link

Question: Can pallet initial state spec become more decoupled from chain spec to achieve something like a standard for migrations? #8062

Open 2075 opened 3 years ago

2075 commented 3 years ago

Currently a chain is setup with a genesis spec, which has to be carried along all runtime and node upgrades, maintaining compatibility for all future upgrades but already defining a lot of initial state. This could become heavy to maintain and centralised as this leads to extra work, whenever you want to update low level specs of pallets, where maybe not all of them are required for the chain to function properly at genesis.

In some frameworks a concept like Migrations is standardised, maintaining a transparent and unified flow to migrate a traditional DB.

How about decomposing this, and moving individual initial pallet state ( not the complete genesis ) into a pallet specific state spec, which will be migrated to after genesis.

Advantage would be, that pallets always self contain all required information to set themselves up, e.g. when adding a pallet, upon invocation of its individual state transition on chain, the state will be set up according to its initial spec.

Whenever you transition to a new spec for a specific pallet, you build a migration, which takes care prior to invocation, to migrate eventual previous state to reflect the new specification.

Another future advantage could be increased flexibility in terms of decomposing the runtime to enable individual pallets to be added or removed vs upgrading the whole runtime whenever something changes, which this could be a basic requirement for achieving this.

Do you believe this makes sense / is achievable with the current codebase? What would be the pitfalls and efforts?

Thanks for feedback

bkchr commented 3 years ago

Are you aware the chain spec is only the genesis and nothing more? Meaning every pallet change you apply using a runtime upgrade or changing the state through transactions doesn't change the chain spec. The chain spec is immutable.

2075 commented 3 years ago

yes, of course. this is not discussing the immutability of the initial chain genesis, but to decouple and offer a more unified way to a. migrate e.g. state when upgrading your chain and b. making pallets more selfcontained like packages, maybe towards more modular upgrade capabilities. this is not about mutating chain spec.

bkchr commented 3 years ago

We already provide some tools to write migrations. In the end it can probably never be that easy as a migration of a SQL database. Rust and the way you can access the state is just not as constrained as SQL, making it a rather hard problem to solve.

What is a self contained package? What should this achieve? How should that work?