search

celestiaorg / celestia-app

PoS application for the consensus portion of the Celestia network. Built using celestia-core (fork of CometBFT) and the cosmos-sdk
https://celestia.org
Apache License 2.0
328 stars 261 forks source link

Prototype versioned app module rolling upgrades #3625

Open evan-forbes opened 2 days ago

evan-forbes commented 2 days ago

We are attempting to gather more information on a flexible and simpler approach to rolling upgrades and also single binary syncs. This approach involves versioning the application struct (and potentially the rest of our modules) in some fashion so that at an upgrade height, it can be swapped out with a different go moduled version of the app.

In theory, for state machine upgrades, this results in something similar to restart upgrades in that we can make arbitrary consensus breaking changes (such as importing a semver minor bump but still consensus breaking version of IBC) while not having to reconnect to peers or completely reload the DB indexed into memory.

AC

To close this issue, we should create a prototype that takes a similar approach to the above to perform a rolling upgrade over 4 nodes in knuu. Ideally there is also a migration included in this upgrade.

cmwaters commented 2 days ago

What I basically had in mind is a multiplexer which adheres to the ABCI interface at one end and proxies calls to the relative state machine. In a way it can be seen as a sort of middleware. Each state machine is responsible for the mechanism they have for bumping the app version. The multiplexer simply stores it to disk somewhere and listens for changes via EndBlock.

cmwaters commented 2 days ago

As a minimal prototype, I had thought of creating a fork, bumping to v3 and having the wrapper import v1 and v2 of celestia-app. In the future we may want to go.mod these state machines for more fine granularity imports, but to start with I think we can leave it as is and see what kind of binary size that gives us.