search

khonsulabs / bonsaidb

A developer-friendly document database that grows with you, written in Rust
https://bonsaidb.io/
Apache License 2.0
1.01k stars 37 forks source link

Add format-breaking migration system #252

Open ecton opened 2 years ago

ecton commented 2 years ago

With #225, any collections that are storing a Header or Revision will have an invalid Header/Revision upon upgrade. These types of upgrades will be rare, but a user shouldn't be able to update their Cargo.toml and have the database break without you first being alerted.

The basic idea I have is to introduce an internal version number representing major format changes. StorageConfiguration can have a new property: allow_storage_version. Users will receive an error upon opening a storage if the database needs to be updated and allow_storage_version is less than the current version number.

We should also consider adding an optional way for a Schema and Collections to opt-in to these checks on initialization as well. This would allow authors of plugins to add sanity checks on major upgrades. If we go this far, we should give a way for a user to override the compatibility so that an under-maintained Schema doesn't block upgrading if they take the burden of testing and validating it themselves.

The last principle I want to have is these migrations should generate a manifest that can be referenced in the future. For example, the upgrade manifest for #225 would be able to provide a map of old Revision to new Revision for each collection. This would allow users to still recover even if they bypassed all of the aforementioned safeguards.

zamazan4ik commented 2 years ago

Also, please document somewhere things like "how to upgrade and downgrade database between versions" and maybe some notes regarding policies about forward and backward compatiblity guarantees