vincent-herlemont / native_db

Drop-in embedded database in Rust
MIT License
433 stars 17 forks source link

Cannot migrate from redb1 while also having pending migrations (newer/additional models defined) #194

Closed Gawdl3y closed 2 months ago

Gawdl3y commented 3 months ago

The library doesn't seem to be able to handle migrating an old database from redb1 to redb2 when there are also pending model migrations to be done.

I have a single model with ID 1 and versions 1 and 2. The old database is from native_db version 0.5, and at the time only version 1 of the model existed. While attempting to open the old database with native_db 0.7, native_db panics at this line:

thread 'tokio-runtime-worker' panicked at C:\Users\Schuyler\.cargo\registry\src\index.crates.io-6f17d22bba15001f\native_db-0.7.0\src\upgrade\redb1_to_redb2.rs:35:14:
called `Result::unwrap()` on an `Err` value: TableDoesNotExist("1_2_id")
stack backtrace:
   0: std::panicking::begin_panic_handler
             at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library\std\src\panicking.rs:645
   1: core::panicking::panic_fmt
             at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library\core\src\panicking.rs:72
   2: core::result::unwrap_failed
             at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6/library\core\src\result.rs:1654
   3: enum2$<core::result::Result<redb::table::ReadOnlyTable<native_db::db_type::key::inner_key_value_redb1::DatabaseInnerKeyValue,ref$<slice2$<u8> > >,enum2$<redb::error::TableError> > >::unwrap
             at /rustc/9b00956e56009bab2aa15d7bff10916599e3d6d6\library\core\src\result.rs:1077
   4: native_db::upgrade::redb1_to_redb2::upgrade_primary_table
             at C:\Users\Schuyler\.cargo\registry\src\index.crates.io-6f17d22bba15001f\native_db-0.7.0\src\upgrade\redb1_to_redb2.rs:33
   5: native_db::upgrade::redb1_to_redb2::upgrade_redb1_to_redb2<ref$<ref$<ref$<std::path::PathBuf> > > >
             at C:\Users\Schuyler\.cargo\registry\src\index.crates.io-6f17d22bba15001f\native_db-0.7.0\src\upgrade\redb1_to_redb2.rs:102
   6: native_db::upgrade::upgrade<ref$<ref$<std::path::PathBuf> > >
             at C:\Users\Schuyler\.cargo\registry\src\index.crates.io-6f17d22bba15001f\native_db-0.7.0\src\upgrade\mod.rs:13
   7: native_db::database_builder::Builder::open<ref$<std::path::PathBuf> >
             at C:\Users\Schuyler\.cargo\registry\src\index.crates.io-6f17d22bba15001f\native_db-0.7.0\src\database_builder.rs:94
   8: resolute::db::ResoluteDatabase::open<std::path::PathBuf>
             at C:\Users\Schuyler\Projects\resolute\crates\resolute\src\db\mod.rs:26
   9: resolute_app::init::async_fn$0::closure$1
             at .\src\main.rs:262
--snip--

Commenting out the new version of the model and the code to trigger the migration allows the migration from redb1 to redb2 to succeed and thus the database opens successfully. After that has happened, the code can be uncommented and then the model migrations will proceed as normal.

This implies that if an application is switching from native_db 0.5 to >=0.6 while at the same time introducing new models or new versions of models, the application has to first open the database with the original set of models defined, allow the redb migration to happen, then close it and reopen it with the current (full) list of models. This is pretty kludgey since that code can essentially never be removed since a user could be on an old version for a while, then update to a much newer version and not hit any intermediate version that has the manual migration code in it.

vincent-herlemont commented 3 months ago

@Gawdl3y Thank you for the feedback, I will fix that!

vincent-herlemont commented 2 months ago

@Gawdl3y the release 0.7.1 has been made with the fix. If you encounter any other issues, please don't hesitate to let me know.