@Migration version other than 0 not being called

DBFlow Version: 3.1.1 Issue Kind (Bug, Question, Feature): Bug

Instant Run is off Using android-apt

MacOS Sierra 10.12.5 (16F73) Android Studio 2.3.3 Nexus 7 Android 6.0.1

Description:

Hi I have a BaseMigration and AlterTableMigration based on the https://agrosner.gitbooks.io/dbflow/content/Migrations.html instructions:

app/src/main/java/my/app/database/migration/Creation.java:

@Migration(version = 0, database = MyDatabase.class)
public class Creation extends BaseMigration {
    public Creation() {
        super();
        Log.v("Creation", "Creation()");
    }

    @Override
    public void migrate(DatabaseWrapper sqLiteDatabase) {
        Log.v("Creation", "migrate()");
    }

    @Override
    public void onPreMigrate() {
        Log.v("Creation", "onPreMigrate()");
    }

    @Override
    public void onPostMigrate() {
        Log.v("Creation", "onPostMigrate()");
    }
}

app/src/main/java/my/app/database/migration/Alteration.java:

@Migration(version = 1, database = MyDatabase.class)
public class Alteration extends AlterTableMigration<MyModel> {
    public Alteration(Class<MyModel> table) {
        super(table);
        Log.v("Alteration", "Alteration()");
    }

//    @Override
//    public void migrate(DatabaseWrapper sqLiteDatabase) {
//        Log.v("Alteration", "migrate()");
//    }

    @Override
    public void onPreMigrate() {
        Log.v("Alteration", "onPreMigrate()");
    }

    @Override
    public void onPostMigrate() {
        Log.v("Alteration", "onPostMigrate()");
    }
}

app/src/main/java/my/app/database/MyDatabase.java

@Database(name = MyDatabase.NAME, version = MyDatabase.VERSION,
        foreignKeysSupported = true, generatedClassSeparator = "$")
public class MyDatabase {
    public static final String NAME = "mydatabase";
    public static final int VERSION = 1;
}

In the device: Settings->Apps->MyApp->Storage->CLEAR DATA Android Studio: Run->Debug...

Output:

08-26 13:54:55.234 123-123/my.app V/Creation: Creation()
08-26 13:54:55.237 123-123/my.app V/Alteration: Alteration()
08-26 13:55:01.136 123-123/my.app V/Creation: onPreMigrate()
08-26 13:55:01.136 123-123/my.app V/Creation: migrate()
08-26 13:55:01.533 123-123/my.app V/Creation: onPostMigrate()
08-26 13:55:01.533 123-123/my.app V/Alteration: onPreMigrate()
08-26 13:55:01.533 123-123/my.app V/Alteration: onPostMigrate()

But if I change the alteration's version to 1 to control the migrations' order of execution like:

@Migration(version = 1, database = MyDatabase.class)
public class Alteration extends AlterTableMigration<MyModel> {

In the device: Settings->Apps->MyApp->Storage->CLEAR DATA Android Studio: Run->Debug...

Output:

08-26 14:01:29.788 123-123/my.app V/Creation: Creation()
08-26 14:01:29.789 123-123/my.app V/Alteration: Alteration()
08-26 14:01:35.686 123-123/my.app V/Creation: onPreMigrate()
08-26 14:01:35.686 123-123/my.app V/Creation: migrate()
08-26 14:01:36.044 123-123/my.app V/Creation: onPostMigrate()

Alteration's migration is never called.

I've verified that both migrations are getting found and added to the database definition:

app/build/generated/source/apt/debug/com/raizlabs/android/dbflow/config/MyDatabasemy$Database.java

public final class MyDatabasemy$Database extends DatabaseDefinition {
  public MyDatabasemy$Database(DatabaseHolder holder) {
    List<Migration> migrations0 = new ArrayList();
    migrationMap.put(0, migrations0);
    migrations0.add(new Creation());
    List<Migration> migrations1 = new ArrayList();
    migrationMap.put(1, migrations1);
    migrations1.add(new Alteration(my.app.database.model.MyModel.class));
    //...
  }
  //...
}

The way in which migrations are found and processed is kind of opaque so I don't really know where to go from here. Also I don't understand why AlterTableMigration::migrate() is final so I can't override it. Should I only use DBFlow commands for alterations (so no raw SQL statements) which is why AlterTableMigration::migrate() is inaccessible? Or am I misunderstanding these other migration types and should only use BaseMigration? I feel like I'm overlooking something fundamental. Thanks for any help you can provide.

I just tried using BaseMigration on the alteration:

@Migration(version = 1, database = MyDatabase.class)
public class Alteration extends BaseMigration {
    public Alteration() {
        super();
        Log.v("Alteration", "Alteration()");
    }

    @Override
    public void migrate(DatabaseWrapper sqLiteDatabase) {
        Log.v("Alteration", "migrate()");
    }

    @Override
    public void onPreMigrate() {
        Log.v("Alteration", "onPreMigrate()");
    }

    @Override
    public void onPostMigrate() {
        Log.v("Alteration", "onPostMigrate()");
    }
}

It still didn't work though unfortunately.

I believe that I isolated the problem. @Migration(version = ... is 1 version behind the version stored in MyDatabase::VERSION. However ONLY that version is run in FlowManager.init(new FlowConfig.Builder(this).build()); (not the versions in between).

So for example, say a new application is released:

app/src/main/java/my/app/database/migration/Creation.java:

@Migration(version = 0, database = MyDatabase.class)
public class Creation extends BaseMigration {

app/src/main/java/my/app/database/MyDatabase.java

@Database(name = MyDatabase.NAME, version = MyDatabase.VERSION,
        foreignKeysSupported = true, generatedClassSeparator = "$")
public class MyDatabase {
    public static final String NAME = "mydatabase";
    public static final int VERSION = 1;
}

The above setup works and runs Creation::version = 0, bringing the app up to MyDatabase::VERSION = 1.

Now imagine a new version of the app is released and has a new migration added:

app/src/main/java/my/app/database/migration/Creation.java:

@Migration(version = 0, database = MyDatabase.class)
public class Creation extends BaseMigration {

app/src/main/java/my/app/database/migration/Alteration.java:

@Migration(version = 1, database = MyDatabase.class)
public class Alteration extends AlterTableMigration<MyModel> {

app/src/main/java/my/app/database/MyDatabase.java

@Database(name = MyDatabase.NAME, version = MyDatabase.VERSION,
        foreignKeysSupported = true, generatedClassSeparator = "$")
public class MyDatabase {
    public static final String NAME = "mydatabase";
    public static final int VERSION = 2;
}

For existing users, the above setup skips Creation::version = 0 (since the current database version is 1) and runs Alteration::version = 1 correctly, bringing the database to MyDatabase::VERSION = 2.

Now imagine that a new user downloads the newest app version but has never run it before. This time the app correctly runs Creation::version = 0, but skips Alteration::version = 1 since the current database version is 0, then sets the current database version to 2.

So as it stands now, I don't see how it's possible for fresh app installs to proceed through the migrations and bring the database up to the current version.

I lost about half a day figuring all of this out. Here are some other links that helped me:

https://github.com/Raizlabs/DBFlow/issues/614

https://agrosner.gitbooks.io/dbflow/content/Migration3Guide.html

https://agrosner.gitbooks.io/dbflow/content/GettingStarted.md

Am I misunderstanding how this works? I inherited this project written in DBFlow 3 but if 4 is ready then I can probably switch.

Also I recommend a more straightforward approach to migrations as practiced by PHP's Laravel framework:

https://laravel.com/docs/migrations

The above method drops the notion of database versions, instead using up() and down() callbacks to apply or rollback changes to the schema. Unfortunately Laravel doesn't have annotations so doesn't generate the migration code from the model attributes automagically. If you could find a way to do that with DBFlow, I think it would be really powerful.

If someone knows how to call FlowManager.init(new FlowConfig.Builder(this).build()); to make it apply all of the in-between migrations on a fresh install, please let me know because I'm still stuck. Hopefully this helps someone thanx!

Found another link about the issue:

https://github.com/Raizlabs/DBFlow/issues/427

It turns out that migrations are handled automagically by dbflow's ORM and annotations generating the needed migration code for initial database creation. However, the developer has to manually create migrations for subsequent releases so that the db schema reflects the new state of the ORM.

So the two possible flows are:

Creation 0 is called if no database exists. New db has schema based on models, but developers can put additional calls to create triggers etc here.
Update 1, Update 2, etc are called sequentially if database exists. These migrations need to update the schema to reflect ORM changes made since the Creation 0 release. Developers should also copy any triggers etc from Creation 0 to Update 1, Update 2, etc that are needed for the new model attributes (since Creation 0 was never called for those new attributes).

Here is a workflow with a few permutations to show adding columns or creating tables (note that if database has already had migrations applied then: Error Can't downgrade database from version Y to X will be thrown so cases where the db is ahead of the current release aren't shown and will never be encountered in practice):

Version 1 creates User
Version 2 adds indexed column to User
Version 3 creates NewModel

app release having database version 1:
    - Add User model (dbflow will generate user table code automatically).
    - Add Creation 0 that creates triggers or performs other tasks outside dbflow.
    since database doesn't exist (equivalent to version 0):
        - dbflow creates user table automatically.
        - Creation 0 called which creates triggers.
        - version set to 1
app release having database version 2:
    - Add User::column and User::@Index column to User model.
    - Add Update 1 migration which adds user.column and creates index on column.
    if database doesn't exist (equivalent to version 0):
        - dbflow creates user table having user.column and indexes it automatically.
        - Creation 0 called which creates triggers.
        - version set to 2
    if database exist (so must be at version 1):
        - Update 1 called which adds user.column and creates index on column.
        - version set to 2
app release having database version 3:
    - Add NewModel model.
    - Add Update 2 migration which adds new_model and creates trigger for it.
    if database doesn't exist (equivalent to version 0):
        - dbflow creates user table having user.column with index, also creates new_model.
        - Creation 0 called which creates triggers.
        - version set to 3
    if database exists and is at version 1:
        - Update 1 called which adds user.column and creates index on column.
        - Update 2 called which adds new_model and creates trigger for it.
        - version set to 3
    if database exists and is at version 2:
        - Update 2 called which adds new_model and creates trigger for it.
        - version set to 3

This cost us a day or two of trial and error to grok so it would be great if you could update the DBFlow instructions to explain this process with examples.

agrosner / DBFlow

@Migration version other than 0 not being called #1415