Verify Update of app works when new versions have more actions and events added to it

[GoogleCodeExporter]

We have to verify if this works.

Upload a version 1 app on the omnidroid market with few events and actions. Add 
some rules and 
save them to the db.

Create a version 2 of the app and publish the update on the market. Version 2 
should have more 
events and actions than version 1.

Make sure when the user updates the app, he/she sees more events/actions and 
the rules are still 
there if possible.

Original issue reported on code.google.com by anurchan...@gmail.com on 20 May 2010 at 8:32

[GoogleCodeExporter]

Start to take a look at this issue.

Original comment by renc...@gmail.com on 9 Jun 2010 at 12:11

[GoogleCodeExporter]

It works in the emulator as long as DbHelper.DATABASE_VERSION is different 
between version 1 app and version 2 app. This also means that the features 
added in issue 76 won't work without uninstalling and re-installing Omnidroid. 
This can be easily remedied by incrementing the DbHelper.DATABASE_VERSION, but 
the current DbHelper.onUpgrade() deletes all existing tables, including rules 
created by the user. To prevent deleting user data as much as possible, I think 
we should somehow provide a mechanism for gracefully updating the database. I 
have an idea inspired from Rails DB Migration, and below is a pseudo code:

  @Override
  public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {

    switch (oldVersion) {
    case 1:
    case 2:
    case 3:
    case 4:
    case 5:
      /* Version 5 is the current version of our DB. case 1-5 basically
       * behaves just like the original code.
       */
      dropTables(db);
      /* Migration is just a class with static methods which are added
       * every time we make changes on the db schema or data we 
       * pre-populate on it.
       */
      Migration.initVersion(db);
    case 6:
      Migration.addCallEndEvent(db);
    case 7:
      Migration.addYYYTable(db);
    case 8:
      Migration.addXXXEvent(db);
    }
  }

What do you guys think? I want to hear your opinions because this is going to 
greatly affect on how we make changes on the underlying DB in Omnidroid. Thanks!

Original comment by renc...@gmail.com on 9 Jun 2010 at 6:38

• Changed state: Started

[GoogleCodeExporter]

This sounds like a plausible solution.  We'll probably need additional 
functions as necessary, possibly: updateUserRules, updateEvent, updateTable, 
updateAction, etc.

If we do it this way (or any other way without just dropping all DBs) we need 
some way to make sure that database changes reflected in code elsewhere
1) Will always result in a Migration function being implemented here as well
2) The DB_VERSION being bumped

Is there a good way to guarantee this?

I could probably build into the autobuild functions 
(http://code.google.com/p/omnidroid/source/browse/#svn/tools/autobuild) a check 
on commit that looks for DB changes and then checks to make sure the DB_VERSION 
and onUpgrade functions were updated as well.  I don't imagine the DB classes 
(names changes/additions/subtraction) will change often enough that providing 
this would be too much of an administrative headache.

A second option would be to move the database schema out of the codebase 
entirely and have the code for the database classes generated on schema 
changes.  Effectively removing the possibility of _forgetting_ to update the 
DB_VERSION.

Other options?  Has anyone looked through google groups to see what other 
people suggest?

Original comment by case.and...@gmail.com on 9 Jun 2010 at 8:10

• Added labels: Maintainability

[GoogleCodeExporter]

Response to Comment 3:

One way of guaranteeing (1) is to refactor the existing code into:

private void migrateToLatest(SQLiteDatabase db, int startDbVersion) {
  switch (startVersion) {
    //same contents as above
  }
}

public void onCteate(...) {
  migrateToLatest(db, 1);
}

public void onUpgrade(...) {
  migrateToLatest(db, oldVersion);
}

The only way of guaranteeing (2) for this approach is to include testing if the 
changes are reflected when one upgrades from an older version to the new one in 
the development process. Or I can write a ruby script for the svn pre-commit 
hook to check if there are changes made on the model or model/db directory and 
if the version is not changed, the script will ask the commiter if he wants to 
proceed checking in the codes.

Original comment by renc...@gmail.com on 9 Jun 2010 at 9:56

[GoogleCodeExporter]

Oops, I just realized that we don't have access to the actual repository in 
Google Code, making the pre-commit hook impossible to do.

Original comment by renc...@gmail.com on 9 Jun 2010 at 11:12

[GoogleCodeExporter]

I like #4.(1), but a potential problem with I see is that over time you'll end 
up with a lot of database updates occurring on install.  For example, if I 
install the app at DB_VERSION=24.  It will got through 24 sets of database 
updates instead of just just installing the database with the most up-to-date 
schema.  As that version gets higher, the amount of time it could take to do 
all the database modifications could actually become significant.  It might be 
worth it to do it this way until we notice significant issues though.

Original comment by case.and...@gmail.com on 9 Jun 2010 at 11:45

[GoogleCodeExporter]

I think it's not going to have a big difference in performance as long as we 
don't have a lot of versions that result in doing nothing like dropping tables, 
removing rows or columns. For example, creating a table in ver 12 and dropping 
it at ver 21 results into doing nothing.

Original comment by renc...@gmail.com on 10 Jun 2010 at 12:51

[GoogleCodeExporter]

Hi, I think you can look at the "void repopulate(SQLiteDatabase)" method in the 
DbHelper. I think it can be used for your purpose. It recreating all the tables 
except for those who's related to user saved rules.

Original comment by huangf100 on 10 Jun 2010 at 12:36

[GoogleCodeExporter]

Yeah, I guess we could use repopulate(), but there is a minor issue though. 
Because we are dropping all tables and repopulating the data again, we cannot 
guarantee that the ids of the new table entry (ie, primary keys) would still be 
the same as before, so if there are existing entries in the rules table, we 
cannot guarantee the foreign keys in the rules table are pointing to the same 
entry as they are before we "repopulated" the database. We are basically in the 
mercy of the android sqlite library and relying that it should return the same 
key as we did versions before. It has a potential to become a maintenance 
nightmare. What do you think?

Original comment by renc...@gmail.com on 11 Jun 2010 at 2:21

[GoogleCodeExporter]

Code review for this issue: http://codereview.appspot.com/1683041

Summary of changes.
1. Renamed DbData to DbMigration
2. Removed DbHelper.prepopulate() as it is no longer used. Another reason for 
removing it is if the migration method would push through, its usage is 
discouraged.
3. Moved contents of DbHelper.createTable() to DbMigration.initialVersion()
4. Added getters for the create statement for all DB adapter classes as that 
string is package protected and cannot be directly accessed in DbMigration.
5. Factored out the phone call ended event to a new method called 
addCallEndEvent() to demonstrate how to change the DB using the proposed 
migration method.
6. In the current migration implementation, newly added events will appear as 
the last elements on the root event activity, to make a better presentation, I 
decided to modify the codes to list the events alphametically.

Original comment by renc...@gmail.com on 14 Jun 2010 at 3:45

[GoogleCodeExporter]

Excellent job!  For the guaranteeing the following, I'll open up another ticket.

> 2) The DB_VERSION being bumped
> 
> Is there a good way to guarantee this?
> 
> I could probably build into the autobuild functions 
> (http://code.google.com/p/omnidroid/source/browse/#svn/tools/autobuild)
> a check on commit that looks for DB changes and then checks to make sure
> the DB_VERSION and onUpgrade functions were updated as well.  I don't
> imagine the DB classes (names changes/additions/subtraction) will change
> often enough that providing this would be too much of an administrative
> headache.

Original comment by case.and...@gmail.com on 14 Jun 2010 at 5:46

[GoogleCodeExporter]

Checked in on r744.

Original comment by renc...@gmail.com on 15 Jun 2010 at 2:05

• Changed state: Fixed