Cheksum errors after upgrade

[stevesaliman]

I just upgraded the liquibase plugin from version 0.6.1 to version 0.7, which uses a newer version of the groovy-liquibase DSL. I am now get checksum errors when I try to run "gradle update". The error is with an old changeSet:

changeSet(id: '25.9.7-base-action_output-constraints', author: 'various (generated)') {
    addForeignKeyConstraint(
        baseColumnNames: 'Action_Output_Type_Id',
        baseTableName: 'action_output', constraintName: 'FK_Action_Output_Action_Output_Type_Id',
        deferrable: false,
        initiallyDeferred: false,
        onDelete: 'NO ACTION',
        onUpdate: 'NO ACTION',
        referencedColumnNames: 'Id',
        referencedTableName: 'action_output_type',
        referencesUniqueColumn: false)
  }

The crux of the problem seems to be with that last bit. I added some logging to Liquibase's MD5Util class to see what was being passed for checksumming. In version 0.7.3 of the DSL, the referencesUniqueColumn parameter is not passed to the checksum routine, but in 0.7.6, it is. This causes a checksum mismatch. I noticed that ChangeSetDelegate.groovy line 269 changed between these versions. What I can't quite figure out is if this is a bug introduced, or a bug fixed.

Can you help shed some light on the subject?

Steve

[mbruner]

Hi Steve,

That was a some kind of bug fixing: some columns in a set of changes were being skipped by parser. You could find related changes in this commit: https://github.com/tlberglund/groovy-liquibase/commit/37ff79e4c790ea05e8fc16d56c3273ca2c251350

Max

[stevesaliman]

I had a feeling that was the issue.

there is still an issue for users that ran changesets with the old code. The new code calculates different hashes, so future updates fail.

A suitable workaround for me was to edit the old changesets and remove the referencesUniqueColumn parameter. Other changes will require updating the databasechangelog with new hashes.

Steve

[mbruner]

You have several options:

• use old version that has bugs: incorrect processing of some parameters such as ignoring the referencesUniqueColumn parameter
• use newer version and run clearCheckSums command

I want to reiterate the fact that earlier it did not work properly and if you have changesets with changed checksums you should check your DB for errors in schema.

As for referencesUniqueColumn parameter - it's used with Oracle DB only and shouldn't be used with other databases.

P.S. You may clear checksums for particalur changesets by editin databasechangelog table manualy.

Max

[stevesaliman]

I had a feeling that the change was a fix to previous bad behavior.

In our case, we are on MySql, so the referencesUniqueColumn parameter can be safely removed from the change sets, which BTW were created by the old plugin's generateChangelog task.

Unfortunately, clearCheckSums causes issues of its own if you have any changeSets that use sqlFile.

The clearCheckSums task itself works fine, but subsequent update tasks will fail. It appears that Liquibase starts by going through all the entries in the databasechangelog table, recalculating checksums for any changeset that is missing one. Then the update operation begins. Update appears to go through all the changesets to see what needs to run. Part of that process is comparing the checksum of the changeset with the one in the database, and that is where the problem lies.

I have Liquibase logging whenever it generates a checksum. On the first pass (to recalculate missing values), we calculate a new checksum for " " for all changesets with sqlFile. On the second pass (the update), we calculate the checksum for the actual SQL inside the referenced sqlFile, which hashes differently.

The net of this is that once you run clearCheckSums on a `sqlFile`` changeset, you'll never be able to get an update to work right.

I'm still looking into why this happens. My hunch is that it is a Liquibase bug. If so, I'll file an issue with that project, but I thought I'd mention it here for anyone who follows this thread later.

Steve

[stevesaliman]

I'm still looking into the `clearChecksums issue, but I can confirm that I'm seeing this in the old version of the plugin as well.

Steve

[stevesaliman]

I think I've found the cause of the problem with updating null checksums, and I think it is a Liquibase bug. Liquibase starts by loading all the changesets into memory. It then checks the databasechangelog table for missing checksums. When it finds a missing checksum, it updates the table with the checksum in the changeset.

For a sqlFile changeset, the checksum is computed from the SQL in the referenced file. That file is loaded by the init method, but Liquibase doesn't call the init method until it is ready to execute the changeset, which means we have no SQL at the time we update the missing checksum.

I think the ultimate fix should be in Liquibase, and I'll file a bug report there. In the meantime, we could workaround this by calling init ourselves after creating a sqlFile changeset.

What do you think?

Steve

[stevesaliman]

Update: The bug I filed in the Liquibase core has been assigned a fix version of 3.0.0. This release has been planned for a while now (the latest release was a year ago), so I'm hoping it will be out relatively soon.

[stevesaliman]

This has not been fixed in Liquibase, but I've worked around it in the Groovy DSL.