A Liquibase extension adding support for Google Cloud Spanner. Include this in your application project to run Liquibase database migration scripts against a Google Cloud Spanner database.
Executing multiple small DDL statements on Cloud Spanner can take a very long time. This means that the standard recommendation to use as small changesets as possible with Liquibase is not always the best choice when working with Cloud Spanner. Instead, it is recommended to create changesets that group multiple DDL statements into one DDL batch. Use SQL change and batch the DDL using batch statements.
You can also create a single change set that contains multiple Liquibase changes (e.g. createTable)
and create these in a batch by adding a SQL command before and after the changes. See
create-multiple-tables.spanner.yaml for
an example.
clear-checksums command (https://github.com/cloudspannerecosystem/liquibase-spanner/pull/354)createSequence change typeCREATE [OR REPLACE] VIEW and DROP VIEW statementsClassNotFoundException for com.google.spanner.admin.database.v1.DatabaseAdminGrpcInstall Liquibase Community CLI from here. Alternatively, there are many other ways to install (e.g. brew on Mac OS/X). These examples were run with Liquibase 4.2.0.
Once Liquibase is installed, use the latest release or build your own liquibase-spanner-SNAPSHOT-all.jar and copy it into the Liquibase lib directory.
You can create a Spanner instance in the GCP console or use an emulator (with JDBC). You will also need to create a database for Liquibase to use.
Spanner CLI is a convenient way to access Spanner. It can be installed from here.
Configure the connection in the file liquibase.properties:
url: jdbc:cloudspanner:/projects/<project>/instances/<instance>/databases/<database>Using the Liquibase CLI the following ChangeLogs are examples of using Spanner. Review Liquibase best practices. In this example, changelog.yaml is used as the master changelog.
Run:
liquibase --changeLog example/changelog.yaml
| Example | Description |
|---|---|
| create-schema.yaml | Create schema, including interleaved tables, column options, and indexes |
| load-data-singers.yaml | Load data into Singers table from CSV |
| load-update-data-singers.yaml | Insert or update data in Singers table from CSV |
| add-lookup-table-singers-countries.yaml | Create countries table as a foreign key from Country field in Singers |
| modify-data-type-singers-lastname.yaml | Alter STRING datatype in Singers LastName column |
| insert.yaml | Insert rows into Singers table |
| delete.yaml | Delete rows from Singers |
| update.yaml | Update rows in Singers |
See the samples directory for specific integrations with other frameworks, such as Spring Boot.
The following Liquibase ChangeTypes are supported:
createTable, dropTable, addColumn, modifyDataType, addNotNullConstraint, dropColumn, createIndex, dropIndex, addForeignKeyConstraint, dropForeignKeyConstraint, dropAllForeignKeyConstraints, addLookupTable, createView, dropView
The following Liquibase ChangeTypes are not allowed with Cloud Spanner:
addAutoIncrement, addDefaultValue, addPrimaryKey, addUniqueConstraint, dropUniqueConstraint, createProcedure, dropDefaultValue, dropNotNullConstraint, dropPrimaryKey, dropProcedure, renameColumn, renameSequence, renameTable, renameView, setColumnRemarks, setTableRemarks, alterSequence
The following data DML ChangeTypes are supported:
insert, update, loadData, loadUpdateData
Note:
See limitations.md for a full list of limitations and unsupported features.
Some Spanner specific SQL, such as INTERLEAVE'd tables or column OPTIONS, require using Liquibase's modifySql. See create-schema.yml for an example of doing this.
In order to limit the number of schema updates in a 7-day period, run Liquibase with small changeSets. Alternatively, use SQL change and batch the DDL using batch statements.
There are DML limits for the number of rows affected during DML. The recommendation is to use partitioned DML. Using Spanner JDBC driver this can be configured using the AUTOCOMMIT_DML_MODE.
This has been implemented in some of the changeSet types such as mergeColumns, but not in all changeSet types such as delete.
There are a number of features that Spanner does not have such as sequences and stored procedures. The Liquibase extension will throw an exception during analysis of the changeSet in most cases, but not all. For example, a DELETE without a WHERE clause will fail in Spanner but not in the Liquibase extension.
| Gradle target | Description |
|---|---|
| test | Run mock and Spanner emulator tests |
| build | Build extension and run above tests |
| integrationTest | Run Spanner in GCP tests |
| jibDocker | Build a local runnable docker container |
Testing requirements:
There are two JARs built:
The first JAR is just the extension itself, while the second one includes all of the dependencies needed to run with Liquibase. Install the second one into the Liquibase lib directory.
For contributions please see contributing and our code of conduct.
If you have any questions, find a bug, or have a feature request please open an issue. Please note that this extension is not officially supported as part of the Cloud Spanner product.