search

golang-migrate / migrate

Database migrations. CLI and Golang library.
Other
14.99k stars 1.38k forks source link

Support for Spanner DML #135

Open lc-jordanfelix opened 5 years ago

lc-jordanfelix commented 5 years ago

Is your feature request related to a problem? Please describe. We would like to be able to load non-transactional data using this tool.

Describe the solution you'd like Running DML statements as well as DDL statements.

Describe alternatives you've considered Using a separate tool. It would be much more useful for us to be able to run both DML and DDL in a migration-like manner using one tool.

Additional context Syntax error on line 1, column 1: Encountered \'INSERT\' while parsing: ddl_statement\nSyntax error on line 1, column 1: Expecting \'EOF\' but found \'INSERT\'

Syntax error on line 1, column 1: Encountered \'UPDATE\' while parsing: ddl_statement\nSyntax error on line 1, column 1: Expecting \'EOF\' but found \'UPDATE\'

We see that these are coming from "cloud.google.com/go/spanner/admin/database/apiv1".DatabaseAdminClient.UpdateDatabaseDdl(...) in migrate/database/spanner/spanner.go and that there may not be support for DML in the same google library, but we hope you might have a solution.

Thanks!

lc-jordanfelix commented 5 years ago

For anyone interested in this as well, we currently are using this script as an alternative for our Spanner DML files. You can find it in our public repo.

We are improving it as we go.

juztin commented 3 years ago

Oh man, just stumbled across this after getting everything all integrated and working within a pipeline. Is it possible to add an asterisk to "Spanner" within the README, stating that it's only partially supported? Not being able to run dml statements is kind of a big deal.

dhui commented 3 years ago

If anyone would like to add DML support for spanner, I'd be happy to review it.

The PR would need to have tests and be backwards compatible. e.g. shouldn't break existing users of the migrate spanner db driver

jorygeerts commented 2 years ago

@dhui Would an approach where each migration file may contain either DDL or DML but not both be acceptable?

If yes, what approach of detecting which a given migration file contains would you prefer, parsing the contents, or adding some kind of "marker" to the filename in case of DML (not required for DDL, to keep things backwards compatible)? Or something else I haven't thought of. :)

plemanach commented 2 years ago

Hi @dhui, I created this PR that add DML for spanner, let me know your thoughts. Thanks, Patrice

afarbos commented 1 year ago

I /fresh created a new PR that can be merged following all your guidelines @dhui, all lines added are tested and it's fully backward compatible with no change from the user pov.