Packaging up migrations

[totallymike]

Hi there.

We've been working to add forklift to a fairly new project recently, and for the most part it's been going well enough. I worked around the scala binary incompatibility by building my own JARs and making them available in the class path.

However, I'm now running into a new problem: we intend to ship our project in a docker container using sbt-native-packager, and we're discovering that (at least to my fairly-new-to-the-java-ecosystem understanding) this entire concept is incompatible with Forklift. Symlinks with absolute paths combined with the Summary object have been rather difficult obstacles for me to overcome.

Have you encountered this, or do you have any insight into how one can package up the code such that we'd be able to bundle up the migrations and run them against our production databases? For what it's worth, running the migrations against the production database from a local machine is not an option.

Thanks very much for your help. I'd be glad to answer any further questions to the best of my ability.

[lastland]

Hi,

Thanks for reporting this! This is indeed a problem since Forklift requires changing the code (it changes the generated code, which application code depend on). The best way I can think of is to run the source code with sbt on the server (can be automated by using git hooks?). However, I don't have an easy solution if that is not an option. Perhaps we can come up with some way to support packaging, but I'm not sure if we can support that in near future.

@cvogt I wonder if you would have any suggestions?

[cvogt]

I haven't followed forklifts details well enough to have a super informed opinion about this, but I believe that is is probably a very good idea for Forklift to document a suggested way of handling production deployments (and support all necessary steps). Here is one way to do it, which could server as an initial draft:

• Developer writes migration
• Developer applies migration to his local dev db
• Forklift updates local Scala schemas
• User compiles and tests user code against new schema and fixes errors.
• Optional: User reverts migration and also tests new code against old schema.
• User commits migration code, changes to code base and newly generated schema code.
• User pushes the commits, which triggers integration server to act (circleci or whatever).
• Integration server compiles the code, bundles it as a jar, tests it against a database with the old schema before the migration (can be a mirror of production if tests are read-only).
• Integration server applies the migration against a test database and tests the code against this database with the new schema, too.
• On success, integration server deploys new code and triggers the production database migration. The easiest would be running the migration on the integration server via sbt. If that's really impossible for some reasons (e.g. migration duration), it might be easier to invoke the migration via the bundled jar, which Forklift would have to allow and require the migrations to be in bundles the jar.

Variations:

• Instead of committing generated code, you can let the deployment server generate it against a test database.
• Integration server can deploy to a staging environment first. Only after deploy to production.
• Integration server can first migrate and deploy afterwards, but that requires the old code to work on the new schema, which also needs to be tested.
• If migrating the production database fails for some reason the previous version of the code should probably be re-deployed.

Happy to discuss this further with you guys and flesh out the above to something clear and easily implementable which can be added to the documentation.

[lastland]

@cvogt That looks great, though I'm not sure why there's a step such that "integration server compiles the code, bundles it as a jar, tests it against a database with the old schema before the migration"?

[cvogt]

Somebody got to produce the class files, package them into a deployable format (e.g. a jar) and deploy them to the production server. A lot of people work in the way that they have a continuous integration server setup for this, that compiles the code, packages it into the jar and deploys it (i.e. copies it to the production server in some way), plus also runs the tests before or after packaging.

Not sure this answers your question?

[lastland]

@totallymike Just think about it a little bit more. Maybe modifying code is not required. We only need a way to apply all the migration files to the database: so the difficulty concerns the symbolic links. I think maybe the discussion in #28 would be helpful to you?

In particular, you can try customizing the part we handled migration files by making copys instead of symlinks. If you want to modify how migration files and migration summary are handled, you can try to override traits in this file(In particular, handleMigrationFiles for creating symlinks for migration files; writeSummary for putting migrations into the summary file). Does this make sense to you?

Let me know if there are any other problems and I would be happy to help figure out a way.

[lastland]

@cvogt but why "tests it against a database with the old schema before the migration"?

[cvogt]

@lastland you have to first deploy the code to your production server and then migrate the production db or first migrate and then deploy, right? In either case, between the two steps, either the old production code has to run against the new schema or the new production code has to run against the old schema, right? And I'd say, better test that that actually works by letting the code fire a bunch of queries against the other db version.

[totallymike]

Thanks for this feedback. This is tremendously useful for helping me isolate the problems and work around them :+1:

[totallymike]

I want to thank you again. Using the comments in this and the other issue referenced, I was able to get this working and usable in a couple hours. For what it's worth, here's a gist that encapsulates the changes I made: https://gist.github.com/totallymike/d969ecd61c3e134471e7c5492f722cc2

The only additional thing in this gist is that I allowed for adding package name to the generated MigrationSummary object. This means that ammonite can find my migration objects with its weird import magic, and in theory I'll be able to programmatically run my migrations from elsewhere in my project.

[cvogt]

Glad this was helpful @totallymike and thank you for sharing your solution! (Forked it here for the record https://gist.github.com/cvogt/f94c579ff8dad164d61a2be238f7e2d7)

Don't hesitate to reach out with further questions or experience reports, etc.

[lastland]

@totallymike Thanks very much for sharing your solution! I will try to add it to Forklift.

[totallymike]

I'm glad you both found it useful. Thanks again for your help and patience :)

[cvogt]

@lastland might be worth re-opening this as a documentation request or feature request

[nafg]

you have to first deploy the code to your production server and then migrate the production db or first migrate and then deploy, right? In either case, between the two steps, either the old production code has to run against the new schema or the new production code has to run against the old schema, right?

If you can tell me how to achieve that with Slick and types, or with fundamentally backwards-incompatible changes, I would love to hear.

I currently use Flyway, and it's a non-problem. I include Flyway as a library and on boot the application calls the migrate method. So the application is not running while migrations are being applied. Of course this doesn't scale horizontally, or allow blue-green deployments. I'm not sure the answer though --- see above.

I confess I'm not familiar enough with forklift to comment, but this seems like a fundamental requirement.

And running SBT on the server is just wrong. It's a build tool. Some of my servers don't even have enough memory to run SBT.

Running migrations from the outside as part of a deploy process might work for some, but relying on that would be really limiting.

[lastland]

Reopen this because it seems more discussions are necessary, and some related features/documentations are required.

[LysanderGG]

On success, integration server deploys new code and triggers the production database migration. The easiest would be running the migration on the integration server via sbt. If that's really impossible for some reasons (e.g. migration duration), it might be easier to invoke the migration via the bundled jar, which Forklift would have to allow and require the migrations to be in bundles the jar.

And running SBT on the server is just wrong. It's a build tool. Some of my servers don't even have enough memory to run SBT.

I couldn't find more info about these points. Is there any way to run the migrations without SBT, from the bundled jar?