Full Flyway integration

AbsaOSS / atum-service

Apache License 2.0

6 stars 1 forks source link

Full Flyway integration #262

Closed benedeki closed 4 weeks ago

benedeki commented 1 month ago

Background

Steps to ensure Atum 0.3.0 release and deploy For now, we stick to Flyway migration tool (despite its drawbacks). It's infinitely better then doing DB changes manually.

Feature

Ensure that Flyway can be used in CD and then we stick to it.

Proposed Solution [Optional]

Solution Ideas:

Identify a basic level of Flyway scripts (somewhere where tables are already deployed)
Get the database manually to that state
Set Flyway watermark to that level (version of script last deployed)

benedeki commented 1 month ago

Solution steps:

[x] figure out SBT Flyway plugin configuration - configuration can be overridden by environment variables.
[x] Reorder scripts - first the utility functions, then tables, then all other functions
[x] set the flyway base - at the last table creation
[x] create drop script for all non-utility functions

benedeki commented 1 month ago

Solution for upgrade from v0.1.0/v0.2.0 to v0.3.0

call manually sbt -Dflyway.password=<postgres user password> -Dflyway.url=<target db url> flywayBaseline= // TODO
add to the deployment script sbt -Dflyway.password=<postgres user password> -Dflyway.url=<target db url> flywayMigrate

github-actions[bot] commented 4 weeks ago

Release notes: Reordered and fixed database structure scripts to allow Flyway usage for incremental migration of existing database (version 0.1.x). Known issues: there might be needed some manual steps if upgrading from database of version 0.2.0.