Configuration example for kotlin-dsl

[davinkevin]

Hello,

I would like to use this plugin in a gradle env, but with the kotlin-dsl which seems to be complicated due to implementation made in this plugin (usage of Closure which seems to not be the best solution from the gradle website documentation).

I've tried a lot of things but without success to allow a simple changelog execution. My last attemps is this and doesn't do anything. I don't know if I'm far or close to the goal so I preffer ask you some help

liquibase {
    activities(
            closureOf<NamedDomainObjectContainer<Activity>> {
                mapOf(
                        "main" to closureOf<Activity> {
                            mapOf(
                                    "changeLogFile" to "src/main/resources/db/changelog.xml",
                                    "url" to "jdbc:h2:mem"
                            )
                        }
                )
            }
    )
}

Thanks for your help

[stevesaliman]

Thank you for raising this issue.

From a quick glance at the linked documentation, it looks like the issue is in the plugin itself. It does indeed expect a closure to create each activity. I'll need to look at the documentation more closely and see if I can update the plugin according to its recommendations.

Until I'm able to update the plugin, it probably won't work right with Kotlin build files.

[Whoops]

I stumbled on this issue trying to get the same thing working. After some trail and error, this is what I landed on, and it seems to work reliably, and the code is pretty clean.

liquibase {
    activities.register("main") {
        val db_url by project.extra.properties
        val db_user by project.extra.properties
        val db_pass by project.extra.properties
        this.arguments = mapOf(
                "logLevel" to "info",
                "changeLogFile" to "src/main/resources/migrations/changelog.yml",
                "url" to db_url,
                "username" to db_user,
                "password" to db_pass
        )
    }
    runList = "main"
}

[alexvas]

@Whoops how do you configure runtime dependency on liquibase?

UPD: Oh, I see. It is just liquibaseRuntime dependency. No more buildscript classpath mangling.

[Chubacca]

When using the code from @Whoops I ran into this issue even on liquibase 3.6.3. If I run the exact same thing in just a normal gradle file (not kotlin DSL) I don't run into these issues. This doesn't work in a build.gradle.kts file:

liquibase {
    activities.register("development") {
        this.arguments = mapOf(
            "changeLogFile" to "src/main/db/db.yaml",
            "url" to "jdbc:postgresql://localhost/test",
            "username" to "test",
            "password" to "test"
        )
    }
}

But this DOES work in a build.gradle file:

liquibase {
    activities {
        development {
            changeLogFile 'src/main/db/db.yaml'
            url 'jdbc:postgresql://localhost/test'
            username 'test'
            password 'test'
        }
    }
}

I ended up fixing it by adding these two lines into my build.gradle.kts:

liquibaseRuntime("ch.qos.logback:logback-core:1.2.3")
liquibaseRuntime("ch.qos.logback:logback-classic:1.2.3")

I'm not sure what's going on or why it might be different, but now everything works.

[mstawick]

Thanks @Chubacca , your comment really helped me (BTW, Java13, Gradle 6.1).

[fabio-filho]

@Whoops , I don't know why, but the delegation does not works. I only get the excepted value when I access the properties directly by the key.

[charlesganza]

To those using test containers:

val testContainer = PostgreSQLContainer("postgres:latest")
    .apply {
        withDatabaseName("liquibase-db")
        withUsername("user")
        withPassword("testing")
        start()
    }

liquibase {
    activities.register("main") {
        this.arguments = mapOf(
            "logLevel" to "info",
            "classpath" to "${project.rootDir}/application/src/main/",
            "changeLogFile" to "resources/db/changelog-main.xml",
            "url" to testContainer.jdbcUrl,
            "username" to testContainer.username,
            "password" to testContainer.password,
            "driver" to "org.postgresql.Driver"
        )
    }
    runList = "main"
}

Extra config when using JOOQ:

tasks.named("generateJooq").configure {
    dependsOn(tasks.named("update"))
    doLast {
        testContainer.stop()
    }
}

[landsman]

There is still not a good example I see. What am I missing is also option how to load configuration from properties.

[nbrugger-tgm]

@charlesganza also this example is problematic because it eagerly starts the container in the configuration phase. This means that even if you do ./gradlew :tasks it will start a docker database container or even worse, fail if the person has no internet or no docker

[charlesganza]

@nbrugger-tgm Yes, this example assumes the user has Docker installed and is connected to the internet. I added it in case it could help someone else, and it works well enough in my case. You're more than welcome to improve on this approach :)

[nbrugger-tgm]

De I am currently in the phase "bashing my head against the wall trying to wire 4 technologies together"

I will most probably write my own shitty gradle plugin that does the testcontainers + liquibase + jooqGen chain. I (funny enought) already have a completely working process but the code is very confusing and not in one place

For everyone who does not care and just want to make it work:

1. Copy the class LiquibaseDatabase from the jooq-meta-liquibase project.
2. Rip out the old connect method 3.replace it with your specific Database container start
3. Wire it together with the jooqAutogen plugin as databaseclass

This is a Very watered down version you will still need to do things on order to have all the dependencies, and so on compile in the right order so that jooq has your compiled testcontainers dB. If anyone is interested I can share more.

BUT there is light on the horizon!! If you look at the master branch of jooq you will see a JOOQ-gradle-plugin folder. Not yet documented but maybe in a far off dream world it will support our useCase as maven already does

[nbrugger-tgm]

@charlesganza , @landsman I created a jooq meta generator extension over at https://github.com/nitok-software/jooq-liquibase-extension

[landsman]

@nbrugger-tgm are you using spring framework, spring boot? I don't like that configuration with db details in gradle task when I have this convered already in property / yml files for spring. It would be great to support it.

[nbrugger-tgm]

@landsman spring has little to do with this. Spring is a runtime framework. Jooq autogenes is purely compile time the database you use for prod&testing has nothing to do with your compile process.

The jdbc:tc URL only exists to tell testcontainers which docker image to use for the autogeneration.

Could you specify how you think this should work with spring?

[landsman]

Could you specify how you think this should work with spring?

just put there path to property file and support replacement for env variables, for example:

spring.datasource.username=${DB_USER}
spring.datasource.password=${DB_PASS}
spring.datasource.url=jdbc:postgresql://${DB_HOST}:5432/mydb

[nbrugger-tgm]

This makes no sense to me.

Why would you need your Gradle/build process to connect to a real dB? Why would gradle need password and username on an empheral container?

The string jdbc:tc:mysql:8:/localhost does not mean "connect to localhost". The "tc" stands for testcontainers and makes it so that when JDBC gets the URL testcontainers will start a temporary mysql:8 docker Container on localhost that is used to generate the jooq schema. If you look at the "example" in my repo you see that there is no user or password because the container doesn't need it. You can even remove the localhost part. Here is the official docs : https://java.testcontainers.org/modules/databases/jdbc/

[landsman]

I can use it to generate diff with real db for example.

[nbrugger-tgm]

Oh I see what you mean. My plugin is meant for Jooq (in conjunction with liquibase) so it only works for the jooq autogeneration. This is not a general/generic "liquibase-gradle" thing. I should have stated this more clearly since we are in a liquibase issue and not jooq!

For a purely liquibase setup connecting to a real database can of course be beneficial. But the specific use case my plugin works for (jooq autogeneration with liquibase and testcontainers) does not benefit from a Prod dB.

On a more general note, in my opinion it's not the job of a plugin author to support spring specific variable substitution or some other format. The way gradle works you can find a plugin or write it yourself that can read a yml/properties file do the substitution and then feed it into said plugin. (with my plugin this would work as long as you encode it as a full URL including user and password but as I said this is not a use case for my plugin anyway)

Example

somePluginThatRequiresPasswordAndUser {
    var props = new Properties();
    props.load(FileInputStream("pathRoFile);
    jdbcUrl= props.get("spring.data.nameOfYourSpecificDatasource.url");
}

This example doesn't do substitutions but it shouldn't be to hard to get it working and there is probably a library/gradle plugin. But reading a spring config file will full spring SPEL support is not that easy and a different issue.

[stevesaliman]

If you put the properties in gradle.properties, you don't need to load a file, you can just refer to the project property. For example, if you have a property named jdbcUrl in gradle.properties, your liquibase block can look like this:

liquibase {
    activities {
         main {
            driver project.ext.jdbcDriver
            url project.ext.jdbcUrl
            username project.ext.jdbcUsername
            password project.ext.jdbcPassword
            changeLogFile "master.groovy"
        }
}

If you need different properties for different environments, you could use https://github.com/stevesaliman/gradle-properties-plugin, which lets you store different values in various gradle-.properties files

[asm0dey]

@nbrugger-tgm the jitpack version does not build :(

[nbrugger-tgm]

@asm0dey fixed it: https://github.com/nitok-software/jooq-liquibase-extension/pull/1 should work now 👍🏻

Also if you have problems with my code/extension you should open an issue there and not here

[sballance]

Are there ever going to be examples for this? I've been struggling to get my liquibase gradle configuration set up correctly in my build.gradle.kts file. I was using the jetbrains liquibase plugin and wanted to switch to this plugin but I can't replicate my setup. This setup creates tasks for each registered activity, e.g. liquibaseMainUpdate and liquibaseLocalUpdate.

plugins {
    id("org.jetbrains.gradle.liquibase") version "1.5.2"
}

dependencies {
    liquibaseRuntime("org.liquibase:liquibase-core:4.27.0")
    liquibaseRuntime(libs.h2)
    liquibaseRuntime(libs.ojdbc)
    liquibaseRuntime(libs.mssqlJdbc)
}

liquibase {
    activities {
        all {
            properties {
                changeLogFile.set("./changelogs/changelog.yaml")
                changeSetAuthor.set("me")
                defaultSchemaName.set("my_schema")
            }
        }
        register("main") {
            properties {
                driver.set("oracle.jdbc.Oracle")
                url.set("jdbc:oracle:thin:@//localhost:1520/MYDB")
                username.set("me")
                password.set("pass")
            }
        }
        register("local") {
            properties {
                driver.set("org.h2.Driver")
                url.set("jdbc:h2:mem:test;DB_CLOSE_DELAY=-1")
                username.set("me")
                password.set("pass")
            }
        }
    }
}

[landsman]

@sballance I had no idea about the existence of Jetbrains plugins, thanks for the link! Why do you want to switch?

By the way, I left this alone and started using JPA Buddy.