flyway migrations not working with fatJar assembled with > 1.0.60

[pabl0rg]

The way Kobalt handles manifests in fat jars changed between 1.0.60 and 1.0.62 and now flyway migrations don't work, even though resources (sql scripts) are being zipped in the same place.

Also, Kobalt includes more than one MANIFEST.MF in the assembled fat jar with kobalt > 1.0.60

The only difference between the two jars is that with 1.0.60, the MANIFEST.MF is empty and 1.0.62 has a MANIFEST.MF but whatever is in there makes Flyway stop finding migrations in the classpath

mixed-example> diff tmp-60/META-INF/MANIFEST.MF tmp-62/META-INF/MANIFEST.MF
0a1,11
> Manifest-Version: 1.0
> Implementation-Title: kotlin-stdlib-jre8
> Implementation-Version: 1.1.1
> Implementation-Vendor-Id: org.jetbrains.kotlin
> Built-By: teamcity
> Build-Jdk: 1.6.0_45
> Kotlin-Version: 1.1
> Kotlin-Runtime-Component: Main
> Created-By: Apache Maven 3.2.5
> Implementation-URL: http://kotlinlang.org/stdlib/kotlin-stdlib-jre8/
> Archiver-Version: Plexus Archiver

You can reproduce the problem with this demo project.

Flyway works correctly with fat jar bundled by Kobalt 1.0.60

mixed-example> rm db.h2db.mv.db; java -cp kobaltBuild/libs/* com.guatec.FlywayMigrator
rm: db.h2db.mv.db: No such file or directory

=== Applying DB Migrations up to 1.12  ===
Apr 27, 2017 4:28:46 AM org.flywaydb.core.internal.util.VersionPrinter printVersion
INFO: Flyway 4.1.2 by Boxfuse
Apr 27, 2017 4:28:46 AM org.flywaydb.core.internal.dbsupport.DbSupportFactory createDbSupport
INFO: Database: jdbc:h2:./db.h2 (H2 1.4)
Apr 27, 2017 4:28:46 AM org.flywaydb.core.internal.command.DbValidate validate
INFO: Successfully validated 2 migrations (execution time 00:00.051s)
Apr 27, 2017 4:28:46 AM org.flywaydb.core.internal.command.DbMigrate migrate
INFO: Current version of schema "PUBLIC": << Empty Schema >>
Apr 27, 2017 4:28:46 AM org.flywaydb.core.internal.command.DbMigrate applyMigration
INFO: Migrating schema "PUBLIC" to version 1.11 - Create tbl bob
Apr 27, 2017 4:28:46 AM org.flywaydb.core.internal.command.DbMigrate applyMigration
INFO: Migrating schema "PUBLIC" to version 1.12 - Create tbl other
Apr 27, 2017 4:28:46 AM org.flywaydb.core.internal.command.DbMigrate logSummary
INFO: Successfully applied 2 migrations to schema "PUBLIC" (execution time 00:00.024s).
0

Flyway does not find migrations in fat jar bundled by Kobalt 1.0.62

mixed-example> rm db.h2db.mv.db; java -cp kobaltBuild/libs/* com.guatec.FlywayMigrator
rm: db.h2db.mv.db: No such file or directory

=== Applying DB Migrations up to 1.12  ===
Apr 27, 2017 4:27:10 AM org.flywaydb.core.internal.util.VersionPrinter printVersion
INFO: Flyway 4.1.2 by Boxfuse
Apr 27, 2017 4:27:10 AM org.flywaydb.core.internal.dbsupport.DbSupportFactory createDbSupport
INFO: Database: jdbc:h2:./db.h2 (H2 1.4)
Apr 27, 2017 4:27:10 AM org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner findResourceNames
WARNING: Unable to resolve location classpath:db/migration
Apr 27, 2017 4:27:10 AM org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner findResourceNames
WARNING: Unable to resolve location classpath:db/migration
Apr 27, 2017 4:27:10 AM org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner findResourceNames
WARNING: Unable to resolve location classpath:db/migration
Apr 27, 2017 4:27:10 AM org.flywaydb.core.internal.util.scanner.classpath.ClassPathScanner findResourceNames
WARNING: Unable to resolve location classpath:db/migration
Apr 27, 2017 4:27:10 AM org.flywaydb.core.internal.command.DbValidate validate
INFO: Successfully validated 0 migrations (execution time 00:00.010s)
Apr 27, 2017 4:27:10 AM org.flywaydb.core.internal.metadatatable.MetaDataTableImpl createIfNotExists
INFO: Creating Metadata table: "PUBLIC"."schema_version"
Apr 27, 2017 4:27:10 AM org.flywaydb.core.internal.command.DbMigrate migrate
INFO: Current version of schema "PUBLIC": << Empty Schema >>
Apr 27, 2017 4:27:10 AM org.flywaydb.core.internal.command.DbMigrate logSummary
INFO: Schema "PUBLIC" is up to date. No migration necessary.
Exception in thread "main" org.h2.jdbc.JdbcSQLException: Table "BOB" not found; SQL statement:
SELECT * FROM bob [42102-192]
    at org.h2.message.DbException.getJdbcSQLException(DbException.java:345)
    at org.h2.message.DbException.get(DbException.java:179)
    at org.h2.message.DbException.get(DbException.java:155)
    at org.h2.command.Parser.readTableOrView(Parser.java:5388)
    at org.h2.command.Parser.readTableFilter(Parser.java:1257)
    at org.h2.command.Parser.parseSelectSimpleFromPart(Parser.java:1896)
    at org.h2.command.Parser.parseSelectSimple(Parser.java:2044)
    at org.h2.command.Parser.parseSelectSub(Parser.java:1890)
    at org.h2.command.Parser.parseSelectUnion(Parser.java:1711)
    at org.h2.command.Parser.parseSelect(Parser.java:1699)
    at org.h2.command.Parser.parsePrepared(Parser.java:445)
    at org.h2.command.Parser.parse(Parser.java:317)
    at org.h2.command.Parser.parse(Parser.java:289)
    at org.h2.command.Parser.prepareCommand(Parser.java:254)
    at org.h2.engine.Session.prepareLocal(Session.java:560)
    at org.h2.engine.Session.prepareCommand(Session.java:501)
    at org.h2.jdbc.JdbcConnection.prepareCommand(JdbcConnection.java:1202)
    at org.h2.jdbc.JdbcStatement.executeQuery(JdbcStatement.java:73)
    at com.guatec.FlywayMigrator.main(FlywayMigrator.kt:18)

[ethauvin]

Looks like the kotlin std lib manifest, likely related to #432.

[cbeust]

I only have a passing familiarity with Flyway, can you elaborate why this fails and how it relates to the manifest in the jar file?

The duplicate MANIFEST.MF is definitely a bug, looking into that, but I'm not sure about that first point.

[pabl0rg]

Well, Flyway is just the concrete example but something about the classpath is different (possibly broken?) in 1.0.62+.

Flyway searches the classpath for *sql files. It may be that there is no bug in Kobalt. In that case, whatever changed between 1.0.60 and 1.0.62 would reveal a bug in Flyway.

I will take a look at changes between those releases and try to be more helpful.

[cbeust]

I see your migration files at what appears the right place:

db/migration/V1_11__Create_tbl_bob.sql
db/migration/V1_12__Create_tbl_other.sql

Is this where Flyway expects them?

[pabl0rg]

Yeah that's the weird thing. It must be something else about the classpath.

[cbeust]

It's troubling that it used to work with an older version but I don't see anything immediately wrong with Kobalt right now. And the manifest you show at the top looks like it was generated by TeamCity? Not sure how it could have ended up in a Kobalt produced jar file (also not sure this has anything to do with the problem).

[pabl0rg]

I will dig further. However, the teamcity manifest may come from kobalt's dependencies, as suggested by @ethauvin

[cbeust]

How would that work?

[cbeust]

By the way, I don't think #432 is a valid issue.

-- Cédric

On Thu, Apr 27, 2017 at 11:41 AM, Juan Liska notifications@github.com wrote:

I will dig further. However, the teamcity manifest may come from kobalt's dependencies, as suggested by @ethauvin https://github.com/ethauvin

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cbeust/kobalt/issues/435#issuecomment-297803104, or mute the thread https://github.com/notifications/unsubscribe-auth/AAFoohlNrAtXM9gjmKWkxNx1P7opf4aeks5r0OFSgaJpZM4NKCWT .

[pabl0rg]

You're right. It's not about the manifest.

This may be a Flyway bug. When I unzip the Kobalt 1.0.62 fatJar and add that directory to the classpath, flyway works OK. I will ping them about this (my company previously sponsored a fix for a classpath scanning problem).

[cbeust]

Cool. Please reopen if needed.