Concurrency issues when running modules and tests in parallel

[hestad]

Hi :)

I've ran into this badboy:

    org.postgresql.util.PSQLException: ERROR: source database "template1" is being accessed by other users
      Detail: There are 2 other sessions using the database.
        at app//org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2552)
        at app//org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:2284)
        at app//org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:322)
        at app//org.postgresql.jdbc.PgStatement.executeInternal(PgStatement.java:481)
        at app//org.postgresql.jdbc.PgStatement.execute(PgStatement.java:401)
        at app//org.postgresql.jdbc.PgPreparedStatement.executeWithFlags(PgPreparedStatement.java:164)
        at app//org.postgresql.jdbc.PgPreparedStatement.execute(PgPreparedStatement.java:153)
        at app//io.zonky.test.db.postgres.embedded.PreparedDbProvider.create(PreparedDbProvider.java:244)
        at app//io.zonky.test.db.postgres.embedded.PreparedDbProvider.access$400(PreparedDbProvider.java:41)
        at app//io.zonky.test.db.postgres.embedded.PreparedDbProvider$PrepPipeline.run(PreparedDbProvider.java:215)
        at java.base@16.0.2/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base@16.0.2/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base@16.0.2/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
        at java.base@16.0.2/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
        at java.base@16.0.2/java.lang.Thread.run(Thread.java:831)

Versions: Kotlin 1.5.30, Java 16.0.2, Postgres 11.7, jupiter 5.7.2, gradle 7.2, Zonky 1.3.1 As the build time increases over time, We're trying to run most of our tests in parallel. Without success this far. We have hundreds of db-test using a separate dataSource thanks to these embedded-pg repos :) It is hard to reproduce at my 8 core / 16 thread intel cpu (but it happens from time to time), but it fails regularly at the Github Action instance.

gradle settings:

        maxParallelForks = (Runtime.getRuntime().availableProcessors() / 2).takeIf { it > 0 } ?: 1
        systemProperties["junit.jupiter.execution.parallel.enabled"] = true
        systemProperties["junit.jupiter.execution.parallel.mode.default"] = "concurrent"

Code setup (package global)

private class CustomFlywayPreparer() : DatabasePreparer {
    override fun prepare(ds: DataSource) {
        ds.connection
            .prepareStatement("""create role "postgres-admin" """)
            .execute()
        ds.connection
            .prepareStatement("""create EXTENSION IF NOT EXISTS "uuid-ossp"""")
            .execute()
        Flyway.configure().target(MigrationVersion.LATEST).dataSource(dataSource).let {
                  it.initSql("SET ROLE \"postgres-admin\"")
            }
            .load()
            .migrate()
    }
}
private val preparer = CustomFlywayPreparer()
// this is used by all the tests
fun withMigratedDb(test: (dataSource: DataSource) -> Unit) {
    test(createNewDatabase(preparer = preparer))
}
private fun createNewDatabase(preparer: DatabasePreparer): DataSource {
  val provider = PreparedDbProvider.forPreparer(preparer)
  val info = provider.createNewDatabase()
  return provider.createDataSourceFromConnectionInfo(info)
}

[tomix26]

Hi, thanks for the post. I guess that the problem could be caused by the implementation of the custom preparer. Because the preparer calls some sql statements, but does not release the underlying resources after the operations are completed. So please make sure that the close methods are called property on all connection and statement objects. Furthermore, for optimal performance, the preparer implementation should also implement equals and hashCode methods. This is necessary for the caching mechanism to work properly.

[hestad]

Closing the connections and statements did the trick :sweat_smile: I also added the equals and hashCode for good measure; but the cache was working already (because it was the same object).

Thanks :) I'll close this on my way out.