search

zonkyio / embedded-postgres

Java embedded PostgreSQL component for testing
Apache License 2.0
344 stars 43 forks source link

Can't init database inside a jenkins pipeline build #81

Closed fabballe closed 2 years ago

fabballe commented 2 years ago

Hi,

First of all , thanks for your amazing lib :+1:

Nevertheless I have an issue while I try to integrate it.

Locally I achieve to launch my test but when I try to launch it through my Jenkins CI (pipeline branch). I got an issue during the init phase:

2021-11-26 13:54:58.088  INFO [user ] 171 --- [initdb:pid(313)] i.z.t.d.p.embedded.EmbeddedPostgres      : initdb: could not look up effective user ID 65534: user does not exist
2021-11-26 13:54:58.096  WARN [user ] 171 --- [           main] o.s.test.context.TestContextManager      : Caught exception while invoking 'beforeTestExecution' callback on TestExecutionListener [io.zonky.test.db.event.EventPublishingTestExecutionListener@3338773d] for test method [void org.talend.dataprep.preparation.PreparationApplicationTests.contextLoads()] and test instance [org.talend.dataprep.preparation.PreparationApplicationTests@5784d19]
io.zonky.test.db.shaded.com.google.common.util.concurrent.UncheckedExecutionException: io.zonky.test.db.provider.ProviderException: Unexpected error when preparing a database cluster; nested exception is java.lang.IllegalStateException: Process [unshare, -U, target/PG-2990f4c1c89fc732d8f8429f6ba24252/bin/initdb, -A, trust, -U, postgres, -D, target/361b27b9-0399-4a50-8c60-f927e0847895, -E, UTF-8] failed
    at io.zonky.test.db.shaded.com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1571)
    at io.zonky.test.db.shaded.com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1556)
    at io.zonky.test.db.context.DefaultDatabaseContext.awaitDatabase(DefaultDatabaseContext.java:282)
    at io.zonky.test.db.context.DefaultDatabaseContext.handleTestStarted(DefaultDatabaseContext.java:177)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

it seems there is an issue during user match: initdb: could not look up effective user ID 65534: user does not exist. As I am inside a temporary kubernetes pod inside my jenkins I work with a root user.

Locally everything seems to work

2021-11-26 14:48:56.156  INFO [user ] 40256 --- [itdb:pid(40351)] i.z.t.d.p.embedded.EmbeddedPostgres      : The files belonging to this database system will be owned by user "fabballe".
2021-11-26 14:48:56.156  INFO [user ] 40256 --- [itdb:pid(40351)] i.z.t.d.p.embedded.EmbeddedPostgres      : This user must also own the server process.
2021-11-26 14:48:56.156  INFO [user ] 40256 --- [itdb:pid(40351)] i.z.t.d.p.embedded.EmbeddedPostgres      : 
2021-11-26 14:48:56.156  INFO [user ] 40256 --- [itdb:pid(40351)] i.z.t.d.p.embedded.EmbeddedPostgres      : The database cluster will be initialized with locales
2021-11-26 14:48:56.156  INFO [user ] 40256 --- [itdb:pid(40351)] i.z.t.d.p.embedded.EmbeddedPostgres      :   COLLATE:  en_US.UTF-8
2021-11-26 14:48:56.156  INFO [user ] 40256 --- [itdb:pid(40351)] i.z.t.d.p.embedded.EmbeddedPostgres      :   CTYPE:    en_US.UTF-8

Is there a way to choose the user to use to launch the embedded postgres ? Do you have any hint to fix this ?

My dependencies if it can help

[INFO] +- io.zonky.test:embedded-database-spring-test:jar:2.1.1:test
[INFO] |  +- io.zonky.test:embedded-database-spring-test-autoconfigure:jar:2.1.1:test
[INFO] |  +- org.testcontainers:postgresql:jar:1.15.3:test
[INFO] |  |  \- org.testcontainers:jdbc:jar:1.15.3:test
[INFO] |  |     \- org.testcontainers:database-commons:jar:1.15.3:test
[INFO] |  |        \- org.testcontainers:testcontainers:jar:1.15.3:test
[INFO] |  |           +- org.rnorth.duct-tape:duct-tape:jar:1.0.8:test
[INFO] |  |           +- org.rnorth.visible-assertions:visible-assertions:jar:2.1.2:test
[INFO] |  |           +- com.github.docker-java:docker-java-api:jar:3.2.8:test
[INFO] |  |           \- com.github.docker-java:docker-java-transport-zerodep:jar:3.2.8:test
[INFO] |  |              \- com.github.docker-java:docker-java-transport:jar:3.2.8:test
[INFO] |  +- org.testcontainers:mssqlserver:jar:1.15.3:test
[INFO] |  +- org.testcontainers:mysql:jar:1.15.3:test
[INFO] |  +- org.testcontainers:mariadb:jar:1.15.3:test
[INFO] |  +- org.springframework:spring-test:jar:5.2.15.RELEASE:test
[INFO] |  \- com.cedarsoftware:java-util:jar:1.34.0:test
[INFO] +- io.zonky.test:embedded-postgres:jar:1.3.1:test
[INFO] |  +- io.zonky.test.postgres:embedded-postgres-binaries-windows-amd64:jar:13.4.0:test
[INFO] |  +- io.zonky.test.postgres:embedded-postgres-binaries-darwin-amd64:jar:13.4.0:test
[INFO] |  +- io.zonky.test.postgres:embedded-postgres-binaries-linux-amd64:jar:13.4.0:test
[INFO] |  +- io.zonky.test.postgres:embedded-postgres-binaries-linux-amd64-alpine:jar:13.4.0:test

Thanks for you reply

tomix26 commented 2 years ago

Hi, @fabballe, I'm sorry for the late response. The embedded postgres should be launched under the same user as the build is running (just inheriting it from the parent process), there is currently no option to change that directly from the library. And If you are running the build as root user, that is probably the cause of the problem. Because PostgreSQL has a restriction that the process must be run as non-root user, otherwise the initialization of the database fails.

I'm not an expert in kubernetes, but in general, you need to create a non-root user inside the pod and use it to run the build.

jameshilliard commented 2 years ago

Probably adding nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin to /etc/passwd will also fix that error as the error indicates the temporary nobody uid 65534(which seems to be the default temporary uid for unshare -U) /etc/passwd entry is missing.

fabballe commented 2 years ago

Hi, I am not able to add new user inside my pod configuration for security reason. For this reason I have to switch to another lib to handle embedded psql inside my UT.