search

quarkusio / quarkus

Quarkus: Supersonic Subatomic Java.
https://quarkus.io
Apache License 2.0
13.54k stars 2.61k forks source link

Test failure with Podman: Quarkus - Integration Tests - JPA - MySQL #33105

Open holly-cummins opened 1 year ago

holly-cummins commented 1 year ago

Describe the bug

I am working on a CI run which uses podman, and it has shown several failing tests. https://github.com/holly-cummins/quarkus/actions/runs/4844611295/jobs/8632987478 is an example run with failures. https://github.com/holly-cummins/quarkus/tree/podman-build has the (messy) workflow code.

Note that I’ve disabled this test when the CI is running with Podman, to keep the build clean. As part of the fix for this, the IS_PODMAN guard should be removed from the pom.xml.

Expected behavior

No response

Actual behavior

2023-04-28T21:41:25.3955201Z 21:41:25.000 MySQL:2023-04-28T21:41:25.293661Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
2023-04-28T21:41:25.3956027Z 21:41:25.000 MySQL:2023-04-28T21:41:25.294555Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/lib/mysql' in the path is accessible to all OS users. Consider choosing a different directory.
2023-04-28T21:41:25.3956963Z 21:41:25.000 MySQL:2023-04-28T21:41:25.312958Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '::' port: 33060, socket: /var/run/mysqld/mysqlx.sock
2023-04-28T21:41:25.3957947Z 21:41:25.000 MySQL:2023-04-28T21:41:25.313012Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.33'  socket: '/var/lib/mysql/mysql.sock'  port: 3306  MySQL Community Server - GPL.
2023-04-28T21:41:38.4706498Z [ERROR] DOCKER> [healthcheck-docker.io/mysql:8.0] "quarkus-test-mysql": Timeout after 20232 ms while waiting on healthcheck '"CMD-SHELL", "mysqladmin ping -h localhost -u hibernate_orm_test -phibernate_orm_test || exit 1"'
2023-04-28T21:41:38.4726277Z [ERROR] DOCKER> Error occurred during container startup, shutting down...
2023-04-28T21:41:38.8871139Z [INFO] DOCKER> [healthcheck-docker.io/mysql:8.0] "quarkus-test-mysql": Stop and removed container e204ca83be7b after 0 ms
2023-04-28T21:41:38.9494147Z [ERROR] DOCKER> I/O Error [[healthcheck-docker.io/mysql:8.0] "quarkus-test-mysql": Timeout after 20232 ms while waiting on healthcheck '"CMD-SHELL", "mysqladmin ping -h localhost -u hibernate_orm_test -phibernate_orm_test || exit 1"']

How to Reproduce?

Reproduction note: I haven't been able to reproduce this locally on mac with podman 4.03. With the latest mac version of podman 4.4.4, I get a permission failure running rootless, and it passes rootful.

Picking up my branch to reproduce on CI should work, or running locally on linux with a podman engine of 4.5.

Output of uname -a or ver

No response

Output of java -version

No response

GraalVM version (if different from Java)

No response

Quarkus version or git rev

No response

Build tool (ie. output of mvnw --version or gradlew --version)

No response

Additional information

Environment info

2023-05-02T10:44:48.4887010Z ##[group]Run docker version
2023-05-02T10:44:48.4887318Z �[36;1mdocker version�[0m
2023-05-02T10:44:48.4949771Z shell: /usr/bin/bash -e {0}
2023-05-02T10:44:48.4950031Z env:
2023-05-02T10:44:48.4950252Z   LANG: en_US.UTF-8
2023-05-02T10:44:48.4950613Z   COMMON_MAVEN_ARGS: -e -B --settings .github/mvn-settings.xml --fail-at-end
2023-05-02T10:44:48.4951271Z   NATIVE_TEST_MAVEN_ARGS: -Dtest-containers -Dstart-containers -Dquarkus.native.native-image-xmx=5g -Dnative -Dnative.surefire.skip -Dformat.skip -Dno-descriptor-tests install -DskipDocs
2023-05-02T10:44:48.4952041Z   JVM_TEST_MAVEN_ARGS: -Dtest-containers -Dstart-containers -Dformat.skip -DskipDocs -Dquarkus.test.hang-detection-timeout=60
2023-05-02T10:44:48.4952507Z   DB_USER: hibernate_orm_test
2023-05-02T10:44:48.4952790Z   DB_PASSWORD: hibernate_orm_test
2023-05-02T10:44:48.4953053Z   DB_NAME: hibernate_orm_test
2023-05-02T10:44:48.4953388Z   IS_PODMAN: true
2023-05-02T10:44:48.4953637Z   MAVEN_OPTS: -Xmx2g -XX:MaxMetaspaceSize=1g
2023-05-02T10:44:48.4954244Z   JAVA_HOME: /opt/hostedtoolcache/Java_Temurin-Hotspot_jdk/17.0.6-10/x64
2023-05-02T10:44:48.4954619Z   JAVA_HOME_17_X64: /opt/hostedtoolcache/Java_Temurin-Hotspot_jdk/17.0.6-10/x64
2023-05-02T10:44:48.4954970Z   DOCKER_HOST: unix:///run/user/1001/podman/podman.sock
2023-05-02T10:44:48.4955210Z ##[endgroup]
2023-05-02T10:44:49.5351123Z Client:
2023-05-02T10:44:49.5351708Z  Version:           20.10.24+azure-1
2023-05-02T10:44:49.5351963Z  API version:       1.41
2023-05-02T10:44:49.5352176Z  Go version:        go1.19.6
2023-05-02T10:44:49.5352453Z  Git commit:        297e1284d3bd092e9bc96076c3ddc4bb33f8c7ab
2023-05-02T10:44:49.5352738Z  Built:             Fri Mar 31 18:34:38 UTC 2023
2023-05-02T10:44:49.5352965Z  OS/Arch:           linux/amd64
2023-05-02T10:44:49.5353186Z  Context:           default
2023-05-02T10:44:49.5353414Z  Experimental:      true
2023-05-02T10:44:49.5353560Z 
2023-05-02T10:44:49.5353709Z Server: linux/amd64/ubuntu-22.04
2023-05-02T10:44:49.5353969Z  Podman Engine:
2023-05-02T10:44:49.5354202Z   Version:          4.5.0
2023-05-02T10:44:49.5354439Z   APIVersion:       4.5.0
2023-05-02T10:44:49.5354636Z   Arch:             amd64
2023-05-02T10:44:49.5355016Z   BuildTime:        1970-01-01T00:00:00Z
2023-05-02T10:44:49.5355260Z   Experimental:     false
2023-05-02T10:44:49.5355459Z   GitCommit:        
2023-05-02T10:44:49.5355677Z   GoVersion:        go1.18.1
2023-05-02T10:44:49.5355968Z   KernelVersion:    5.15.0-1036-azure
2023-05-02T10:44:49.5356195Z   MinAPIVersion:    4.0.0
2023-05-02T10:44:49.5356410Z   Os:               linux
2023-05-02T10:44:49.5356604Z  Conmon:
2023-05-02T10:44:49.5356824Z   Version:          conmon version 2.1.7, commit: 
2023-05-02T10:44:49.5357197Z   Package:          conmon_2:2.1.7-0ubuntu22.04+obs15.24_amd64
2023-05-02T10:44:49.5357457Z  OCI Runtime (crun):
2023-05-02T10:44:49.5357678Z   Version:          crun version 1.8.4
2023-05-02T10:44:49.5357962Z commit: 5a8fa99a5e41facba2eda4af12fa26313918805b
2023-05-02T10:44:49.5358237Z rundir: /run/user/1001/crun
2023-05-02T10:44:49.5358444Z spec: 1.0.0
2023-05-02T10:44:49.5358706Z +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
2023-05-02T10:44:49.5359097Z   Package:          crun_101:1.8.4-0ubuntu22.04+obs55.10_amd64
2023-05-02T10:44:49.5359334Z  Engine:
2023-05-02T10:44:49.5359516Z   Version:          4.5.0
2023-05-02T10:44:49.5359765Z   API version:      1.41 (minimum version 1.24)
2023-05-02T10:44:49.5360012Z   Go version:       go1.18.1
2023-05-02T10:44:49.5360212Z   Git commit:       
2023-05-02T10:44:49.5360442Z   Built:            Thu Jan  1 00:00:00 1970
2023-05-02T10:44:49.5360673Z   OS/Arch:          linux/amd64
2023-05-02T10:44:49.5360884Z   Experimental:     false
Sanne commented 1 year ago

Could not reproduce this locally on Fedora v38 / Podman 4.5.0, it could indeed be a timeout issue.

Sanne commented 1 year ago

Could not reproduce this locally on Fedora v38 / Podman 4.5.0, it could indeed be a timeout issue.

Same results from my home workstation (not surprising, as it's also running Fedora 38 and same podman versions).

Sanne commented 7 months ago

@holly-cummins do you know if this is still a problem? I could never reproduce it, wondering if I should try again..