search

quarkusio / quarkus

Quarkus: Supersonic Subatomic Java.
https://quarkus.io
Apache License 2.0
13.63k stars 2.64k forks source link

Native build with Helm charts image upload corrupt #28944

Open csscriptplus opened 1 year ago

csscriptplus commented 1 year ago

Describe the bug

My native build with a self-created Mandrel image (GraalVM 22.2.0.0-Final Java 17 Mandrel Distribution) and Helm Chart (0.1.2), cannot upload the generated Docker image. I get an HTTP 403 error. I have 2 repositories on the Nexus server. One for the Helm Charts and one for the Docker Images. Each repository has its own user with permissions. For the Docker image build I use JIB. In the configuration the users are assigned correctly. A normal Docker image build works. The Helm Charts and the Docker Image can be uploaded. I have verified that. If I give the Helm user permission to the Docker repository, then everything works. So there is a problem internally here with the user/password mapping.

My Jenkins maven call configuration

stage("Build / Upload Release Image") {
            steps {
                 withCredentials([usernamePassword(credentialsId: "Docker", passwordVariable: "DOCKER_PASSWORD", usernameVariable: "DOCKER_USERNAME"),
                                  usernamePassword(credentialsId: "Nexus", passwordVariable: "NEXUS_PASSWORD", usernameVariable: "NEXUS_USERNAME"),
                                  usernamePassword(credentialsId: "Helm", passwordVariable: "HELM_PASSWORD", usernameVariable: "HELM_USERNAME")]) { 
                    container('mandrel') {                                            
                        sh(''' 
                                export JAVA_HOME="${GRAALVM_HOME}"
                                export PATH="${JAVA_HOME}/bin:${PATH}"  

                            mvn clean package -U -Pnative \
                                -Dintegration-tests.skip=true \
                                -Dmaven.test.skip=true \
                                -DsendCredentialsOverHttp=true \
                                -Dquarkus.container-image.push=true \
                                -Dquarkus.container-image.group=kubernetes \
                                -Dquarkus.container-image.tag=${SNAPSHOT_VERSION} \
                                -Dquarkus.container-image.name=${ARTIFACT_ID} \
                                -Dquarkus.container-image.insecure=true \
                                -Dquarkus.container-image.registry=nexus-docker.demmtop.local \
                                -Dquarkus.container-image.username=${NEXUS_USERNAME} \
                                -Dquarkus.container-image.password=${NEXUS_PASSWORD} \
                                -Dquarkus.container-image.build=true \
                                -Dquarkus.jib.base-registry-username=${DOCKER_USERNAME} \
                                -Dquarkus.jib.base-registry-password=${DOCKER_PASSWORD} \
                                -Dquarkus.jib.base-native-image=quay.io/quarkus/ubi-quarkus-native-image:22.3.0-java17 \
                                -Dquarkus.kubernetes.version=${SNAPSHOT_VERSION} \
                                -Dquarkus.kubernetes.image-pull-secrets=regcred \
                                -Dquarkus.kubernetes.deployment-target=kubernetes \
                                -Dquarkus.kubernetes.deployment-kind=StatefulSet \
                                -Dquarkus.kubernetes.namespace=development-rest \
                                -Dquarkus.kubernetes.ingress.expose=true \
                                -Dquarkus.kubernetes.ingress.host=dev.${ARTIFACT_ID_LOWERCASE}.demmtop.local \
                                -Dquarkus.kubernetes.resources.requests.memory=64Mi \
                                -Dquarkus.kubernetes.resources.requests.cpu=250m \
                                -Dquarkus.kubernetes.resources.limits.memory=256Mi \
                                -Dquarkus.kubernetes.resources.limits.cpu=500m \
                                -Dquarkus.kubernetes.readiness-probe.initial-delay=20s \
                                -Dquarkus.kubernetes.readiness-probe.period=45s \
                                -Dquarkus.helm.repository.url=http://nexus.demmtop.local/repository/Demmtop-Helm-Development \
                                -Dquarkus.helm.name=${ARTIFACT_ID} \
                                -Dquarkus.helm.repository.push=true \
                                -Dquarkus.helm.repository.type=NEXUS \
                                -Dquarkus.helm.repository.username=${HELM_USERNAME} \
                                -Dquarkus.helm.repository.password=${HELM_PASSWORD}
                        ''')
                    }
                 }
             }
        }

2.13.2_native_helm.txt

Expected behavior

No response

Actual behavior

No response

How to Reproduce?

No response

Output of uname -a or ver

No response

Output of java -version

No response

GraalVM version (if different from Java)

No response

Quarkus version or git rev

2.13.2

Build tool (ie. output of mvnw --version or gradlew --version)

No response

Additional information

No response

quarkus-bot[bot] commented 1 year ago

/cc @Karm, @Sgitario, @galderz, @geoand, @iocanel, @zakkak

Karm commented 1 year ago

Hello @csscriptplus, I don't think this is a Native specific issue. The build went fine. The error is Unauthorized access. Perhaps some credentials resources are missing? Env vars not populated?

csscriptplus commented 1 year ago

Hello @Karm , I don't know if it's a native build issue or a Quarkus Maven plugin issue. I talked to the Helm Chart developer and he said:

Hi @csscriptplus , I could not reproduce this issue either using Quarkus 2.13.2 or a newer version. Yet looking at the logs you provided, this seems to be an issue with the Quarkus Container Image Jib extension, not Quarkus Helm. Please, report it at https://github.com/quarkusio/quarkus/issues and perhaps provide a reproducer too.

If I delete the -Pnative parameter, then the build works.

Docker image is uploaded to the Docker repository and the helm charts are uploaded to the helm repository.

If I add the -Pnative Parameter and delete the helm parameter, then the build works.

Docker image is uploaded to the Docker repository.

If I add the -Pnative Parameter and add the helm parameter, then the build doesn't work.

The helm user has not permission to the docker repository. If I add the permission then the build can upload the image.

Sgitario commented 1 year ago

Hello, I'm the Quarkus Helm extension developer ;) So, the issue only happens when using the quarkus.helm.repository.xxx properties? This is rather weird because the exception printed in the attached log is thrown when building the Docker image by the Container Image Jib extension which is not using the Helm configuration. However, if you're using: the image.registry use the helm user account <- Http 403 then it's expected that the Helm user account does need permission to upload the image, or I'm missing something else?

csscriptplus commented 1 year ago

the image.registry use the helm user account <- Http 403

This is not my configuration. The build behaves like this. According to my configuration it is defined that image.registry should use the Nexus user and Helm charts should use the Helm user. I was just testing what happens when I give the Helm user permission to the Docker repository. The test was then successful and the build runs through without errors. So this means that image.registry does not use the Nexus user but the Helm user and this is an error. However, this error only occurs with a native build.

iocanel commented 1 year ago

With a quick glance at the JibProducer, I see that there are some differences in how push is handled between native and jar. Not sure if I have the cycles to invetigate further at the moment. So, if anyone wants to jump in, please do.

csscriptplus commented 1 year ago

Ok. The workaround for me is I use only one user account on the nexus server. And this account has the permission to the Helm and Docker image repository.