eclipse-jkube / jkube

Build and Deploy java applications on Kubernetes
https://www.eclipse.dev/jkube/
Eclipse Public License 2.0
773 stars 519 forks source link

Unable to build quickstart spring-boot in Openshift 4.12 #2323

Open glennodickson opened 1 year ago

glennodickson commented 1 year ago

Hi New to JKube and I've tried to build the above it seems that the OCP build config cannot pull the jkube-java due to an incompatible certificate. Example documentation does not indicate that a certificate needs to be setup in JKube for the build config.

....
[INFO] --- oc:1.13.1:build (default-cli) @ spring-boot ---
[INFO] oc: Using OpenShift build with strategy S2I
[INFO] oc: Building Docker image
[INFO] oc: Running generator spring-boot
[INFO] oc: spring-boot: Using Docker image quay.io/jkube/jkube-java:0.0.19 as base / builder
[INFO] oc: [spring-boot:1.13.1] "spring-boot": Created docker source tar C:\data\project\jkube-master\quickstarts\maven\spring-boot\target\docker\spring-boot\1.13.1\tmp\docker-build.tar
[INFO] oc: Adding to Secret pullsecret-jkube
[INFO] oc: Using Secret pullsecret-jkube
[INFO] oc: Creating BuildServiceConfig spring-boot-s2i for Source build
[INFO] oc: Creating ImageStream spring-boot
[INFO] oc: Starting Build spring-boot-s2i
[INFO] oc: Waiting for build spring-boot-s2i-1 to complete...
[INFO] oc: time="2023-08-07T16:43:42Z" level=info msg="Not using native diff for overlay, this may cause degraded performance for building images: kernel has CONFIG_OVERLAY_FS_REDIRECT_DIR enabled"
[INFO] oc: I0807 16:43:42.495680       1 defaults.go:112] Defaulting to storage driver "overlay" with options [mountopt=metacopy=on].    
[INFO] oc: Caching blobs under "/var/cache/blobs".
[INFO] oc: Trying to pull quay.io/jkube/jkube-java:0.0.19...
[INFO] oc: Warning: Pull failed, retrying in 5s ...
[INFO] oc: Trying to pull quay.io/jkube/jkube-java:0.0.19...
[INFO] oc: Warning: Pull failed, retrying in 5s ...
[INFO] oc: Trying to pull quay.io/jkube/jkube-java:0.0.19...
[INFO] oc: Warning: Pull failed, retrying in 5s ...
[INFO] oc: error: build error: After retrying 2 times, Pull image still failed due to error: initializing source docker://quay.io/jkube/jkube-java:0.0.19: pinging container registry quay.io: Get "https://quay.io/v2/": x509: certificate is valid for *.apps.test.fritz.box, not quay.io
[ERROR] oc: Failed to execute the build [Unable to build the image using the OpenShift build service]

I've struggled with this for sometime now and can't seem to solve. Any help would be very appreciated.

TIA

rohanKanojia commented 1 year ago

@glennodickson : Looks like your OpenShift cluster is configured to pull only from a specific registry .apps.test.fritz.box . You'd need to add your own base S2I image to registry and configure property jkube.generator.from to override default base image.

manusa commented 1 year ago

It looks more like you're behind a router, firewall, proxy or gateway that might be intercepting your TLS request to quay.io. Could your provide more details about your setup? Is a similar build working when performed directly on the cluster through other means?

glennodickson commented 1 year ago

Hi Marc Thanks for the hint. if you have any suggestions of how to test/isolate this interception I'd appreciate it.

Yeah my Openshift cluster system is running on a separate Proxmox hypervisor box in its own subnet. The cluster can update and install operators which I assume also pull images from quay on the internet so the build should not be any different using Quay.io.

My build process starts with my PC (in main subnet) building the quickstart spring-boot app and then deploys the buildconfig on the Openshift cluster (in own subnet). The buildconfig start then errors are indicated previously.

Regards Glenn

On Tue, 8 Aug 2023 at 12:33, Marc Nuri @.***> wrote:

It looks more like you're behind a router, firewall, proxy or gateway that might be intercepting your TLS request to quay.io. Could your provide more details about your setup? Is a similar build working when performed directly on the cluster through other means?

— Reply to this email directly, view it on GitHub https://github.com/eclipse/jkube/issues/2323#issuecomment-1668891896, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA5ROAPYHQ7PUOH2XUGMSTLXUG6SDANCNFSM6AAAAAA3HIMQJ4 . You are receiving this because you were mentioned.Message ID: @.***>

manusa commented 1 year ago

I'm checking OpenShift docs. In the meantime, could you please try the (OpenShift) Docker build strategy to see if this works any better:

-Djkube.build.strategy=docker
glennodickson commented 1 year ago

Hi Marc Thanks for your response. When trying to access Quay.io from a pod it seems to have a selfsigned cert issue with Quay.io not sure why. Here is output I get is:

curl -v https://quay.io

* Rebuilt URL to: https://quay.io/ https://quay.io/ Trying 192.168.179.73... TCP_NODELAY set Connected to quay.io http://quay.io (192.168.179.73) port 443 (#0) ALPN, offering h2 ALPN, offering http/1.1 successfully set certificate verify locations: CAfile: /etc/pki/tls/certs/ca-bundle.crt CApath: none TLSv1.3 (OUT), TLS handshake, Client hello (1): TLSv1.3 (IN), TLS handshake, Server hello (2): TLSv1.3 (IN), TLS handshake, [no content] (0): TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8): TLSv1.3 (IN), TLS handshake, [no content] (0): TLSv1.3 (IN), TLS handshake, Certificate (11): TLSv1.3 (OUT), TLS alert, unknown CA (560): SSL certificate problem: self signed certificate in certificate chain Closing connection 0curl: (60) SSL certificate problem: self signed certificate in certificate chainMore details here: https://curl.haxx.se/docs/sslcerts.html https://curl.haxx.se/docs/sslcerts.htmlcurl failed to verify the legitimacy of the server and therefore could notestablish a secure connection to it. To learn more about this situation andhow to fix it, please visit the web page mentioned above.

Seems like self-signed is the issue - agree? Perhaps getting a cert from LetsEncrypt etc would solve this?

Regards Glenn

On Tue, 8 Aug 2023 at 13:30, Marc Nuri @.***> wrote:

I'm checking OpenShift docs. In the meantime, could you please try the (OpenShift) Docker build strategy to see if this works any better:

-Djkube.build.strategy=docker

— Reply to this email directly, view it on GitHub https://github.com/eclipse/jkube/issues/2323#issuecomment-1668933710, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA5ROALMNEM6VU6X65AICYDXUHFHRANCNFSM6AAAAAA3HIMQJ4 . You are receiving this because you were mentioned.Message ID: @.***>

manusa commented 1 year ago

Seems like self-signed is the issue - agree? Perhaps getting a cert from LetsEncrypt etc would solve this?

I'm not that knowledgeable with OpenShift. I think we definitely have an issue with your cluster's certificates. So maybe LetsEncrypt does help without the need to provide further cluster configurations (but I really don't know).

Yesterday I was reading about adding your cluster certs to the trusted certificate authorities (CA). However, it wasn't clear to me if this would help with the OpenShift S2I builds.

friedeas commented 1 year ago

For me this also looks like an configuration issue with the CA certificates used for egress connections. From the initial build error massage: Get "https://quay.io/v2/": x509: certificate is valid for *.apps.test.fritz.box, not quay.io

So it looks like the used CA only contains the self-signed one that is valid for *.apps.test.fritz.box, not quay.io, but quay.io requires Amazon Root CA 1 to be also covered by the used CA configuration.

Maybe this documentation is helpful: https://docs.openshift.com/container-platform/4.12/security/certificate_types_descriptions/proxy-certificates.html