(Apologies for posting here as it might not be directly a helm-chart issue but we only encounter the issue in k8s environments)
Background
We're building and using arm64 images since quite some time.
Recently, we aimed to get the WB server pod running in a mixed-arch fashion on a cluster.
We are building the images using buildx in a multi-arch way and can hence ensure that both architectures have exactly the same content.
Running the images on a VM with docker works just fine, i.e. startup works and the WB is usable.
However, for an unknown reason, the arm64 pods always get stuck during startup when using it in k8s (the amd64 counterparts work just fine).
Logs
With debug logging:
2024-01-09T09:25:33.824831Z [rserver] INFO Reading OpenID secret from '/mnt/secret-configmap/rstudio/openid-client-secret' │
│ 2024-01-09T09:25:36.984354Z [rstudio-kubernetes-launcher, log-source: Kubernetes] DEBUG Queueing response message [msg: {"messageType":0,"requestId":0,"responseId":0}] │
│ 2024-01-09T09:25:36.984501Z [rstudio-launcher] DEBUG Received message from plugin [plugin: Kubernetes, msg: {"messageType":0,"requestId":0,"responseId":0}] │
│ 2024-01-09T09:25:36.984749Z [rstudio-launcher] DEBUG Sending message to plugin [msg: {"messageType":0,"requestId":0}, plugin: Kubernetes]
The startup via /usr/lib/rstudio-server/bin/rserver --server-daemonize 0 >/dev/stderr always stops at Reading OpenID secret from '/mnt/secret-configmap/rstudio/openid-client-secret' while the amd64 pods continue after this point with
│ 2024-01-09T09:27:06.581937Z [rserver] INFO Reading OpenID secret from '/mnt/secret-configmap/rstudio/openid-client-secret' │
│ 2024-01-09T09:27:06.587874Z [rserver] INFO rserver-openid OpenID Version 0.6.0 │
│ 2024-01-09T09:27:06.588190Z [rserver] INFO Reading server user profiles from '/mnt/configmap/rstudio/profiles' │
│ 2024-01-09T09:27:06.588312Z [rserver] DEBUG PyVersionsScanner: /usr/bin/python does not exist │
│ 2024-01-09T09:27:06.593626Z [rserver] INFO PyVersionsScanner: Found Python version 3.10.12 at /usr/bin/python3
Is there any way to debug what might be happening between Reading OpenID secret and rserver-openid OpenID Version (which is never reached in the arm64 pod)?
Details
Interesting side fact: starting the same images outside k8s via docker works just fine, letting us assume that the "normal" startup is OK.
(Apologies for posting here as it might not be directly a helm-chart issue but we only encounter the issue in k8s environments)
Background
We're building and using arm64 images since quite some time. Recently, we aimed to get the WB server pod running in a mixed-arch fashion on a cluster.
We are building the images using
buildxin a multi-arch way and can hence ensure that both architectures have exactly the same content. Running the images on a VM with docker works just fine, i.e. startup works and the WB is usable. However, for an unknown reason, the arm64 pods always get stuck during startup when using it in k8s (the amd64 counterparts work just fine).Logs
With debug logging:
The startup via
/usr/lib/rstudio-server/bin/rserver --server-daemonize 0 >/dev/stderralways stops atReading OpenID secret from '/mnt/secret-configmap/rstudio/openid-client-secret'while the amd64 pods continue after this point withIs there any way to debug what might be happening between
Reading OpenID secretandrserver-openid OpenID Version(which is never reached in the arm64 pod)?Details
Interesting side fact: starting the same images outside k8s via docker works just fine, letting us assume that the "normal" startup is OK.