Closed matbun closed 2 months ago
@Surax98 any idea on how to tackle this?
@matbun the investigation took a bit since I had to dig into the Virtual Kubelet repository, since it seems related to their code, more than to the interLink provider itself. Let me explain a bit what's going on. Let's take as example this snippet from your pod:
- name: MLFLOW_TRACKING_USERNAME
valueFrom:
secretKeyRef:
name: mlflow-server
key: username
- name: MLFLOW_TRACKING_PASSWORD
valueFrom:
secretKeyRef:
name: mlflow-server
key: password
When using Secrets or ConfigMaps to set ENVs, a specific code on the Virtual Kubelet package is executed, leading to the retrieve of the specified resource. During this phase, the resource gathering, for a still under investigation reason, fails, probably due to a permission issue (bad cluster role?). Using ConfigMaps and Secrets as Volumes, works as expected, so you can workaround the issue by using Volumes, for the moment. By using a ClientSet within the interLink provider, I can easily retrieve everything, so it's seems a bit odd for the moment, until I don't get a much clearer understanding of the issue. For reference, you can see the executed Virtual Kubelet code at this link; this is the function used to populate ENVs for the container, both for ConfigMaps and Secrets, even if the link points to the Secret case only. Digging in the code takes time, so feel free to help if you want!
Merged and fixed
Short Description of the issue
When using secrets, the pod stays in PENDING state forever. Removing them, the pods is executed correctly.
Environment
InterTwin env on Vega.
Steps to reproduce
Create secrets:
Pod I am using:
Logs, stacktrace, or other symptoms