replicatedhq / troubleshoot

Preflight Checks and Support Bundles Framework for Kubernetes Applications
https://troubleshoot.sh
Apache License 2.0
543 stars 93 forks source link

Enhance runPod analyzer to accept vendor/enduser-provided resources like Secrets or ConfigMaps to be created during runPod execution #1530

Open adamancini opened 5 months ago

adamancini commented 5 months ago

Describe the rationale for the suggested feature.

Some users might rely on things like externally provided TLS certs, such as for connecting to TLS-encrypted databases like Postgres or Mysql. The runPod collector can be useful for performing arbitrary checks, but may need to utilize a given TLS cert or other "large" strings/files.

Example: We're designing a collector using the runPod feature to execute a container that performs different checks related with our application.

This pod needs some files and for the moment the approach that we're using is creating an initContainer that receives the files content and path through env variables, and it's executing a bash script to write them into an emptyDir which is passed to the main Pod for processing.

The limitation we're seeing is that this customer is using an SSL certificate which chain is too long, and for that reason the content can't be passed in a single env variable. We're working on an alternative approach to divide the value in multiple env vars and join it with the bash script, but it's a really tricky approach.

Counterpoint: Is there a reason you can't mount the relevant secret/configmap as a volume directly?

Like this example, stolen from here:

    volumeMounts:
      - mountPath: "/var/my-app"
        name: ssh-key
        readOnly: true
  volumes:
    - name: ssh-key
      secret:
        secretName: ssh-key

The RunPod collector takes a full PodSpec, so I would expect there to be no issues.

Counter-Counterpoint: that works, but the problem is that it's not a valid approach for the first time that preflights are executed. When they're executed when installing for the first time, there are no secrets or configmaps created in the namespace, so they can't be used. We don't really like the approach of injecting the value directly in the args field as the same issue could appear again in the future. We've performed some changes to split the content in different env vars and join them in the container.

IMHO the ideal solution for this would be to create a temporary secret or configmap that can be specified during the preflights execution in the same way that the runPod collector runs a pod and deletes it automatically 🤔

Describe the feature

Not sure exactly how this might be implemented yet, but I'm imagining solutions involving templating into the static data collector, or actually creating/deleting the necessary Kubernetes resources. The runPod spec would have to be extended and this would significantly change the scope of the collector so I think this needs some more thinking through.

Describe alternatives you've considered

Additional context