Specify custom Certificate Authority

[eoinoreilly30]

We export metrics to an internal proxy that uses a self hosted CA. Is there an ability to specify a custom CA for the telegraf sidecar to use?

What products and version are you using?

Telegraf 1.22.4

[l0calhost]

Same here.

My app runs with our custom CA applied and is able to communicate with internal https systems. Adding the telegraf-operator sidecar container leads to

E! [agent] Error writing to outputs.http: Post "https:///telegraf": x509: certificate signed by unknown authority

It seems it does not respect the main containers CA.

Trying to workaround by setting up k8s volume and volumemount providing a tls_ca to the [[outputs.http]] like

[[outputs.http]] url = "https:///telegraf" method = "POST" data_format = "json" content_encoding = "gzip" tls_ca = "/etc/custom-ca/my-custom-ca.pem" [...]

results in another error:

2023-07-03T07:54:11Z E! [telegraf] Error running agent: Error loading config file /etc/telegraf/telegraf.conf: error parsing http array, could not read certificate "/etc/custom-ca/my-custom-ca.pem": open /etc/custom-ca/my-custom-ca.pem: no such file or directory

although the file is readable in the main container.

Is there a way to configure telegraf-operator to use the main containers CA(s) or provide a custom one on global configuration level?

[eoinoreilly30]

@l0calhost How did you add the CA secret volume? Via the helm chart or something else?

[l0calhost]

@eoinoreilly30 I added my CA as a ConfigMap to my helm chart. And i guess i figured it out. #104 gave me the missing hint.

Here's what i did:

1. Add a config map containing the custom CA:

   ---
   apiVersion: v1
   kind: ConfigMap
   metadata:
   name: certs
   data:
   customca.pem: |
   <INSRT_CUSTOM_CA_HERE>

2. Add a volume referencing the ConfigMap:

     volumes:
     - name: my-custom-ca
       configMap:
         name: certs

3. Add a pod annotation configuring the mount:

   telegraf.influxdata.com/volume-mounts: '{"my-custom-ca":"/path/to/ca"}'

4. Configure telegraf http output to use the cert:

   [[outputs.http]]
   url = "https:///telegraf"
   method = "POST"
   data_format = "json"
   content_encoding = "gzip"
   tls_ca = "/path/to/ca/customca.pem"
   [...]

[eoinoreilly30]

Thanks @l0calhost! I was able to configure the volume-mount and use the CA successfully

We already had the CA as a secret volume in the pod that we want the telegraf sidecar to be placed, so using the telegraf.influxdata.com/volume-mounts: '{"my-custom-ca": "/etc/ssl/certs"}' annotation I was able to mount the CA at /etc/ssl/certs in the telegraf sidecar successfully. Then used tls_ca = "/etc/ssl/certs/ca.crt" in the output plugin.

Are you able to look inside the telegraf sidecar container with kubectl exec bash to see if the volume is mounted successfully?

I am using helm chart version: 1.3.11 and telegraf 1.27.1

[l0calhost]

Hey @eoinoreilly30, as far as i can see, there is no shell installed in the telegraf sidecar container, so no. To check the mounts, i used kubectl describe pod <pod-name> and inspected the "Mounts" section. But the missing error message in the telegraf container and arriving metrics were the main indicators everything went smooth. ;)