Better way to set certificate_authorities for metricbeat / filebeat containers

[sratz]

Proposal

Consider the following situation:

• Certificate provided via cert-manager using non-public CA
• The resulting certificate secret will not contain a ca.crt.
• For Kibana to be able to connect to ES, this C must be added manually according to https://github.com/elastic/cloud-on-k8s/issues/1494#issuecomment-519003310, which is rather straight-forward:

  
  apiVersion: kibana.k8s.elastic.co/v1
  kind: Kibana
  metadata:
  name: logsearch
  namespace: logsearch-prod
  spec:
  version: 8.6.2
  count: 1
  elasticsearchRef:
    name: logsearch

• config:
• elasticsearch.ssl.certificateAuthorities: /mnt/rootcas/ca.crt
• podTemplate:
• spec:
• volumes:
• • name: rootcas
• secret:
• secretName: rootcas
• containers:
• • name: kibana
• volumeMounts:
• • name: rootcas
• mountPath: /mnt/rootcas http: tls: certificate: secretName: elk-http-tls selfSignedCertificate: disabled: true

  * But now consider a simple [(self-)monitoring](https://www.elastic.co/guide/en/cloud-on-k8s/current/k8s-stack-monitoring.html) setup:

  monitoring: metrics: elasticsearchRefs:

  • name: logsearch logs: elasticsearchRefs:
  • name: logsearch

The config.elasticsearch.ssl.certificateAuthorities option does not apply to the meticbeat and filebeat configuration.

The only way to get this set is:

  apiVersion: kibana.k8s.elastic.co/v1
  kind: Kibana
  metadata:
    name: logsearch
    namespace: logsearch-prod
  spec:
    version: 8.6.2
    count: 1
    elasticsearchRef:
      name: logsearch
    config:
      elasticsearch.ssl.certificateAuthorities: /mnt/rootcas/ca.crt
    podTemplate:
      spec:
        volumes:
        - name: rootcas
          secret:
            secretName: rootcas
        containers:
        - name: kibana
          volumeMounts:
          - name: rootcas
            mountPath: /mnt/rootcas
+       - name: filebeat
+         args:
+           - '-c'
+           - /etc/filebeat-config/filebeat.yml
+           - '-e'
+           - '-E'
+           - 'output.elasticsearch.ssl.certificate_authorities=["/mnt/rootcas/ca.crt"]'
+           - '-E'
+           - 'setup.kibana.ssl.certificate_authorities=["/mnt/rootcas/ca.crt"]'
+         volumeMounts:
+         - name: rootcas
+           mountPath: /mnt/rootcas
+       - name: metricbeat
+         args:
+           - '-c'
+           - /etc/metricbeat-config/metricbeat.yml
+           - '-e'
+           - '-E'
+           - 'output.elasticsearch.ssl.certificate_authorities=["/mnt/rootcas/ca.crt"]'
+           - '-E'
+           - 'metricbeat.modules.0.ssl.certificate_authorities=["/mnt/rootcas/ca.crt"]'
+         volumeMounts:
+         - name: rootcas
+           mountPath: /mnt/rootcas
    http:
      tls:
        certificate:
          secretName: elk-http-tls
        selfSignedCertificate:
          disabled: true
    monitoring:
      metrics:
        elasticsearchRefs:
        - name: logsearch
      logs:
        elasticsearchRefs:
        - name: logsearch

Overwriting args like this is quite error-prone. Alternatively, the whole filebeat.yml / elasticbeat.yml would have to be provided.

It would be better if this could be properly set via a dedicated configuration option just like config.elasticsearch.ssl.certificateAuthorities.

Environment

• ECK version:

  2.7.0

[pebrc]

One thing we could potentially do is add section similar to what we did recently for transport:

  http:
    tls:
      certificateAuthorities:
        configMapName: trust

and then have the operator ensure that:

1. the trusted CA certificates are propagated to the monitoring Beats
2. the trusted CA certificates are propagated across associations e.g. Kibana