Beat installation - Missing Documentation - Elastic Cloud

[jcamu]

Due to the end of the current beat helm chart, we should apparently migrate to this new chart.

But there is no documentation on how to install and configure only beats in K8s and send data to Elastic Cloud ?

[brsolomon-deloitte]

Has the chart actually been uploaded?

$ helm pull --repo https://helm.elastic.co --untar --version 0.1.0 eck-beats
Error: chart "eck-beats" version "0.1.0" not found in https://helm.elastic.co repository

[brsolomon-deloitte]

$ helm repo add elastic https://helm.elastic.co
"elastic" already exists with the same configuration, skipping
$ helm repo update
...
Update Complete. ⎈Happy Helming!⎈

$ helm search repo beats
NAME                        CHART VERSION   APP VERSION DESCRIPTION
elastic/eck-operator        2.5.0           2.5.0       A Helm chart for deploying the Elastic Cloud on...
elastic/eck-operator-crds   2.5.0           2.5.0       A Helm chart for installing the ECK operator Cu...
$ helm pull elastic/eck-beats
Error: chart "eck-beats" matching  not found in elastic index. (try 'helm repo update'): no chart name found

None of the charts here or here actually seem to be present.

[jcamu]

In addition, even if the beats chart is present in the main branch, it seems to not be included in the coming release 2.6.X.

Is it expected or an omission ?

[jcamu]

Hello. Any news on this subject ? How to use this chart to target Elastic Cloud instance and not Reference to ECK-managed Elasticsearch instance. Is it expected ? Schedule ? If not, what is the recommended solution for elastic cloud ?

[naemono]

@brsolomon-deloitte @jcamu Apologies for the lack of update here. The 1.0 version of ECK-Beats Helm chart was released with the 2.6.x version of ECK Operator on Jan 11. These Helm charts are released during the standard Elastic Stack release schedule, which is roughly every couple of months.

❯ helm repo list | grep 'helm\.elastic'
elastic                 https://helm.elastic.co
❯ helm search repo elastic/eck-beats
NAME                CHART VERSION   APP VERSION DESCRIPTION
elastic/eck-beats   0.1.0                       A Helm chart to deploy Elastic Beats managed by...

I'll investigate the documentation difference between the deprecated Beat Helm chart, and this ECK Beat Helm chart in relation to Elastic Cloud and update here.

[naemono]

To connect to Elastic Cloud Instance from Beat using ECK Helm Charts:

Create secret containing password for Beats to use

kubectl create secret -n default generic elastic-cloud-credentials --from-literal=output.elasticsearch.password=your-password

Example values.yaml file

name: filebeat-elastic-cloud
version: 8.6.1
spec:
  type: filebeat
  elasticsearchRef: null
  config:
    filebeat.inputs:
    - type: container
      paths:
      - /var/log/containers/*.log
    processors:
    - add_host_metadata: {}
    - add_cloud_metadata: {}
    output.elasticsearch.hosts:
    - https://your-fqdn.elastic-cloud.com
    output.elasticsearch.username: elastic
  secureSettings:
    - secretName: elastic-cloud-credentials
  daemonSet:
    podTemplate:
      spec:
        automountServiceAccountToken: true
        terminationGracePeriodSeconds: 30
        dnsPolicy: ClusterFirstWithHostNet
        hostNetwork: true # Allows to provide richer host metadata
        initContainers:
        - name: elastic-internal-init-keystore
          securityContext:
            privileged: false
            runAsUser: 0
        containers:
        - name: filebeat
          securityContext:
            runAsUser: 0
            # If using Red Hat OpenShift uncomment this:
            #privileged: true
          volumeMounts:
          - name: varlogcontainers
            mountPath: /var/log/containers
          - name: varlogpods
            mountPath: /var/log/pods
          - name: varlibdockercontainers
            mountPath: /var/lib/docker/containers
        volumes:
        - name: varlogcontainers
          hostPath:
            path: /var/log/containers
        - name: varlogpods
          hostPath:
            path: /var/log/pods
        - name: varlibdockercontainers
          hostPath:
            path: /var/lib/docker/containers

Install

helm upgrade -i eck-beats elastic -f ./examples/elastic-cloud.yaml -n default

I'll keep this open to get the official docs/examples updated.