Kubernetes Integration with kibana operator

[hailrend]

Hi everybody. I'm facing a problem with the kubernetes integration on the Kibana operator instance. I should modify the default values of the integration, but I cannot understand what the structure of the inputs is, could somebody explain it to me or guide me to a reference? I should practically add some processors and enable some other fields

This is the whole Kibana crd

apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
  name: kibana
  namespace: analytics
spec:
  version: 8.5.3
  count: 1
  elasticsearchRef:
    name: elasticsearch
  http:
    tls:
      selfSignedCertificate:
        disabled: true
  podTemplate:
    spec:
      containers:
        - name: kibana
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /status
              port: 5601
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 5
          env:
            - name: ELASTIC_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: elasticsearch-es-elastic-user
                  key: elastic
  config:
    xpack:
      security.authc.providers:
        anonymous.anonymous1:
          order: 0
          credentials:
            username: "elastic"
            password: "${ELASTIC_PASSWORD}"
    xpack.fleet.agents.elasticsearch.hosts: ["https://elasticsearch-es-http.analytics.svc:9200"]
    xpack.fleet.agents.fleet_server.hosts: ["https://fleet-server-agent-http.analytics.svc:8220"]
    xpack.fleet.packages:
      - name: system
        version: latest
      - name: elastic_agent
        version: latest
      - name: fleet_server
        version: latest
      - name: kubernetes
        version: latest
      - name: apm
        version: latest
    xpack.fleet.agentPolicies:
      - name: Fleet Server on ECK policy
        id: eck-fleet-server-policy
        namespace: analytics
        monitoring_enabled:
        - logs
        - metrics
        unenroll_timeout: 900
        package_policies:
        - name: fleet_server-1
          id: fleet_server-1
          package:
            name: fleet_server
      - name: Elastic Agent on ECK policy
        id: eck-agent
        namespace: analytics
        monitoring_enabled:
        - logs
        - metrics
        unenroll_timeout: 900
        package_policies:
        - name: system-1
          id: system-1
          package:
            name: system
        - package:
            name: apm
          name: apm-1
          id: apm-1
          inputs:
          - type: apm
            enabled: true
            vars:
            - name: host
              value: apm-server-apm-http.analytics:8200
        - name: elastic_agent-1
          id: elastic_agent-1
          package:
            name: elastic_agent
        - name: kubernetes-1
          id: kubernetes-1
          package:
            name: kubernetes
          inputs:
          - enabled: true
            type: container_logs
            vars:
            - name: processors
              value:
              - add_fields:
                  fields:
                    name: ${env.KUBERNETES_CLUSTER}
                  target: kubernetes.cluster

The part that is driving me crazy is this one

       - name: kubernetes-1
          id: kubernetes-1
          package:
            name: kubernetes
          inputs:
          - enabled: true
            type: container_logs
            vars:
            - name: processors
              value:
              - add_fields:
                  fields:
                    name: ${env.KUBERNETES_CLUSTER}
                  target: kubernetes.cluster

How am I supposed to configure the inputs?

[lfauchille]

Hey @hailrend,

I just have succeeded using this configuration:

        package_policies:
        - package:
            name: kubernetes
          name: kubernetes
          inputs:
            - type: filestream
              enabled: true
              streams:
                - data_stream:
                    dataset: kubernetes.container_logs
                  enabled: true
                  vars:
                    - name: processors
                      value: |-
                        - add_fields:
                            fields:
                              name: ${env.KUBERNETES_CLUSTER}
                            target: kubernetes.cluster

Hope it helps

NB: I'm also using the ECK Operator and what drove me crazy is that updating a policy does not work, you have to recreate it to see the modifications

[botelastic[bot]]

Hi! We just realized that we haven't looked into this issue in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!