Document APM setup with Fleet/Elastic Agent

[pebrc]

The APM integration for Fleet is replacing the standalone APM server. We should document how to set up APM through Fleet. How to create a service fronting the Agents, how to configure the secret token and maybe how to configure a client to talk to the APM integration for example Kibana itself. This could just be an amendment to our existing Fleet APM recipe.

Something along the lines of:

apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
  name: kibana
spec:
  version: 8.4.2
  count: 1
  elasticsearchRef:
    name: elasticsearch
  config:
    elastic:
      apm:
        active: true
        serverUrl: "http://apm.default.svc:8200"
        secretToken: apm-token
    xpack.fleet.agents.elasticsearch.hosts: ["https://elasticsearch-es-http.default.svc:9200"]
    xpack.fleet.agents.fleet_server.hosts: ["https://fleet-server-agent-http.default.svc:8220"]
    xpack.fleet.packages:
    - name: system
      version: latest
    - name: elastic_agent
      version: latest
    - name: fleet_server
      version: latest
    - name: apm
      version: latest
    xpack.fleet.agentPolicies:
    - name: Fleet Server on ECK policy
      id: eck-fleet-server
      is_default_fleet_server: true
      namespace: default
      monitoring_enabled:
      - logs
      - metrics
      unenroll_timeout: 900
      package_policies:
      - name: fleet_server-1
        id: fleet_server-1
        package:
          name: fleet_server
    - name: Elastic Agent on ECK policy
      id: eck-agent
      namespace: default
      monitoring_enabled:
      - logs
      - metrics
      unenroll_timeout: 900
      is_default: true
      package_policies:
      - name: system-1
        id: system-1
        package:
          name: system
      - package:
          name: apm
        name: apm-1
        inputs:
        - type: apm
          enabled: true
          vars:
          - name: host
            value: 0.0.0.0:8200
          - name: auth.secret_token
            value: apm-token
---
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: elasticsearch
spec:
  version: 8.4.2
  nodeSets:
  - name: default
    count: 3
    config:
      node.store.allow_mmap: false
---
apiVersion: agent.k8s.elastic.co/v1alpha1
kind: Agent
metadata:
  name: fleet-server
spec:
  version: 8.4.2
  kibanaRef:
    name: kibana
  elasticsearchRefs:
  - name: elasticsearch
  mode: fleet
  fleetServerEnabled: true
  deployment:
    replicas: 1
    podTemplate:
      spec:
        serviceAccountName: fleet-server
        automountServiceAccountToken: true
        securityContext:
          runAsUser: 0
---
apiVersion: agent.k8s.elastic.co/v1alpha1
kind: Agent
metadata: 
  name: elastic-agent
spec:
  version: 8.4.2
  kibanaRef:
    name: kibana
  fleetServerRef: 
    name: fleet-server
  mode: fleet
  deployment:
    replicas: 1
    podTemplate:
      spec:
        securityContext:
          runAsUser: 0
---
apiVersion: v1
kind: Service
metadata:
  name: apm
spec:
  selector:
    agent.k8s.elastic.co/name: elastic-agent
  ports:
  - protocol: TCP
    port: 8200
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: fleet-server
rules:
- apiGroups: [""]
  resources:
  - pods
  - namespaces
  - nodes
  verbs:
  - get
  - watch
  - list
- apiGroups: ["coordination.k8s.io"]
  resources:
  - leases
  verbs:
  - get
  - create
  - update
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: fleet-server
  namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: fleet-server
subjects:
- kind: ServiceAccount
  name: fleet-server
  namespace: default
roleRef:
  kind: ClusterRole
  name: fleet-server
  apiGroup: rbac.authorization.k8s.io
...

[fzyzcjy]

Hi, is there any updates? thanks

[thomazfohr]

Thanks for the example @pebrc, you saved my day!

In addition to the changes above I also had to expose port 8200 in the agents in order to get requests from apm client libraries.

apiVersion: agent.k8s.elastic.co/v1alpha1
kind: Agent
metadata: 
  name: elastic-agent
spec:
  version: 8.4.2
  kibanaRef:
    name: kibana
  fleetServerRef: 
    name: fleet-server
  mode: fleet
  deployment:
    replicas: 1
    podTemplate:
      spec:
        securityContext:
          runAsUser: 0
        containers:
          - name: agent  
            ports:
              - name: apm
                containerPort: 8200
                protocol: TCP

Although this works, it would be nice to have a complete example in the official docs.

[AnarchicSoul]

Sorry if I come years later...

I have some issues and cannot find information about it... I want enable SSL for APM but it seems not working. Tried same syntaxe used for secret-token but discovered that it's not working too...

      - name: Elastic Agent
        id: eck-agent
        namespace: plouf
        monitoring_enabled:
        - logs
        - metrics
        unenroll_timeout: 900
        package_policies:
        - package:
            name: apm
          name: apm-1
          inputs:
          - type: apm
            enabled: true
            vars:
            - name: host
              value: 0.0.0.0:8200
            - name: auth.secret_token 
              value: apm-token
            - name: enable_rum
              value: true
            - name: ssl.enabled 
              value: true
            - name: ssl.certificate 
              value: /usr/share/elastic-agent/apm/tls.crt 
            - name: ssl.key 
              value: /usr/share/elastic-agent/apm/tls.key 

auth.secret_token, ssl.enabled, ssl.certificate and ssl.key are not working. I've manage to add certs correctly on containers. And it's working if I add ssl configuration directly with Kibana UI.

[mileskelsey]

It's amazing to me that this is a supported configuration for ECK but still undocumented. I have been struggling to get it working for days. Seems like maybe I need to expose port 8200 in the Elastic Agent configuration. That would be an important thing to put in the configuration documentation, if only that documentation existed.