Support conditions based autodiscover for Docker containers for Fleet managed Agents

[pgeyman]

Describe the enhancement: There is documentation that describes how to configure standalone Elastic Agents with autodiscover, but nothing about how to use this for Fleet Managed agents.

Provide support for configuring conditions based autodiscover for Docker containers for Fleet managed Agents that will allow processing the docker log json files for different autodiscovered Docker container apps (eg. Apache HTTP, MySQL, Spring Boot App).

If this functionality does exist, please provide documentation on how to configure it for Fleet Managed Agents.

Describe a specific use case for the enhancement or feature: We are currently using Filebeat with autodiscover to send logs from Docker containers of different flavours (Apache HTTP, MySQL, etc).

We are now trying to migrate over to Elastic Agent with Fleet managed agents, but I cannot find any documentation or examples of how to configure the agents with an equivalent setup to what we have with Filebeat and autodiscover.

We need to be able to setup a Fleet Agent Policy that will process the docker log json files for different autodiscovered Docker container apps (eg. Apache HTTP, MySQL, Spring Boot App)

Example filebeat autodiscover setup:

filebeat.autodiscover:
  # List of enabled autodiscover providers
  providers:
    - type: docker
      templates:
        - condition:
            equals.docker.container.labels.com.docker.compose.service: apache
          config:
            - module: apache
              access:
                input:
                  type: container
                  stream: stdout
                  paths:
                    - "/var/lib/docker/containers/${data.docker.container.id}/*-json.log"
              error:
                input:
                  type: container
                  stream: stderr
                  paths:
                    - "/var/lib/docker/containers/${data.docker.container.id}/*-json.log"
        - condition:
            equals.docker.container.labels.com.docker.compose.service: webapp
          config:
            - type: container
              paths:
                - "/var/lib/docker/containers/${data.docker.container.id}/*-json.log"
              multiline.pattern: '^[[:space:]]+(at|\.{3})\b|^Caused by:'
              multiline.negate: false
              multiline.match: after
        - condition:
            equals.docker.container.labels.com.docker.compose.service: mysql
          config:
            - module: mysql
              error:
                input:
                  type: container
                  paths:
                    - "/var/lib/docker/containers/${data.docker.container.id}/*-json.log"
              slowlog: 
                enabled: false

What is the definition of done? Being able to setup a Fleet Agent Policy that will process the docker log json files for different autodiscovered Docker container apps (eg. Apache HTTP, MySQL, Spring Boot App)?

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[pgeyman]

@cmacknz Is this something that is possible or that you will be adding in the near term?

[cmacknz]

Not near term, but we haven't forgotten that this limitation exists.