DataDog / datadog-agent

Main repository for Datadog Agent
https://docs.datadoghq.com/
Apache License 2.0
2.84k stars 1.19k forks source link

[BUG] unable to specify log port in docker label based configuration #20731

Open kflan-hrp opened 10 months ago

kflan-hrp commented 10 months ago

I'm trying to run a datadog agent to allow for log collecting from another service we run. I'd like to do this without doing a docker build just to add a yaml file to the container for a few lines of configuration. So I'm trying to do it from a docker label.

Agent Environment agent version 7.48.1 from the public repository

env var: { "name": "DD_LOGS_ENABLED", "value": "true" },

AWS Fargate Task Container dockerLabels "com.datadoghq.ad.logs": "[{\"source\":\"abc\",\"service\":\"abc\",\"type\":\"tcp\",\"port\":10518}]",

Error logs:

        "message": "Unknown type: float64",
        "attributes": {
            "agent": "core",
            "file": "pkg/autodiscovery/configresolver/configresolver.go",
            "func": "resolveDataWithTemplateVars",
            "level": "ERROR",
            "line": "285",
            "source": "stdout",
            "timestamp": 1699473886901

or when I try putting the port in quotes

        "message": "Invalid configuration: could not parse JSON logs config: json: cannot unmarshal string into Go struct field LogsConfig.Port of type int",
        "attributes": {
            "agent": "core",
            "file": "pkg/logs/schedulers/ad/scheduler.go",
            "func": "Schedule",
            "level": "WARN",
            "line": "74",
            "source": "stdout",
            "timestamp": 1699473809512

Docker repro .sh

docker run \
    -e "DD_API_KEY=${DD_API_KEY}" \
    -e DD_SITE=datadoghq.com \
    -e DD_LOGS_ENABLED=true \
    -e DD_LOG_LEVEL=INFO \
    -v /var/run/docker.sock:/var/run/docker.sock:ro \
    -l com.datadoghq.ad.logs='[{
        "source": "abc",
        "service": "abc",
        "type": "tcp",
        "port": "10518"
     }]' \
    -p 10518:10518 \
    gcr.io/datadoghq/agent:7.48.1
kflan-hrp commented 10 months ago

I found a workaround via forcing env var substitution, which gets the logs-agent to start, but I can't seem to send logs into that port. The connection from nc on MacOS just auto-closes, or the AWS Load Balance health check fails.

    -e LOG_PORT=10518 \
    -v /var/run/docker.sock:/var/run/docker.sock:ro \
    -l com.datadoghq.ad.logs='[{
        "source": "abc",
        "service": "abc",
        "type": "tcp",
        "port": "%%env_LOG_PORT%%"