search

DataDog / integrations-core

Core integrations of the Datadog Agent
BSD 3-Clause "New" or "Revised" License
935 stars 1.4k forks source link

[auto discovery] `%%port%%` template/placeholder does not have the correct value #1380

Closed aaronjwhiteside closed 6 years ago

aaronjwhiteside commented 6 years ago

Output of the info page

Getting the status from the agent.

==============
Agent (v6.1.2)
==============

  Status date: 2018-04-12 00:07:15.824979 UTC
  Pid: 339
  Python Version: 2.7.14
  Logs: 
  Check Runners: 1
  Log Level: info

  Paths
  =====
    Config File: /etc/datadog-agent/datadog.yaml
    conf.d: /etc/datadog-agent/conf.d
    checks.d: /etc/datadog-agent/checks.d

  Clocks
  ======
    NTP offset: -0.003034407 s
    System UTC time: 2018-04-12 00:07:15.824979 UTC

  Host Info
  =========
    bootTime: 2018-04-11 00:51:39.000000 UTC
    kernelVersion: 4.9.75-25.55.amzn1.x86_64
    os: linux
    platform: debian
    platformFamily: debian
    platformVersion: 9.4
    procs: 73
    uptime: 75896
    virtualizationRole: guest
    virtualizationSystem: xen

  Hostnames
  =========
    ec2-hostname: ip-10-70-4-121.dv.xxxxx.com
    hostname: i-0f410813af785d1a3
    instance-id: i-0f410813af785d1a3
    socket-fqdn: ip-10-70-4-121
    socket-hostname: ip-10-70-4-121

=========
Collector
=========

  Running Checks
  ==============
    cpu
    ---
      Total Runs: 522
      Metrics: 6, Total Metrics: 3126
      Events: 0, Total Events: 0
      Service Checks: 0, Total Service Checks: 0

    disk
    ----
      Total Runs: 522
      Metrics: 136, Total Metrics: 70992
      Events: 0, Total Events: 0
      Service Checks: 0, Total Service Checks: 0

    docker
    ------
      Total Runs: 522
      Metrics: 53, Total Metrics: 27666
      Events: 0, Total Events: 8
      Service Checks: 1, Total Service Checks: 522

    file_handle
    -----------
      Total Runs: 522
      Metrics: 1, Total Metrics: 522
      Events: 0, Total Events: 0
      Service Checks: 0, Total Service Checks: 0

    http_check
    ----------
      Total Runs: 522
      Metrics: 4, Total Metrics: 2088
      Events: 0, Total Events: 0
      Service Checks: 2, Total Service Checks: 1044
      Warning: Skipping SSL certificate validation for https://172.17.0.2:8443/xxxxxx/health based on configuration

    io
    --
      Total Runs: 522
      Metrics: 130, Total Metrics: 67770
      Events: 0, Total Events: 0
      Service Checks: 0, Total Service Checks: 0

    load
    ----
      Total Runs: 522
      Metrics: 6, Total Metrics: 3132
      Events: 0, Total Events: 0
      Service Checks: 0, Total Service Checks: 0

    memory
    ------
      Total Runs: 522
      Metrics: 14, Total Metrics: 7308
      Events: 0, Total Events: 0
      Service Checks: 0, Total Service Checks: 0

    network
    -------
      Total Runs: 522
      Metrics: 32, Total Metrics: 16704
      Events: 0, Total Events: 0
      Service Checks: 0, Total Service Checks: 0

    ntp
    ---
      Total Runs: 522
      Metrics: 1, Total Metrics: 515
      Events: 0, Total Events: 0
      Service Checks: 1, Total Service Checks: 522

    uptime
    ------
      Total Runs: 522
      Metrics: 1, Total Metrics: 522
      Events: 0, Total Events: 0
      Service Checks: 0, Total Service Checks: 0

========
JMXFetch
========

  Initialized checks
  ==================
    no checks

  Failed checks
  =============
    no checks

=========
Forwarder
=========

  CheckRunsV1: 522
  IntakeV1: 49
  RetryQueueSize: 0
  Success: 1093
  TimeseriesV1: 522

  API Keys status
  ===============
    https://6-1-2-app.agent.datadoghq.com,*************************c64e7: API Key valid

==========
Logs Agent
==========

  Logs Agent is not running

=========
DogStatsD
=========

  Checks Metric Sample: 211828
  Event: 9
  Events Flushed: 9
  Number Of Flushes: 522
  Series Flushed: 156078
  Service Check: 7830
  Service Checks Flushed: 8337
  Dogstatsd Metric Sample: 2090

Additional environment details (Operating System, Cloud provider, etc):

# uname -a
Linux ip-10-70-4-121 4.9.75-25.55.amzn1.x86_64 #1 SMP Fri Jan 5 23:50:27 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Steps to reproduce the issue: Given the following container (I've retracted chunks of this, so it may not be valid json):

# docker inspect b1e6d192814f
[
    {
        "Id": "b1e6d192814f7432b7e9b0c527d9b82fdd6db3015210dd4d3980ac833ce5d073",
        "Created": "2018-04-11T00:57:25.911434321Z",
        "Path": "/bin/sh",
        "Args": [
            "-c",
            "exec /usr/bin/supervisord -c /etc/supervisord.conf -e $SUPERVISOR_LOG_LEVEL"
        ],
        "State": {
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 3994,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2018-04-11T00:57:27.632656188Z",
            "FinishedAt": "0001-01-01T00:00:00Z",
            "Health": {
                "Status": "healthy",
                "FailingStreak": 0,
                "Log": [
                    {
                        "Start": "2018-04-12T00:04:05.284713554Z",
                        "End": "2018-04-12T00:04:05.337687995Z",
                        "ExitCode": 0,
                        "Output": ""
                    },
                    {
                        "Start": "2018-04-12T00:04:10.33788445Z",
                        "End": "2018-04-12T00:04:10.390478072Z",
                        "ExitCode": 0,
                        "Output": ""
                    },
                    {
                        "Start": "2018-04-12T00:04:15.390671218Z",
                        "End": "2018-04-12T00:04:15.443394217Z",
                        "ExitCode": 0,
                        "Output": ""
                    },
                    {
                        "Start": "2018-04-12T00:04:20.44360813Z",
                        "End": "2018-04-12T00:04:20.496204751Z",
                        "ExitCode": 0,
                        "Output": ""
                    },
                    {
                        "Start": "2018-04-12T00:04:25.496405185Z",
                        "End": "2018-04-12T00:04:25.548918737Z",
                        "ExitCode": 0,
                        "Output": ""
                    }
                ]
            }
        },
        "Image": "sha256:e4b619ab79bd7f0a51bee2e2bc8d2ae67e2a906efdf528f439bb7137302eec2c",
        "ResolvConfPath": "/var/lib/docker/containers/b1e6d192814f7432b7e9b0c527d9b82fdd6db3015210dd4d3980ac833ce5d073/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/b1e6d192814f7432b7e9b0c527d9b82fdd6db3015210dd4d3980ac833ce5d073/hostname",
        "HostsPath": "/var/lib/docker/containers/b1e6d192814f7432b7e9b0c527d9b82fdd6db3015210dd4d3980ac833ce5d073/hosts",
        "LogPath": "",
        "Name": "/xxxxxxxx",
        "RestartCount": 0,
        "Driver": "devicemapper",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": null,
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "awslogs",
                "Config": {
                    "awslogs-group": "xxxxx",
                    "awslogs-region": "xxxxx"
                }
            },
            "NetworkMode": "bridge",
            "PortBindings": {
                "8443/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "0"
                    }
                ]
            },
            "RestartPolicy": {
                "Name": "",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "Dns": null,
            "DnsOptions": null,
            "DnsSearch": null,
            "ExtraHosts": [
            ],
            "GroupAdd": null,
            "IpcMode": "",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 10,
            "Memory": 1610612736,
            "NanoCpus": 0,
            "CgroupParent": "/ecs/812edcba-af08-4d72-b696-b321cd4443ac",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": null,
            "DeviceCgroupRules": null,
            "DiskQuota": 0,
            "KernelMemory": 0,
            "MemoryReservation": 0,
            "MemorySwap": 3221225472,
            "MemorySwappiness": 0,
            "OomKillDisable": false,
            "PidsLimit": 0,
            "Ulimits": [
                {
                    "Name": "nofile",
                    "Hard": 4096,
                    "Soft": 1024
                }
            ],
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0
        },
        "GraphDriver": {
            "Data": {
                "DeviceId": "23885",
                "DeviceName": "docker-202:1-263303-5a171545f20bf79a62beeba2db1c551c38c274e53035acef0b3ef15851b7a0e4",
                "DeviceSize": "10737418240"
            },
            "Name": "devicemapper"
        },
        "Mounts": [
            {
                "Type": "volume",
                "Name": "9fd9bb8be006ce0b65e3709152456d39f9f5c4d033c8844410d5793de0173b7c",
                "Source": "/var/lib/docker/volumes/9fd9bb8be006ce0b65e3709152456d39f9f5c4d033c8844410d5793de0173b7c/_data",
                "Destination": "/var/log",
                "Driver": "local",
                "Mode": "",
                "RW": true,
                "Propagation": ""
            }
        ],
        "Config": {
            "Hostname": "",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "ExposedPorts": {
                "8080/tcp": {},
                "8443/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
            ],
            "Cmd": null,
            "Healthcheck": {
                "Test": [
                    "CMD-SHELL",
                    "wget -qO /dev/null \"http://localhost:${SERVER_PORT}/xxxxxx/health\" || exit 1"
                ],
                "Interval": 5000000000,
                "Timeout": 3000000000,
                "Retries": 12
            },
            "ArgsEscaped": true,
            "Image": "maven.it.xxxxxx.com:5000/xxxxxx/xxxxxx:1.5.0-SNAPSHOT",
            "Volumes": {
                "/var/log": {}
            },
            "WorkingDir": "",
            "Entrypoint": [
                "/bin/sh",
                "-c",
                "exec /usr/bin/supervisord -c /etc/supervisord.conf -e $SUPERVISOR_LOG_LEVEL"
            ],
            "OnBuild": null,
            "Labels": {
                "architecture": "x86_64",
                "com.amazonaws.ecs.cluster": "xxxxx",
                "com.amazonaws.ecs.container-name": "xxxxx",
                "com.amazonaws.ecs.task-arn": "xxxxx",
                "com.amazonaws.ecs.task-definition-family": "xxxxxx",
                "com.amazonaws.ecs.task-definition-version": "23"
            }
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "8011774d4379501f167e23f391e224d3584400508a55872addbcf3705a711254",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "8080/tcp": null,
                "8443/tcp": [
                    {
                        "HostIp": "0.0.0.0",
                        "HostPort": "32768"
                    }
                ]
            },
            "SandboxKey": "/var/run/docker/netns/8011774d4379",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "27e2e2d529cf54e56a8b6dfc6870c78c6fc1cb5b9b7ac5dfb5b3b9c5bd6f4838",
            "Gateway": "172.17.0.1",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "172.17.0.2",
            "IPPrefixLen": 16,
            "IPv6Gateway": "",
            "MacAddress": "02:42:ac:11:00:02",
            "Networks": {
                "bridge": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "96126dd29b26f4bd91743ac5678a5a5551f3235ab511ee578e86fb77cc1ff2ee",
                    "EndpointID": "27e2e2d529cf54e56a8b6dfc6870c78c6fc1cb5b9b7ac5dfb5b3b9c5bd6f4838",
                    "Gateway": "172.17.0.1",
                    "IPAddress": "172.17.0.2",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:11:00:02",
                    "DriverOpts": null
                }
            }
        }
    }
]

And the following template:

ad_identifiers:
    - xxxxxx

init_config:

instances:
    - name: xxxxxx
      url: https://%%host%%:%%port%%/xxxxx/health
      timeout: 5
      http_response_status_code: 2\d\d
      include_content: true
      skip_proxy: true

Describe the results you received: %%port%% is ALWAYS replaced with 8080, despite 8443 being the highest exposed port. No matter what I try, %%port_0%% or %%port_1%% it's always 8080, never 8443.

Describe the results you expected: %%port%% to be replaced with the highest exposed port 8443 %%port_0%% to be replaced with 8080 %%port_1%% to be replaced with 8443

Additional information you deem important (e.g. issue happens only occasionally): Always reproducible.

aaronjwhiteside commented 6 years ago

Closing issue, I opened it on the wrong repo, sorry