search

DataDog / integrations-extras

Community developed integrations and plugins for the Datadog Agent.
BSD 3-Clause "New" or "Revised" License
254 stars 742 forks source link

NS1 Integration fails to load when run in datadog/agent container #1113

Closed medavisjr closed 2 years ago

medavisjr commented 2 years ago

Output of the info page

Getting the status from the agent.

===============
Agent (v7.32.4)
===============

  Status date: 2021-12-29 23:19:03.889 UTC (1640819943889)
  Agent start: 2021-12-29 23:01:21.184 UTC (1640818881184)
  Pid: 398
  Go Version: go1.16.7
  Python Version: 3.8.11
  Build arch: amd64
  Agent flavor: agent
  Check Runners: 4
  Log Level: info

  Paths
  =====
    Config File: /etc/datadog-agent/datadog.yaml
    conf.d: /etc/datadog-agent/conf.d
    checks.d: /etc/datadog-agent/checks.d

  Clocks
  ======
    NTP offset: -3µs
    System time: 2021-12-29 23:19:03.889 UTC (1640819943889)

  Host Info
  =========
    bootTime: 2021-12-21 22:32:38 UTC (1640125958000)
    kernelArch: x86_64
    kernelVersion: 5.4.156-83.273.amzn2.x86_64
    os: linux
    platform: ubuntu
    platformFamily: debian
    platformVersion: 21.04
    procs: 197
    uptime: 192h29m13s

  Hostnames
  =========
    ec2-hostname: ip-10-112-100-231.awscloud.lijit.com
    hostname: datadog-agent-internal-7f8fb5856b-ncbj7
    instance-id: i-0921550af5be64396
    socket-fqdn: datadog-agent-internal-7f8fb5856b-ncbj7
    socket-hostname: datadog-agent-internal-7f8fb5856b-ncbj7
    host tags:
      cluster_name:sovrn-aws-core-eks-blue
      kube_cluster_name:sovrn-aws-core-eks-blue
    hostname provider: os
    unused hostname providers:
      aws: not retrieving hostname from AWS: the host is not an ECS instance and other providers already retrieve non-default hostnames
      azure: azure_hostname_style is set to 'os'
      configuration/environment: hostname is empty
      container: Unable to get hostname from container API
      gce: unable to retrieve hostname from GCE: status code 404 trying to GET http://169.254.169.254/computeMetadata/v1/instance/hostname

  Metadata
  ========
    cloud_provider: AWS
    hostname_source: os

=========
Collector
=========

  Running Checks
  ==============

    cpu
    ---
      Instance ID: cpu [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/cpu.d/conf.yaml.default
      Total Runs: 68
      Metric Samples: Last Run: 9, Total: 605
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s
      Last Execution Date : 2021-12-29 23:18:50 UTC (1640819930000)
      Last Successful Execution Date : 2021-12-29 23:18:50 UTC (1640819930000)

    disk (4.4.0)
    ------------
      Instance ID: disk:e5dffb8bef24336f [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/disk.d/conf.yaml.default
      Total Runs: 69
      Metric Samples: Last Run: 196, Total: 13,524
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 110ms
      Last Execution Date : 2021-12-29 23:18:57 UTC (1640819937000)
      Last Successful Execution Date : 2021-12-29 23:18:57 UTC (1640819937000)

    file_handle
    -----------
      Instance ID: file_handle [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/file_handle.d/conf.yaml.default
      Total Runs: 68
      Metric Samples: Last Run: 5, Total: 340
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s
      Last Execution Date : 2021-12-29 23:18:49 UTC (1640819929000)
      Last Successful Execution Date : 2021-12-29 23:18:49 UTC (1640819929000)

    http_check (6.1.0)
    ------------------
      Instance ID: http_check:jenkins_ext_lijit_com:8a6db1fb6d9a2747 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/jenkins-ext-endpoint-dd-check.yaml
      Total Runs: 4
      Metric Samples: Last Run: 5, Total: 20
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 8
      Average Execution Time : 408ms
      Last Execution Date : 2021-12-29 23:16:51 UTC (1640819811000)
      Last Successful Execution Date : 2021-12-29 23:16:51 UTC (1640819811000)

      Instance ID: http_check:jenkins_lijit_com:638270e397119145 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/jenkins-int-endpoint-dd-check.yaml
      Total Runs: 3
      Metric Samples: Last Run: 5, Total: 15
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 6
      Average Execution Time : 566ms
      Last Execution Date : 2021-12-29 23:14:21 UTC (1640819661000)
      Last Successful Execution Date : 2021-12-29 23:14:21 UTC (1640819661000)

      Instance ID: http_check:jenkins_standby_lijit_com:91906e83794dfd9b [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/jenkins-int-endpoint-dd-check.yaml
      Total Runs: 3
      Metric Samples: Last Run: 5, Total: 15
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 6
      Average Execution Time : 107ms
      Last Execution Date : 2021-12-29 23:16:49 UTC (1640819809000)
      Last Successful Execution Date : 2021-12-29 23:16:49 UTC (1640819809000)

      Instance ID: http_check:vap1ams1:7ac180296d0e1278 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 3
      Metric Samples: Last Run: 5, Total: 15
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 6
      Average Execution Time : 456ms
      Last Execution Date : 2021-12-29 23:16:38 UTC (1640819798000)
      Last Successful Execution Date : 2021-12-29 23:16:38 UTC (1640819798000)

      Instance ID: http_check:vap1dca1:8a60c60b8cd91344 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 4
      Metric Samples: Last Run: 5, Total: 20
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 8
      Average Execution Time : 108ms
      Last Execution Date : 2021-12-29 23:18:56 UTC (1640819936000)
      Last Successful Execution Date : 2021-12-29 23:18:56 UTC (1640819936000)

      Instance ID: http_check:vap1ewr1:931c32f9d7720884 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 3
      Metric Samples: Last Run: 5, Total: 15
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 6
      Average Execution Time : 104ms
      Last Execution Date : 2021-12-29 23:14:14 UTC (1640819654000)
      Last Successful Execution Date : 2021-12-29 23:14:14 UTC (1640819654000)

      Instance ID: http_check:vap1ord1:efe4de8c2d8b4b2c [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 3
      Metric Samples: Last Run: 5, Total: 15
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 6
      Average Execution Time : 265ms
      Last Execution Date : 2021-12-29 23:16:31 UTC (1640819791000)
      Last Successful Execution Date : 2021-12-29 23:16:31 UTC (1640819791000)

      Instance ID: http_check:vap1sea1:997b482e457a6e58 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 3
      Metric Samples: Last Run: 5, Total: 15
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 6
      Average Execution Time : 379ms
      Last Execution Date : 2021-12-29 23:16:24 UTC (1640819784000)
      Last Successful Execution Date : 2021-12-29 23:16:24 UTC (1640819784000)

      Instance ID: http_check:vap1sfo1:a4d3a38af480e1a0 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 3
      Metric Samples: Last Run: 5, Total: 15
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 6
      Average Execution Time : 408ms
      Last Execution Date : 2021-12-29 23:14:19 UTC (1640819659000)
      Last Successful Execution Date : 2021-12-29 23:14:19 UTC (1640819659000)

      Instance ID: http_check:vap2ams1:43d7624f24194194 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 3
      Metric Samples: Last Run: 5, Total: 15
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 6
      Average Execution Time : 462ms
      Last Execution Date : 2021-12-29 23:14:07 UTC (1640819647000)
      Last Successful Execution Date : 2021-12-29 23:14:07 UTC (1640819647000)

      Instance ID: http_check:vap2dca1:7770248f5c779e94 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 3
      Metric Samples: Last Run: 5, Total: 15
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 6
      Average Execution Time : 637ms
      Last Execution Date : 2021-12-29 23:16:26 UTC (1640819786000)
      Last Successful Execution Date : 2021-12-29 23:16:26 UTC (1640819786000)

      Instance ID: http_check:vap2ewr1:95e58159338baff8 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 3
      Metric Samples: Last Run: 5, Total: 15
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 6
      Average Execution Time : 112ms
      Last Execution Date : 2021-12-29 23:16:43 UTC (1640819803000)
      Last Successful Execution Date : 2021-12-29 23:16:43 UTC (1640819803000)

      Instance ID: http_check:vap2ord1:1fb2f3bb16100348 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 69
      Metric Samples: Last Run: 5, Total: 345
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 138
      Average Execution Time : 268ms
      Last Execution Date : 2021-12-29 23:18:51 UTC (1640819931000)
      Last Successful Execution Date : 2021-12-29 23:18:51 UTC (1640819931000)

      Instance ID: http_check:vap2sea1:1e1c04e49c0e06ac [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 4
      Metric Samples: Last Run: 5, Total: 20
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 8
      Average Execution Time : 398ms
      Last Execution Date : 2021-12-29 23:18:53 UTC (1640819933000)
      Last Successful Execution Date : 2021-12-29 23:18:53 UTC (1640819933000)

      Instance ID: http_check:vap2sfo1:99049a9c87a80bd8 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 3
      Metric Samples: Last Run: 5, Total: 15
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 6
      Average Execution Time : 414ms
      Last Execution Date : 2021-12-29 23:16:48 UTC (1640819808000)
      Last Successful Execution Date : 2021-12-29 23:16:48 UTC (1640819808000)

      Instance ID: http_check:vap3ams1:22d16eed4cef8bb0 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 3
      Metric Samples: Last Run: 5, Total: 15
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 6
      Average Execution Time : 661ms
      Last Execution Date : 2021-12-29 23:16:36 UTC (1640819796000)
      Last Successful Execution Date : 2021-12-29 23:16:36 UTC (1640819796000)

      Instance ID: http_check:vap3dca1:b8044bc4a4b5e840 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 4
      Metric Samples: Last Run: 5, Total: 20
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 8
      Average Execution Time : 102ms
      Last Execution Date : 2021-12-29 23:18:54 UTC (1640819934000)
      Last Successful Execution Date : 2021-12-29 23:18:54 UTC (1640819934000)

      Instance ID: http_check:vap3ewr1:171f3a064b7f81d8 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 3
      Metric Samples: Last Run: 5, Total: 15
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 6
      Average Execution Time : 307ms
      Last Execution Date : 2021-12-29 23:14:12 UTC (1640819652000)
      Last Successful Execution Date : 2021-12-29 23:14:12 UTC (1640819652000)

      Instance ID: http_check:vap3ord1:72995529632f7018 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 4
      Metric Samples: Last Run: 5, Total: 20
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 8
      Average Execution Time : 712ms
      Last Execution Date : 2021-12-29 23:19:01 UTC (1640819941000)
      Last Successful Execution Date : 2021-12-29 23:19:01 UTC (1640819941000)

      Instance ID: http_check:vap3sfo1:bc52d2c040abd770 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 3
      Metric Samples: Last Run: 5, Total: 15
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 6
      Average Execution Time : 410ms
      Last Execution Date : 2021-12-29 23:14:17 UTC (1640819657000)
      Last Successful Execution Date : 2021-12-29 23:14:17 UTC (1640819657000)

      Instance ID: http_check:vap4ams1:dc940d175702c0fc [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 3
      Metric Samples: Last Run: 5, Total: 15
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 6
      Average Execution Time : 477ms
      Last Execution Date : 2021-12-29 23:14:05 UTC (1640819645000)
      Last Successful Execution Date : 2021-12-29 23:14:05 UTC (1640819645000)

      Instance ID: http_check:vap4ewr1:841ed11426570e44 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 3
      Metric Samples: Last Run: 5, Total: 15
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 6
      Average Execution Time : 104ms
      Last Execution Date : 2021-12-29 23:16:41 UTC (1640819801000)
      Last Successful Execution Date : 2021-12-29 23:16:41 UTC (1640819801000)

      Instance ID: http_check:vap4ord1:22d6d60ad34879c [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 3
      Metric Samples: Last Run: 5, Total: 15
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 6
      Average Execution Time : 265ms
      Last Execution Date : 2021-12-29 23:16:29 UTC (1640819789000)
      Last Successful Execution Date : 2021-12-29 23:16:29 UTC (1640819789000)

      Instance ID: http_check:vap4sfo1:212f468dd5b47a28 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 3
      Metric Samples: Last Run: 5, Total: 15
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 6
      Average Execution Time : 581ms
      Last Execution Date : 2021-12-29 23:16:46 UTC (1640819806000)
      Last Successful Execution Date : 2021-12-29 23:16:46 UTC (1640819806000)

      Instance ID: http_check:vap5ams1:7248ef579f08b580 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 3
      Metric Samples: Last Run: 5, Total: 15
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 6
      Average Execution Time : 502ms
      Last Execution Date : 2021-12-29 23:16:34 UTC (1640819794000)
      Last Successful Execution Date : 2021-12-29 23:16:34 UTC (1640819794000)

      Instance ID: http_check:vap5ewr1:812f2da7db72610c [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 3
      Metric Samples: Last Run: 5, Total: 15
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 6
      Average Execution Time : 599ms
      Last Execution Date : 2021-12-29 23:14:11 UTC (1640819651000)
      Last Successful Execution Date : 2021-12-29 23:14:11 UTC (1640819651000)

      Instance ID: http_check:vap5ord1:f9b59fc2d551132c [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 4
      Metric Samples: Last Run: 5, Total: 20
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 8
      Average Execution Time : 209ms
      Last Execution Date : 2021-12-29 23:18:58 UTC (1640819938000)
      Last Successful Execution Date : 2021-12-29 23:18:58 UTC (1640819938000)

      Instance ID: http_check:vap6ams1:972436407f4e6514 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 4
      Metric Samples: Last Run: 5, Total: 20
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 8
      Average Execution Time : 505ms
      Last Execution Date : 2021-12-29 23:19:03 UTC (1640819943000)
      Last Successful Execution Date : 2021-12-29 23:19:03 UTC (1640819943000)

      Instance ID: http_check:vap6ewr1:29bf7a5b017939a8 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 3
      Metric Samples: Last Run: 5, Total: 15
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 6
      Average Execution Time : 104ms
      Last Execution Date : 2021-12-29 23:16:39 UTC (1640819799000)
      Last Successful Execution Date : 2021-12-29 23:16:39 UTC (1640819799000)

      Instance ID: http_check:vap6ord1:44d92da9f1b2a3f0 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 3
      Metric Samples: Last Run: 2, Total: 6
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 6
      Average Execution Time : 20.076s
      Last Execution Date : 2021-12-29 23:16:47 UTC (1640819807000)
      Last Successful Execution Date : 2021-12-29 23:16:47 UTC (1640819807000)

      Instance ID: http_check:vap7ewr1:bc096a22abb477f8 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vaps-ext-endpoint-dd-check.yaml
      Total Runs: 3
      Metric Samples: Last Run: 5, Total: 15
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 6
      Average Execution Time : 114ms
      Last Execution Date : 2021-12-29 23:14:08 UTC (1640819648000)
      Last Successful Execution Date : 2021-12-29 23:14:08 UTC (1640819648000)

      Instance ID: http_check:vault_ext_lijit_com:151ba43d8194781f [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vault-ext-endpoint-dd-check.yaml
      Total Runs: 3
      Metric Samples: Last Run: 5, Total: 15
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 6
      Average Execution Time : 336ms
      Last Execution Date : 2021-12-29 23:16:21 UTC (1640819781000)
      Last Successful Execution Date : 2021-12-29 23:16:21 UTC (1640819781000)

      Instance ID: http_check:vault_lijit_com:6cacbea7d36fb43e [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vault-int-endpoint-dd-check.yaml
      Total Runs: 4
      Metric Samples: Last Run: 5, Total: 20
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 8
      Average Execution Time : 527ms
      Last Execution Date : 2021-12-29 23:18:51 UTC (1640819931000)
      Last Successful Execution Date : 2021-12-29 23:18:51 UTC (1640819931000)

      Instance ID: http_check:vault_standby_lijit_com:d32d413dff3f99fc [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/http_check.d/vault-int-endpoint-dd-check.yaml
      Total Runs: 3
      Metric Samples: Last Run: 5, Total: 15
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 2, Total: 6
      Average Execution Time : 102ms
      Last Execution Date : 2021-12-29 23:16:19 UTC (1640819779000)
      Last Successful Execution Date : 2021-12-29 23:16:19 UTC (1640819779000)

    io
    --
      Instance ID: io [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/io.d/conf.yaml.default
      Total Runs: 69
      Metric Samples: Last Run: 39, Total: 2,664
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s
      Last Execution Date : 2021-12-29 23:18:56 UTC (1640819936000)
      Last Successful Execution Date : 2021-12-29 23:18:56 UTC (1640819936000)

    kubelet (7.1.0)
    ---------------
      Instance ID: kubelet:5bbc63f3938c02f4 [ERROR]
      Configuration Source: file:/etc/datadog-agent/conf.d/kubelet.d/conf.yaml.default
      Total Runs: 52
      Metric Samples: Last Run: 0, Total: 0
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 1ms
      Last Execution Date : 2021-12-29 23:18:51 UTC (1640819931000)
      Last Successful Execution Date : Never
      Error: Unable to detect the kubelet URL automatically: impossible to reach Kubelet with host: . Please check if your setup requires kubelet_tls_verify = false. Activate debug logs to see all attempts made
      Traceback (most recent call last):
        File "/opt/datadog-agent/embedded/lib/python3.8/site-packages/datadog_checks/base/checks/base.py", line 1017, in run
          self.check(instance)
        File "/opt/datadog-agent/embedded/lib/python3.8/site-packages/datadog_checks/kubelet/kubelet.py", line 293, in check
          raise CheckException("Unable to detect the kubelet URL automatically: " + kubelet_conn_info.get('err', ''))
      datadog_checks.base.errors.CheckException: Unable to detect the kubelet URL automatically: impossible to reach Kubelet with host: . Please check if your setup requires kubelet_tls_verify = false. Activate debug logs to see all attempts made

    load
    ----
      Instance ID: load [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/load.d/conf.yaml.default
      Total Runs: 69
      Metric Samples: Last Run: 6, Total: 414
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s
      Last Execution Date : 2021-12-29 23:19:03 UTC (1640819943000)
      Last Successful Execution Date : 2021-12-29 23:19:03 UTC (1640819943000)

    memory
    ------
      Instance ID: memory [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/memory.d/conf.yaml.default
      Total Runs: 69
      Metric Samples: Last Run: 18, Total: 1,242
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s
      Last Execution Date : 2021-12-29 23:18:55 UTC (1640819935000)
      Last Successful Execution Date : 2021-12-29 23:18:55 UTC (1640819935000)

    network (2.4.0)
    ---------------
      Instance ID: network:d884b5186b651429 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/network.d/conf.yaml.default
      Total Runs: 69
      Metric Samples: Last Run: 67, Total: 4,623
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 2ms
      Last Execution Date : 2021-12-29 23:19:02 UTC (1640819942000)
      Last Successful Execution Date : 2021-12-29 23:19:02 UTC (1640819942000)

    ntp
    ---
      Instance ID: ntp:d884b5186b651429 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/ntp.d/conf.yaml.default
      Total Runs: 2
      Metric Samples: Last Run: 1, Total: 2
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 1, Total: 2
      Average Execution Time : 0s
      Last Execution Date : 2021-12-29 23:16:51 UTC (1640819811000)
      Last Successful Execution Date : 2021-12-29 23:16:51 UTC (1640819811000)

    tcp_check (4.1.0)
    -----------------
      Instance ID: tcp_check:SSH check:87339d56143e3c51 [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/tcp_check.d/ad_servers.yaml
      Total Runs: 68
      Metric Samples: Last Run: 1, Total: 68
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 1, Total: 68
      Average Execution Time : 10.006s
      Last Execution Date : 2021-12-29 23:18:53 UTC (1640819933000)
      Last Successful Execution Date : 2021-12-29 23:18:53 UTC (1640819933000)

    uptime
    ------
      Instance ID: uptime [OK]
      Configuration Source: file:/etc/datadog-agent/conf.d/uptime.d/conf.yaml.default
      Total Runs: 69
      Metric Samples: Last Run: 1, Total: 69
      Events: Last Run: 0, Total: 0
      Service Checks: Last Run: 0, Total: 0
      Average Execution Time : 0s
      Last Execution Date : 2021-12-29 23:18:54 UTC (1640819934000)
      Last Successful Execution Date : 2021-12-29 23:18:54 UTC (1640819934000)

  Loading Errors
  ==============
    ns1
    ---
      Core Check Loader:
        Check ns1 not found in Catalog

      JMX Check Loader:
        check is not a jmx check, or unable to determine if it's so

      Python Check Loader:
        unable to import module 'ns1': No module named 'ns1'

========
JMXFetch
========

  Information
  ==================
  Initialized checks
  ==================
    no checks

  Failed checks
  =============
    no checks

=========
Forwarder
=========

  Transactions
  ============
    Cluster: 0
    ClusterRole: 0
    ClusterRoleBinding: 0
    CronJob: 0
    DaemonSet: 0
    Deployment: 0
    Dropped: 0
    HighPriorityQueueFull: 0
    Job: 0
    Node: 0
    PersistentVolume: 0
    PersistentVolumeClaim: 0
    Pod: 0
    ReplicaSet: 0
    Requeued: 0
    Retried: 0
    RetryQueueSize: 0
    Role: 0
    RoleBinding: 0
    Service: 0
    ServiceAccount: 0
    StatefulSet: 0

  Transaction Successes
  =====================
    Total number: 148
    Successes By Endpoint:
      check_run_v1: 70
      intake: 8
      series_v1: 70

  API Keys status
  ===============
    API key ending with 3decb: API Key valid

==========
Endpoints
==========
  https://app.datadoghq.com - API Key ending with:
      - 3decb

==========
Logs Agent
==========

  Logs Agent is not running

=========
APM Agent
=========
  Status: Running
  Pid: 400
  Uptime: 1060 seconds
  Mem alloc: 8,280,168 bytes
  Hostname: datadog-agent-internal-7f8fb5856b-ncbj7
  Receiver: 0.0.0.0:8126
  Endpoints:
    https://trace.agent.datadoghq.com

  Receiver (previous minute)
  ==========================
    No traces received in the previous minute.
    Default priority sampling rate: 100.0%

  Writer (previous minute)
  ========================
    Traces: 0 payloads, 0 traces, 0 events, 0 bytes
    Stats: 0 payloads, 0 stats buckets, 0 bytes

=========
Aggregator
=========
  Checks Metric Sample: 26,118
  Dogstatsd Metric Sample: 8,369
  Event: 1
  Events Flushed: 1
  Number Of Flushes: 70
  Series Flushed: 18,435
  Service Check: 1,270
  Service Checks Flushed: 1,332
=========
DogStatsD
=========
  Event Packets: 0
  Event Parse Errors: 0
  Metric Packets: 8,368
  Metric Parse Errors: 0
  Service Check Packets: 0
  Service Check Parse Errors: 0
  Udp Bytes: 639,481
  Udp Packet Reading Errors: 0
  Udp Packets: 5,958
  Uds Bytes: 0
  Uds Origin Detection Errors: 0
  Uds Packet Reading Errors: 0
  Uds Packets: 0
  Unterminated Metric Errors: 0

=============
Autodiscovery
=============
  Enabled Features
  ================
    kubernetes

Additional environment details (Operating System, Cloud provider, etc):

Running in a standalone pod (non-DaemonSet) on an AWS EKS worker node. Worker nodes are running on Amazon Linux 2 kernel version 5.4.156-83.273.amzn2.x86_64, using the containerd://1.4.6 container runtime.

Steps to reproduce the issue:

  1. Install ns1 integration in container as described here: https://docs.datadoghq.com/agent/guide/use-community-integrations/?tab=agentv721v621
  2. Create (via k8s ConfigMap) a config file at /etc/datadog-agent/conf.d/ns1.d/conf.yaml with the following config:
init_config:
  service: ns1-stats
instances:
  - api_endpoint: https://api.nsone.net
    api_key: <redacted>
    metrics:
      qps:
        - lijit.com:
          - apa.vap: A
          - lam.vap: A
          - nae.vap: A
          - ane.vap: A
          - nase.vap: A
          - adc.vap: A
          - nac.vap: A
          - nace.vap: A
          - nad.vap: CNAME
          - namo.vap: CNAME
          - anac.vap: A
          - emea.vap: A
          - emeas.vap: CNAME
          - oeu.vap: A
          - aeu.vap: A
          - neu.vap: A
          - nava.vap: A
          - naw.vap: A
          - nanw.vap: A
          - asf.vap: A
          - vap: CNAME
          - aws: NS
          - pt: CNAME
          - www: CNAME
          - e-vap: CNAME
          - w-vap: CNAME
          - pnava.vap: CNAME
          - ap: CNAME
          - ce: CNAME
          - update.ap: CNAME
          - s.update.ap: CNAME
          - t.update.ap: CNAME
          - he: CNAME
          - gslbeacon: CNAME
      usage:
        - lijit.com:
          - apa.vap: A
          - lam.vap: A
          - nae.vap: A
          - ane.vap: A
          - nase.vap: A
          - adc.vap: A
          - nac.vap: A
          - nace.vap: A
          - nad.vap: CNAME
          - namo.vap: CNAME
          - anac.vap: A
          - emea.vap: A
          - emeas.vap: CNAME
          - oeu.vap: A
          - aeu.vap: A
          - neu.vap: A
          - nava.vap: A
          - naw.vap: A
          - nanw.vap: A
          - asf.vap: A
          - vap: CNAME
          - aws: NS
          - pt: CNAME
          - www: CNAME
          - e-vap: CNAME
          - w-vap: CNAME
          - pnava.vap: CNAME
          - ap: CNAME
          - ce: CNAME
          - update.ap: CNAME
          - s.update.ap: CNAME
          - t.update.ap: CNAME
          - he: CNAME
          - gslbeacon: CNAME
      account:
        - billing:
        - ttl:
      query_params:
        usage_networks: "*"
      min_collection_interval: 15
  1. Create the pod via a deployment resource

Describe the results you received:

No metrics appear are received by Datadog, and an agent status command run on the container reports the following:

Loading Errors
  ==============
    ns1
    ---
      Core Check Loader:
        Check ns1 not found in Catalog

      JMX Check Loader:
        check is not a jmx check, or unable to determine if it's so

      Python Check Loader:
        unable to import module 'ns1': No module named 'ns1'

However, checking agent integration show datadog-ns1 shows that the integration was installed successfully:

root@datadog-agent-internal-76488968c9-c2bld:/# agent integration show datadog-ns1
Package datadog-ns1:
Installed version: 0.0.5

Describe the results you expected:

We would expect the integration to be found and correctly loaded

Additional information you deem important (e.g. issue happens only occasionally):

We've tried this with both v0.0.4 and v0.0.5 (latest) of the ns1 integration, and both fail in the same way. This same config file was used on a baremetal server and was working well. Running in a container in EKS, however, we get this error.

medavisjr commented 2 years ago

Mentioning folks who have recently worked on this particular integration, as otherwise I have no idea if anyone will ever see this. :)

@rseste-daitan @cleosson-daitan

fanny-jiang commented 2 years ago

Hi @rodalli, thanks for reporting this issue.

Re this step:

Install ns1 integration in container as described here: https://docs.datadoghq.com/agent/guide/use-community-integrations/?tab=agentv721v621

Did you run the agent integration install command inside the agent container or did you build an updated image of the datadog-agent with the integration installed? The recommended method to install an integration from the integrations-extras repo in a containerized environment is to build a new version of the agent. See the Docker tab of the documentation: https://docs.datadoghq.com/agent/guide/use-community-integrations/?tab=docker. I was able to install the ns1 integration successfully using this method.

medavisjr commented 2 years ago

@fanny-jiang Forgot I had opened this issue - yes, I was trying to run the command inside the container before I noticed the recommendation to build a new image. I was able to get it working with the new image build. I'm not sure why this is necessary, maybe something to do with permissions for the datadog user.

I really don't like the overhead of having to manage the additional operation complexity and toil of keeping a redundant image up-to-date. My hope was that this could be done with a k8s init container in the same pod. Ideally, this would be as simple as adding some labels or annotations on the agent pod to get it to bootstrap the custom integration itself.