search

DataDog / datadog-agent

Main repository for Datadog Agent
https://docs.datadoghq.com/
Apache License 2.0
2.75k stars 1.18k forks source link

Kakfa integration appears broken in 6.3.3 #2062

Open epinzur opened 5 years ago

epinzur commented 5 years ago

Note that your link for the getting the info page is broken: https://help.datadoghq.com/hc/en-us/articles/203764635-Agent-Status-and-Information

Output of the info page (if this is a bug)

Getting the status from the agent.

==============
Agent (v6.3.3)
==============

  Status date: 2018-07-25 16:49:16.273987 UTC
  Pid: 8530
  Python Version: 2.7.14
  Logs: /var/log/datadog/agent.log
  Check Runners: 3
  Log Level: info

  Paths
  =====
    Config File: /etc/datadog-agent/datadog.yaml
    conf.d: /etc/datadog-agent/conf.d
    checks.d: /etc/datadog-agent/checks.d

  Clocks
  ======
    NTP offset: -0.000396109 s
    System UTC time: 2018-07-25 16:49:16.273987 UTC

  Host Info
  =========
    bootTime: 2018-07-23 21:11:58.000000 UTC
    kernelVersion: 4.14.47-56.37.amzn1.x86_64
    os: linux
    platform: amazon
    platformFamily: rhel
    platformVersion: 2018.03
    procs: 150
    uptime: 156777
    virtualizationRole: guest
    virtualizationSystem: xen

  Hostnames
  =========
    ec2-hostname: ip-10-50-26-44.ec2.internal
    hostname: kafka-test-4.prod.internal.cio
    instance-id: i-05af9024c16172131
    socket-fqdn: ip-10-50-26-44.ec2.internal.
    socket-hostname: ip-10-50-26-44

=========
Collector
=========

  Running Checks
  ==============
    cpu
    ---
      Total Runs: 17
      Metric Samples: 6, Total: 96
      Events: 0, Total: 0
      Service Checks: 0, Total: 0
      Average Execution Time : 0ms

    disk
    ----
      Total Runs: 17
      Metric Samples: 36, Total: 612
      Events: 0, Total: 0
      Service Checks: 0, Total: 0
      Average Execution Time : 17ms

    file_handle
    -----------
      Total Runs: 17
      Metric Samples: 1, Total: 17
      Events: 0, Total: 0
      Service Checks: 0, Total: 0
      Average Execution Time : 0ms

    io
    --
      Total Runs: 17
      Metric Samples: 39, Total: 636
      Events: 0, Total: 0
      Service Checks: 0, Total: 0
      Average Execution Time : 2ms

    kafka_consumer
    --------------
      Total Runs: 17
      Metric Samples: 0, Total: 0
      Events: 0, Total: 0
      Service Checks: 0, Total: 0
      Average Execution Time : 38ms

    load
    ----
      Total Runs: 17
      Metric Samples: 6, Total: 102
      Events: 0, Total: 0
      Service Checks: 0, Total: 0
      Average Execution Time : 0ms

    memory
    ------
      Total Runs: 17
      Metric Samples: 17, Total: 289
      Events: 0, Total: 0
      Service Checks: 0, Total: 0
      Average Execution Time : 0ms

    network
    -------
      Total Runs: 17
      Metric Samples: 20, Total: 340
      Events: 0, Total: 0
      Service Checks: 0, Total: 0
      Average Execution Time : 0ms

    ntp
    ---
      Total Runs: 17
      Metric Samples: 1, Total: 17
      Events: 0, Total: 0
      Service Checks: 1, Total: 17
      Average Execution Time : 31ms

    uptime
    ------
      Total Runs: 17
      Metric Samples: 1, Total: 17
      Events: 0, Total: 0
      Service Checks: 0, Total: 0
      Average Execution Time : 0ms

========
JMXFetch
========

  Initialized checks
  ==================
    jmx
      instance_name : jmx-localhost-9999
      message :
      metric_count : 13
      service_check_count : 0
      status : OK
    kafka
      instance_name : kafka-localhost-9999
      message :
      metric_count : 45
      service_check_count : 0
      status : OK
  Failed checks
  =============
    no checks

=========
Forwarder
=========

  CheckRunsV1: 17
  DroppedOnInput: 0
  IntakeV1: 2
  RetryQueueSize: 0
  Success: 36
  TimeseriesV1: 17

  API Keys status
  ===============
    https://6-3-3-app.agent.datadoghq.com,*************************19462: API Key valid

==========
Logs Agent
==========

  Logs Agent is not running

=========
DogStatsD
=========

  Checks Metric Sample: 2465
  Event: 1
  Events Flushed: 1
  Number Of Flushes: 17
  Series Flushed: 1980
  Service Check: 187
  Service Checks Flushed: 193

Describe what happened: I've been struggling at getting the kafka integration working. I'm currently using the default config with no modifications. (I copied datadog-agent/conf.d/kafka.d/conf.yaml.example to datadog-agent/conf.d/kafka.d/conf.yaml)

When I try to run just the kafka check with debug logging enabled, I get this:

>sudo datadog-agent check kafka -l debug
2018-07-25 16:52:34 UTC | INFO | (tagger.go:79 in Init) | starting the tagging system
2018-07-25 16:52:35 UTC | DEBUG | (tagger.go:149 in tryCollectors) | ecs_fargate tag collector cannot start: Failed to connect to task metadata API, ECS tagging will not work
2018-07-25 16:52:35 UTC | DEBUG | (ecs.go:79 in GetUtil) | ECS init error: temporary failure in ecsutil, will retry later: could not detect ECS agent, tried URLs: [http://localhost:51678/]
2018-07-25 16:52:35 UTC | DEBUG | (tagger.go:145 in tryCollectors) | will retry ecs later: temporary failure in ecsutil, will retry later: could not detect ECS agent, tried URLs: [http://localhost:51678/]
2018-07-25 16:52:35 UTC | DEBUG | (global.go:49 in GetDockerUtil) | Docker init error: temporary failure in dockerutil, will retry later: could not determine docker server API version: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
2018-07-25 16:52:35 UTC | DEBUG | (kubelet.go:88 in GetKubeUtil) | Init error: temporary failure in kubeutil, will retry later: unable to get hostname from docker, please set the kubernetes_kubelet_host option: temporary failure in dockerutil, will retry later: could not determine docker server API version: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
2018-07-25 16:52:35 UTC | DEBUG | (tagger.go:145 in tryCollectors) | will retry kubelet later: temporary failure in kubeutil, will retry later: unable to get hostname from docker, please set the kubernetes_kubelet_host option: temporary failure in dockerutil, will retry later: could not determine docker server API version: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
2018-07-25 16:52:35 UTC | DEBUG | (kubelet.go:88 in GetKubeUtil) | Init error: temporary failure in kubeutil, will retry later: try delay not elapsed yet
2018-07-25 16:52:35 UTC | DEBUG | (tagger.go:145 in tryCollectors) | will retry kube-metadata-collector later: temporary failure in kubeutil, will retry later: try delay not elapsed yet
2018-07-25 16:52:35 UTC | DEBUG | (global.go:49 in GetDockerUtil) | Docker init error: temporary failure in dockerutil, will retry later: try delay not elapsed yet
2018-07-25 16:52:35 UTC | DEBUG | (tagger.go:145 in tryCollectors) | will retry docker later: temporary failure in dockerutil, will retry later: try delay not elapsed yet
2018-07-25 16:52:35 UTC | INFO | (runner.go:92 in NewRunner) | Runner started with 1 workers.
2018-07-25 16:52:35 UTC | DEBUG | (scheduler.go:125 in func1) | Starting scheduler loop...
2018-07-25 16:52:35 UTC | DEBUG | (runner.go:226 in work) | Ready to process checks...
2018-07-25 16:52:35 UTC | INFO | (collector.go:52 in NewCollector) | Embedding Python 2.7.14 (default, Jul 16 2018, 19:01:07) [GCC 4.4.7 20120313 (Red Hat 4.4.7-18)]
2018-07-25 16:52:35 UTC | DEBUG | (collector.go:53 in NewCollector) | Python Home: /opt/datadog-agent/embedded
2018-07-25 16:52:35 UTC | DEBUG | (collector.go:54 in NewCollector) | Python path: ['/opt/datadog-agent/embedded/lib/python27.zip', '/opt/datadog-agent/embedded/lib/python2.7', '/opt/datadog-agent/embedded/lib/python2.7/plat-linux2', '/opt/datadog-agent/embedded/lib/python2.7/lib-tk', '/opt/datadog-agent/embedded/lib/python2.7/lib-old', '/opt/datadog-agent/embedded/lib/python2.7/lib-dynload', '/opt/datadog-agent/embedded/lib/python2.7/site-packages', '/opt/datadog-agent/embedded/lib/python2.7/site-packages/setuptools-28.8.0.post20180716-py2.7.egg', '/opt/datadog-agent/bin/agent/dist', '/opt/datadog-agent/checks.d', '/opt/datadog-agent/bin/agent/dist/checks.d', '/etc/datadog-agent/checks.d']
2018-07-25 16:52:35 UTC | DEBUG | (collector.go:63 in NewCollector) | Collector up and running!
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:41 in SetupAutoConfig) | Added Python Check Loader to AutoConfig
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:41 in SetupAutoConfig) | Added Core Check Loader to AutoConfig
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:41 in SetupAutoConfig) | Added JMX Check Loader to AutoConfig
2018-07-25 16:52:35 UTC | INFO | (file.go:69 in Collect) | File Configuration Provider: searching for configuration files at: /etc/datadog-agent/conf.d
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/activemq.d/metrics.yaml
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/apache.d/auto_conf.yaml
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/cassandra.d/metrics.yaml
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/consul.d/auto_conf.yaml
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/couch.d/auto_conf.yaml
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/couchbase.d/auto_conf.yaml
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/cpu.d/conf.yaml.default
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/disk.d/conf.yaml.default
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/elastic.d/auto_conf.yaml
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/etcd.d/auto_conf.yaml
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/file_handle.d/conf.yaml.default
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/io.d/conf.yaml.default
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/jmx.d/conf.yaml
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/kafka.d/conf.yaml
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/kafka.d/metrics.yaml
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/kafka_consumer.d/conf.yaml
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/kube_dns.d/auto_conf.yaml
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/kube_proxy.d/auto_conf.yaml
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/kubernetes_state.d/auto_conf.yaml
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/kyototycoon.d/auto_conf.yaml
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/load.d/conf.yaml.default
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/mcache.d/auto_conf.yaml
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/memory.d/conf.yaml.default
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/network.d/conf.yaml.default
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/ntp.d/conf.yaml.default
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/redisdb.d/auto_conf.yaml
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/riak.d/auto_conf.yaml
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/solr.d/metrics.yaml
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/tomcat.d/auto_conf.yaml
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/tomcat.d/metrics.yaml
2018-07-25 16:52:35 UTC | DEBUG | (file.go:191 in collectEntry) | Found valid configuration in file: /etc/datadog-agent/conf.d/uptime.d/conf.yaml.default
2018-07-25 16:52:35 UTC | INFO | (file.go:69 in Collect) | File Configuration Provider: searching for configuration files at: /opt/datadog-agent/bin/agent/dist/conf.d
2018-07-25 16:52:35 UTC | WARN | (file.go:73 in Collect) | Skipping, open /opt/datadog-agent/bin/agent/dist/conf.d: no such file or directory
2018-07-25 16:52:35 UTC | DEBUG | (configresolver.go:124 in ResolveTemplate) | No service found with this AD identifier: httpd
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:317 in resolve) | Can't resolve the template for apache at this moment.
2018-07-25 16:52:35 UTC | DEBUG | (configresolver.go:124 in ResolveTemplate) | No service found with this AD identifier: consul
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:317 in resolve) | Can't resolve the template for consul at this moment.
2018-07-25 16:52:35 UTC | DEBUG | (configresolver.go:124 in ResolveTemplate) | No service found with this AD identifier: couchdb
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:317 in resolve) | Can't resolve the template for couch at this moment.
2018-07-25 16:52:35 UTC | DEBUG | (configresolver.go:124 in ResolveTemplate) | No service found with this AD identifier: couchbase
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:317 in resolve) | Can't resolve the template for couchbase at this moment.
2018-07-25 16:52:35 UTC | DEBUG | (configresolver.go:124 in ResolveTemplate) | No service found with this AD identifier: elasticsearch
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:317 in resolve) | Can't resolve the template for elastic at this moment.
2018-07-25 16:52:35 UTC | DEBUG | (configresolver.go:124 in ResolveTemplate) | No service found with this AD identifier: etcd
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:317 in resolve) | Can't resolve the template for etcd at this moment.
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (configresolver.go:124 in ResolveTemplate) | No service found with this AD identifier: kubedns-amd64
2018-07-25 16:52:35 UTC | DEBUG | (configresolver.go:124 in ResolveTemplate) | No service found with this AD identifier: k8s-dns-kube-dns-amd64
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:317 in resolve) | Can't resolve the template for kube_dns at this moment.
2018-07-25 16:52:35 UTC | DEBUG | (configresolver.go:124 in ResolveTemplate) | No service found with this AD identifier: kube-proxy
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:317 in resolve) | Can't resolve the template for kube_proxy at this moment.
2018-07-25 16:52:35 UTC | DEBUG | (configresolver.go:124 in ResolveTemplate) | No service found with this AD identifier: kube-state-metrics
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:317 in resolve) | Can't resolve the template for kubernetes_state at this moment.
2018-07-25 16:52:35 UTC | DEBUG | (configresolver.go:124 in ResolveTemplate) | No service found with this AD identifier: kyototycoon
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:317 in resolve) | Can't resolve the template for kyototycoon at this moment.
2018-07-25 16:52:35 UTC | DEBUG | (configresolver.go:124 in ResolveTemplate) | No service found with this AD identifier: memcached
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:317 in resolve) | Can't resolve the template for mcache at this moment.
2018-07-25 16:52:35 UTC | DEBUG | (configresolver.go:124 in ResolveTemplate) | No service found with this AD identifier: redis
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:317 in resolve) | Can't resolve the template for redisdb at this moment.
2018-07-25 16:52:35 UTC | DEBUG | (configresolver.go:124 in ResolveTemplate) | No service found with this AD identifier: riak
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:317 in resolve) | Can't resolve the template for riak at this moment.
2018-07-25 16:52:35 UTC | DEBUG | (configresolver.go:124 in ResolveTemplate) | No service found with this AD identifier: tomcat
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:317 in resolve) | Can't resolve the template for tomcat at this moment.
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (secrets.go:122 in Decrypt) | No data to decrypt or no secretBackendCommand set: skipping
2018-07-25 16:52:35 UTC | DEBUG | (loader.go:89 in Load) | Unable to load python module - datadog_checks.jmx: No module named jmx
2018-07-25 16:52:35 UTC | DEBUG | (loader.go:89 in Load) | Unable to load python module - jmx: No module named jmx
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:567 in getChecks) | Python Check Loader: unable to load the check 'jmx': No module named jmx
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:567 in getChecks) | Core Check Loader: unable to load the check 'jmx': check jmx appears to be a JMX check - skipping
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:556 in getChecks) | JMX Check Loader: successfully loaded check 'jmx'
2018-07-25 16:52:35 UTC | DEBUG | (loader.go:89 in Load) | Unable to load python module - datadog_checks.kafka: No module named kafka
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:567 in getChecks) | Python Check Loader: unable to load the check 'kafka': Unable to find a check class in the module: cannot find a subclass of <class 'datadog_checks.checks.base.AgentCheck'> in module <module 'kafka' from '/opt/datadog-agent/embedded/lib/python2.7/site-packages/kafka/__init__.pyc'>
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:567 in getChecks) | Core Check Loader: unable to load the check 'kafka': check kafka appears to be a JMX check - skipping
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:556 in getChecks) | JMX Check Loader: successfully loaded check 'kafka'
2018-07-25 16:52:35 UTC | WARN | (check.go:244 in Configure) | could not get a check instance with the new api: __init__() takes at least 4 arguments (4 given)
2018-07-25 16:52:35 UTC | WARN | (check.go:245 in Configure) | trying to instantiate the check with the old api, passing agentConfig to the constructor
2018-07-25 16:52:35 UTC | WARN | (check.go:270 in Configure) | passing `agentConfig` to the constructor is deprecated, please use the `get_config` function from the 'datadog_agent' package (kafka_consumer).
2018-07-25 16:52:35 UTC | DEBUG | (check.go:272 in Configure) | python check configure done kafka_consumer
2018-07-25 16:52:35 UTC | DEBUG | (loader.go:121 in Load) | python loader: done loading check kafka_consumer
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:556 in getChecks) | Python Check Loader: successfully loaded check 'kafka_consumer'
2018-07-25 16:52:35 UTC | DEBUG | (loader.go:89 in Load) | Unable to load python module - datadog_checks.cpu: No module named cpu
2018-07-25 16:52:35 UTC | DEBUG | (loader.go:89 in Load) | Unable to load python module - cpu: No module named cpu
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:567 in getChecks) | Python Check Loader: unable to load the check 'cpu': No module named cpu
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:556 in getChecks) | Core Check Loader: successfully loaded check 'cpu'
2018-07-25 16:52:35 UTC | WARN | (check.go:244 in Configure) | could not get a check instance with the new api: __init__() takes at least 4 arguments (4 given)
2018-07-25 16:52:35 UTC | WARN | (check.go:245 in Configure) | trying to instantiate the check with the old api, passing agentConfig to the constructor
2018-07-25 16:52:35 UTC | WARN | (check.go:270 in Configure) | passing `agentConfig` to the constructor is deprecated, please use the `get_config` function from the 'datadog_agent' package (disk).
2018-07-25 16:52:35 UTC | DEBUG | (check.go:272 in Configure) | python check configure done disk
2018-07-25 16:52:35 UTC | DEBUG | (loader.go:121 in Load) | python loader: done loading check disk
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:556 in getChecks) | Python Check Loader: successfully loaded check 'disk'
2018-07-25 16:52:35 UTC | DEBUG | (loader.go:89 in Load) | Unable to load python module - datadog_checks.file_handle: No module named file_handle
2018-07-25 16:52:35 UTC | DEBUG | (loader.go:89 in Load) | Unable to load python module - file_handle: No module named file_handle
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:567 in getChecks) | Python Check Loader: unable to load the check 'file_handle': No module named file_handle
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:556 in getChecks) | Core Check Loader: successfully loaded check 'file_handle'
2018-07-25 16:52:35 UTC | DEBUG | (loader.go:89 in Load) | Unable to load python module - datadog_checks.io: No module named io
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:567 in getChecks) | Python Check Loader: unable to load the check 'io': Unable to find a check class in the module: cannot find a subclass of <class 'datadog_checks.checks.base.AgentCheck'> in module <module 'io' from '/opt/datadog-agent/embedded/lib/python2.7/io.pyc'>
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:556 in getChecks) | Core Check Loader: successfully loaded check 'io'
2018-07-25 16:52:35 UTC | DEBUG | (loader.go:89 in Load) | Unable to load python module - datadog_checks.load: No module named load
2018-07-25 16:52:35 UTC | DEBUG | (loader.go:89 in Load) | Unable to load python module - load: No module named load
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:567 in getChecks) | Python Check Loader: unable to load the check 'load': No module named load
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:556 in getChecks) | Core Check Loader: successfully loaded check 'load'
2018-07-25 16:52:35 UTC | DEBUG | (loader.go:89 in Load) | Unable to load python module - datadog_checks.memory: No module named memory
2018-07-25 16:52:35 UTC | DEBUG | (loader.go:89 in Load) | Unable to load python module - memory: No module named memory
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:567 in getChecks) | Python Check Loader: unable to load the check 'memory': No module named memory
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:556 in getChecks) | Core Check Loader: successfully loaded check 'memory'
2018-07-25 16:52:35 UTC | WARN | (check.go:244 in Configure) | could not get a check instance with the new api: __init__() takes at least 4 arguments (4 given)
2018-07-25 16:52:35 UTC | WARN | (check.go:245 in Configure) | trying to instantiate the check with the old api, passing agentConfig to the constructor
2018-07-25 16:52:35 UTC | WARN | (check.go:270 in Configure) | passing `agentConfig` to the constructor is deprecated, please use the `get_config` function from the 'datadog_agent' package (network).
2018-07-25 16:52:35 UTC | DEBUG | (check.go:272 in Configure) | python check configure done network
2018-07-25 16:52:35 UTC | DEBUG | (loader.go:121 in Load) | python loader: done loading check network
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:556 in getChecks) | Python Check Loader: successfully loaded check 'network'
2018-07-25 16:52:35 UTC | DEBUG | (loader.go:89 in Load) | Unable to load python module - datadog_checks.ntp: No module named ntp
2018-07-25 16:52:35 UTC | DEBUG | (loader.go:89 in Load) | Unable to load python module - ntp: No module named ntp
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:567 in getChecks) | Python Check Loader: unable to load the check 'ntp': No module named ntp
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:556 in getChecks) | Core Check Loader: successfully loaded check 'ntp'
2018-07-25 16:52:35 UTC | DEBUG | (loader.go:89 in Load) | Unable to load python module - datadog_checks.uptime: No module named uptime
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:567 in getChecks) | Python Check Loader: unable to load the check 'uptime': Unable to find a check class in the module: cannot find a subclass of <class 'datadog_checks.checks.base.AgentCheck'> in module <module 'uptime' from '/opt/datadog-agent/embedded/lib/python2.7/site-packages/uptime/__init__.pyc'>
2018-07-25 16:52:35 UTC | DEBUG | (autoconfig.go:556 in getChecks) | Core Check Loader: successfully loaded check 'uptime'
2018-07-25 16:52:35 UTC | DEBUG | (jmxfetch.go:224 in Start) | Args: [-Xmx200m -Xms50m -classpath /opt/datadog-agent/bin/agent/dist/jmx/jmxfetch-0.20.1-jar-with-dependencies.jar org.datadog.jmxfetch.App --ipc_host localhost --ipc_port 5001 --check_period 15000 --log_level INFO --reporter statsd:localhost:8125 collect]
2018-07-25 16:52:35 UTC | INFO | (jmxfetch.go:208 in func1) | Log location is not set, will output log to stdout.
2018-07-25 16:52:35 UTC | INFO | (jmxfetch.go:208 in func1) | 2018-07-25 16:52:35,904 | INFO | App | JMX Fetch has started
2018-07-25 16:52:36 UTC | INFO | (jmxfetch.go:208 in func1) | 2018-07-25 16:52:36,583 | WARN | App | Failed collecting JSON configs: [403]
2018-07-25 16:52:36 UTC | INFO | (jmxfetch.go:208 in func1) | 2018-07-25 16:52:36,583 | WARN | App | No instance could be initiated. Retrying initialization.
2018-07-25 16:52:51 UTC | INFO | (jmxfetch.go:208 in func1) | 2018-07-25 16:52:51,691 | WARN | App | Failed collecting JSON configs: [403]
2018-07-25 16:52:51 UTC | INFO | (jmxfetch.go:208 in func1) | 2018-07-25 16:52:51,691 | WARN | App | No instance could be initiated. Retrying initialization.

Those last 2 lines repeat forever until I Ctrl-C the check.

I have confirmed that kafka is running and JMX is available on port 9999, by creating a tunnel to my local machine and connecting with Jconsole.

Describe what you expected:

I expect the check to work, show up in the sudo datadog-agent status results, and ship to datadog.

Additional environment details (Operating System, Cloud provider, etc):

Running on amazon linux 2018.03. Java 1.8.0

Thanks for the help!

epinzur commented 5 years ago

Note that I also get similar results with the jmx check, with config set as: 

init_config:

instances:
  - host: localhost
    port: 9999

I release this isn't set to collect any metrics, but I wanted to see if it would at least connect.

tommynsong commented 5 years ago

+1