search

DataDog / datadog-agent

Main repository for Datadog Agent
https://docs.datadoghq.com/
Apache License 2.0
2.88k stars 1.21k forks source link

No core integrations found #1844

Open mheppner opened 6 years ago

mheppner commented 6 years ago

After compiling and running the agent on FreeBSD, I can't seem to figure out how to get core integrations to work. I understand that FreeBSD is mostly unsupported, so feel free to close this.

I'm pretty sure this is a similar issue to #1651.

Output of the agent status

=========
Collector
=========

  Running Checks
  ==============
    cpu
    ---
      Total Runs: 38
      Metrics: 6, Total Metrics: 222
      Events: 0, Total Events: 0
      Service Checks: 0, Total Service Checks: 0
      Average Execution Time : 0ms

    file_handle
    -----------
      Total Runs: 38
      Metrics: 0, Total Metrics: 0
      Events: 0, Total Events: 0
      Service Checks: 0, Total Service Checks: 0
      Average Execution Time : 0ms
      Error: open /proc/sys/fs/file-nr: no such file or directory
      No traceback

    io
    --
      Total Runs: 38
      Metrics: 156, Total Metrics: 5820
      Events: 0, Total Events: 0
      Service Checks: 0, Total Service Checks: 0
      Average Execution Time : 0ms

    load
    ----
      Total Runs: 38
      Metrics: 6, Total Metrics: 228
      Events: 0, Total Events: 0
      Service Checks: 0, Total Service Checks: 0
      Average Execution Time : 0ms

    memory
    ------
      Total Runs: 38
      Metrics: 10, Total Metrics: 380
      Events: 0, Total Events: 0
      Service Checks: 0, Total Service Checks: 0
      Average Execution Time : 0ms

    ntp
    ---
      Total Runs: 38
      Metrics: 1, Total Metrics: 38
      Events: 0, Total Events: 0
      Service Checks: 1, Total Service Checks: 38
      Average Execution Time : 225ms

    uptime
    ------
      Total Runs: 38
      Metrics: 1, Total Metrics: 38
      Events: 0, Total Events: 0
      Service Checks: 0, Total Service Checks: 0
      Average Execution Time : 0ms

  Loading Errors
  ==============
    apm
    ---
      Core Check Loader:
        Check apm not found in Catalog

      Python Check Loader:
        No module named apm

    elastic
    -------
      Core Check Loader:
        Check elastic not found in Catalog

      Python Check Loader:
        No module named elastic

    mcache
    ------
      Core Check Loader:
        Check mcache not found in Catalog

      Python Check Loader:
        No module named mcache

    mysql
    -----
      Core Check Loader:
        Check mysql not found in Catalog

      Python Check Loader:
        No module named mysql

    postgres
    --------
      Core Check Loader:
        Check postgres not found in Catalog

      Python Check Loader:
        No module named postgres

    rabbitmq
    --------
      Core Check Loader:
        Check rabbitmq not found in Catalog

      Python Check Loader:
        No module named rabbitmq

    redisdb
    -------
      Core Check Loader:
        Check redisdb not found in Catalog

      Python Check Loader:
        No module named redisdb

    winproc
    -------
      Core Check Loader:
        Check winproc not found in Catalog

      Python Check Loader:
        No module named winproc

Putput of the agent DEBUG logs:

2018-06-14 14:09:47 EDT | INFO | (start.go:156 in StartAgent) | Starting Datadog Agent v6.2.1
2018-06-14 14:09:47 EDT | INFO | (start.go:167 in StartAgent) | pid '94940' written to pid file '/home/datadog/run/agent.pid'
2018-06-14 14:09:47 EDT | DEBUG | (hostname.go:108 in GetHostname) | Unable to get the hostname from the config file: host name is empty
2018-06-14 14:09:47 EDT | DEBUG | (hostname.go:109 in GetHostname) | Trying to determine a reliable host name automatically...
2018-06-14 14:09:47 EDT | DEBUG | (hostname.go:118 in GetHostname) | GetHostname trying GCE metadata...
2018-06-14 14:09:47 EDT | DEBUG | (hostname.go:135 in GetHostname) | GetHostname trying os...
2018-06-14 14:09:47 EDT | INFO | (start.go:174 in StartAgent) | Hostname is: server
2018-06-14 14:09:48 EDT | INFO | (start.go:191 in StartAgent) | GUI server port -1 specified: not starting the GUI.
2018-06-14 14:09:48 EDT | DEBUG | (start.go:202 in StartAgent) | Starting forwarder
2018-06-14 14:09:48 EDT | INFO | (forwarder.go:235 in Start) | DefaultForwarder started (1 workers), sending to 1 endpoint(s): "https://6-2-1-app.agent.datadoghq.com" (1 api key(s))
2018-06-14 14:09:48 EDT | DEBUG | (start.go:204 in StartAgent) | Forwarder started
2018-06-14 14:09:48 EDT | DEBUG | (forwarder.go:313 in healthCheckLoop) | Waiting for APIkey validity to be confirmed.
2018-06-14 14:09:48 EDT | DEBUG | (udp.go:57 in NewUDPListener) | dogstatsd-udp: 127.0.0.1:8125 successfully initialized
2018-06-14 14:09:48 EDT | DEBUG | (start.go:219 in StartAgent) | statsd started
2018-06-14 14:09:48 EDT | INFO | (start.go:231 in StartAgent) | logs-agent disabled
2018-06-14 14:09:48 EDT | INFO | (tagger.go:78 in Init) | starting the tagging system
2018-06-14 14:09:48 EDT | DEBUG | (tagger.go:126 in startCollectors) | candidate list empty, stopping detection
2018-06-14 14:09:48 EDT | INFO | (runner.go:92 in NewRunner) | Runner started with 1 workers.
2018-06-14 14:09:48 EDT | DEBUG | (scheduler.go:125 in func1) | Starting scheduler loop...
2018-06-14 14:09:48 EDT | DEBUG | (runner.go:226 in work) | Ready to process checks...
2018-06-14 14:09:48 EDT | INFO | (udp.go:63 in Listen) | dogstatsd-udp: starting to listen on 127.0.0.1:8125
2018-06-14 14:09:48 EDT | INFO | (collector.go:51 in NewCollector) | Embedding Python 2.7.15 (default, May 16 2018, 09:49:45) [GCC 4.2.1 Compatible FreeBSD Clang 4.0.0 (tags/RELEASE_400/final 297347)]
2018-06-14 14:09:48 EDT | DEBUG | (collector.go:52 in NewCollector) | Python Home:
2018-06-14 14:09:48 EDT | DEBUG | (collector.go:53 in NewCollector) | Python path: ['/usr/local/lib/python27.zip', '/usr/local/lib/python2.7', '/usr/local/lib/python2.7/plat-freebsd11', '/usr/local/lib/python2.7
/lib-tk', '/usr/local/lib/python2.7/lib-old', '/usr/local/lib/python2.7/lib-dynload', '/usr/local/lib/python2.7/site-packages', '/usr/home/datadog/go/src/github.com/DataDog/datadog-agent/bin/agent/dist', '/usr/h
ome/datadog/go/src/github.com/DataDog/datadog-agent/checks.d', '/usr/home/datadog/go/src/github.com/DataDog/datadog-agent/bin/agent/dist/checks.d', '/etc/datadog-agent/checks.d']
2018-06-14 14:09:48 EDT | DEBUG | (collector.go:56 in NewCollector) | Collector up and running!
2018-06-14 14:09:48 EDT | DEBUG | (autoconfig.go:41 in SetupAutoConfig) | Added Python Check Loader to AutoConfig
2018-06-14 14:09:48 EDT | DEBUG | (autoconfig.go:41 in SetupAutoConfig) | Added Core Check Loader to AutoConfig
2018-06-14 14:09:48 EDT | INFO | (file.go:69 in Collect) | File Configuration Provider: searching for configuration files at: /home/datadog/config/conf.d
2018-06-14 14:09:48 EDT | DEBUG | (file.go:193 in collectEntry) | Found valid configuration in file:/home/datadog/config/conf.d/apm.yaml.default
2018-06-14 14:09:48 EDT | DEBUG | (file.go:193 in collectEntry) | Found valid configuration in file:/home/datadog/config/conf.d/cpu.d/conf.yaml.default
2018-06-14 14:09:48 EDT | DEBUG | (file.go:193 in collectEntry) | Found valid configuration in file:/home/datadog/config/conf.d/elastic.d/conf.yaml
2018-06-14 14:09:48 EDT | DEBUG | (file.go:193 in collectEntry) | Found valid configuration in file:/home/datadog/config/conf.d/file_handle.d/conf.yaml.default
2018-06-14 14:09:48 EDT | DEBUG | (file.go:193 in collectEntry) | Found valid configuration in file:/home/datadog/config/conf.d/io.d/conf.yaml.default
2018-06-14 14:09:48 EDT | DEBUG | (file.go:193 in collectEntry) | Found valid configuration in file:/home/datadog/config/conf.d/load.d/conf.yaml.default
2018-06-14 14:09:48 EDT | DEBUG | (file.go:193 in collectEntry) | Found valid configuration in file:/home/datadog/config/conf.d/mcache.yaml
2018-06-14 14:09:48 EDT | DEBUG | (file.go:193 in collectEntry) | Found valid configuration in file:/home/datadog/config/conf.d/memory.d/conf.yaml.default
2018-06-14 14:09:48 EDT | DEBUG | (file.go:193 in collectEntry) | Found valid configuration in file:/home/datadog/config/conf.d/mysql.yaml
2018-06-14 14:09:48 EDT | DEBUG | (file.go:193 in collectEntry) | Found valid configuration in file:/home/datadog/config/conf.d/ntp.d/conf.yaml.default
2018-06-14 14:09:48 EDT | DEBUG | (file.go:193 in collectEntry) | Found valid configuration in file:/home/datadog/config/conf.d/postgres.yaml
2018-06-14 14:09:48 EDT | DEBUG | (file.go:193 in collectEntry) | Found valid configuration in file:/home/datadog/config/conf.d/rabbitmq.yaml
2018-06-14 14:09:48 EDT | DEBUG | (file.go:193 in collectEntry) | Found valid configuration in file:/home/datadog/config/conf.d/redisdb.yaml
2018-06-14 14:09:48 EDT | DEBUG | (file.go:193 in collectEntry) | Found valid configuration in file:/home/datadog/config/conf.d/uptime.d/conf.yaml.default
2018-06-14 14:09:48 EDT | DEBUG | (file.go:193 in collectEntry) | Found valid configuration in file:/home/datadog/config/conf.d/winproc.d/conf.yaml.default
2018-06-14 14:09:48 EDT | INFO | (file.go:69 in Collect) | File Configuration Provider: searching for configuration files at: /usr/home/datadog/go/src/github.com/DataDog/datadog-agent/bin/agent/dist/conf.d
2018-06-14 14:09:48 EDT | DEBUG | (file.go:193 in collectEntry) | Found valid configuration in file:/usr/home/datadog/go/src/github.com/DataDog/datadog-agent/bin/agent/dist/conf.d/apm.yaml.default
2018-06-14 14:09:48 EDT | DEBUG | (file.go:193 in collectEntry) | Found valid configuration in file:/usr/home/datadog/go/src/github.com/DataDog/datadog-agent/bin/agent/dist/conf.d/cpu.d/conf.yaml.default
2018-06-14 14:09:48 EDT | DEBUG | (file.go:193 in collectEntry) | Found valid configuration in file:/usr/home/datadog/go/src/github.com/DataDog/datadog-agent/bin/agent/dist/conf.d/file_handle.d/conf.yaml.default
2018-06-14 14:09:48 EDT | DEBUG | (file.go:193 in collectEntry) | Found valid configuration in file:/usr/home/datadog/go/src/github.com/DataDog/datadog-agent/bin/agent/dist/conf.d/io.d/conf.yaml.default
2018-06-14 14:09:48 EDT | DEBUG | (file.go:193 in collectEntry) | Found valid configuration in file:/usr/home/datadog/go/src/github.com/DataDog/datadog-agent/bin/agent/dist/conf.d/load.d/conf.yaml.default
2018-06-14 14:09:48 EDT | DEBUG | (file.go:193 in collectEntry) | Found valid configuration in file:/usr/home/datadog/go/src/github.com/DataDog/datadog-agent/bin/agent/dist/conf.d/memory.d/conf.yaml.default
2018-06-14 14:09:48 EDT | DEBUG | (file.go:193 in collectEntry) | Found valid configuration in file:/usr/home/datadog/go/src/github.com/DataDog/datadog-agent/bin/agent/dist/conf.d/ntp.d/conf.yaml.default
2018-06-14 14:09:48 EDT | DEBUG | (file.go:193 in collectEntry) | Found valid configuration in file:/usr/home/datadog/go/src/github.com/DataDog/datadog-agent/bin/agent/dist/conf.d/uptime.d/conf.yaml.default
2018-06-14 14:09:48 EDT | DEBUG | (file.go:193 in collectEntry) | Found valid configuration in file:/usr/home/datadog/go/src/github.com/DataDog/datadog-agent/bin/agent/dist/conf.d/winproc.d/conf.yaml.default
2018-06-14 14:09:48 EDT | DEBUG | (loader.go:88 in Load) | Unable to load python module - datadog_checks.elastic: No module named datadog_checks.elastic
2018-06-14 14:09:48 EDT | DEBUG | (loader.go:88 in Load) | Unable to load python module - elastic: No module named elastic
2018-06-14 14:09:48 EDT | DEBUG | (autoconfig.go:491 in getChecks) | Python Check Loader: unable to load the check 'elastic': No module named elastic
2018-06-14 14:09:48 EDT | DEBUG | (autoconfig.go:491 in getChecks) | Core Check Loader: unable to load the check 'elastic': Check elastic not found in Catalog
2018-06-14 14:09:48 EDT | ERROR | (autoconfig.go:249 in getChecksFromConfigs) | Unable to load the check: unable to load any check from config 'elastic'
2018-06-14 14:09:48 EDT | DEBUG | (loader.go:88 in Load) | Unable to load python module - datadog_checks.mcache: No module named datadog_checks.mcache
2018-06-14 14:09:48 EDT | DEBUG | (loader.go:88 in Load) | Unable to load python module - mcache: No module named mcache
2018-06-14 14:09:48 EDT | DEBUG | (autoconfig.go:491 in getChecks) | Python Check Loader: unable to load the check 'mcache': No module named mcache
2018-06-14 14:09:48 EDT | DEBUG | (autoconfig.go:491 in getChecks) | Core Check Loader: unable to load the check 'mcache': Check mcache not found in Catalog
2018-06-14 14:09:48 EDT | ERROR | (autoconfig.go:249 in getChecksFromConfigs) | Unable to load the check: unable to load any check from config 'mcache'
2018-06-14 14:09:48 EDT | DEBUG | (loader.go:88 in Load) | Unable to load python module - datadog_checks.mysql: No module named datadog_checks.mysql
2018-06-14 14:09:48 EDT | DEBUG | (loader.go:88 in Load) | Unable to load python module - mysql: No module named mysql
2018-06-14 14:09:48 EDT | DEBUG | (autoconfig.go:491 in getChecks) | Python Check Loader: unable to load the check 'mysql': No module named mysql
2018-06-14 14:09:48 EDT | DEBUG | (autoconfig.go:491 in getChecks) | Core Check Loader: unable to load the check 'mysql': Check mysql not found in Catalog
2018-06-14 14:09:48 EDT | ERROR | (autoconfig.go:249 in getChecksFromConfigs) | Unable to load the check: unable to load any check from config 'mysql'
2018-06-14 14:09:48 EDT | DEBUG | (loader.go:88 in Load) | Unable to load python module - datadog_checks.postgres: No module named datadog_checks.postgres
2018-06-14 14:09:48 EDT | DEBUG | (loader.go:88 in Load) | Unable to load python module - postgres: No module named postgres
2018-06-14 14:09:48 EDT | DEBUG | (autoconfig.go:491 in getChecks) | Python Check Loader: unable to load the check 'postgres': No module named postgres
2018-06-14 14:09:48 EDT | DEBUG | (autoconfig.go:491 in getChecks) | Core Check Loader: unable to load the check 'postgres': Check postgres not found in Catalog
2018-06-14 14:09:48 EDT | ERROR | (autoconfig.go:249 in getChecksFromConfigs) | Unable to load the check: unable to load any check from config 'postgres'

...and so forth.

Describe what you expected: Are core integrations not available in agent v6? Or just on FreeBSD? Or did I miss an installation step?

Steps to reproduce the issue: Built the agent from readme instructions:

git clone https://github.com/DataDog/datadog-agent.git $GOPATH/src/github.com/DataDog/datadog-agent
cd $GOPATH/src/github.com/DataDog/datadog-agent
git checkout 6.2.1
invoke deps
invoke agent.build --build-exclude=snmp

Setup the config directory (and edited datadog.yaml):

cp -R $GOPATH/src/github.com/DataDog/datadog-agent/bin/agent/dist ~/config
# added ~/config/conf.d/elastic.d/conf.yaml, etc

Started the agent:

agent start --cfgpath ~/config

Additional environment details (Operating System, Cloud provider, etc):

amontalban commented 5 years ago

I'm having the exact same problem, tried to configure NGiNX and Redis integrations and looks like they have to be installed, however, I installed it using different options:

https://gist.github.com/amontalban/06fc1f679de5fe5f831dcf243ef63fae

I would love to have a simple command that builds everything and allows you to define an output folder with all the hierarchy there so you can just throw wherever you want.

urosgruber commented 3 years ago

I was working on a FreeBSD port and it's finally here. Yesterday was committed to ports tree. Port for integrations will follow in the next days.

therzv commented 3 years ago

is this already solve?