DataDog / datadog-agent

Main repository for Datadog Agent
https://docs.datadoghq.com/
Apache License 2.0
2.9k stars 1.21k forks source link

JMX plugin defaults to null host #4651

Open jkataja opened 4 years ago

jkataja commented 4 years ago

Output of the info page (if this is a bug)

● datadog-agent.service - Datadog Agent
   Loaded: loaded (/usr/lib/systemd/system/datadog-agent.service; enabled; vendor preset: disabled)
   Active: active (running) since Fri 2019-12-20 15:22:18 UTC; 7min ago
 Main PID: 4224 (agent)
   CGroup: /system.slice/datadog-agent.service
           ├─4224 /opt/datadog-agent/bin/agent/agent run -p /opt/datadog-agent/run/agent.pid
           └─4300 java -Xmx200m -Xms50m -classpath /opt/datadog-agent/bin/agent/dist/jmx/jmxfetch.jar org.datadog.jmxfetch.App --ipc_host localhost --ipc_port 5051 --check_period 15000 --thread_pool_size 3 --collection_timeout 60 --reconnection_timeout 10 --reconnectio...
Dec 20 15:29:06 ip-172-18-2-142.eu-west-1.compute.internal agent[4224]: 2019-12-20 15:29:06 UTC | CORE | INFO | (pkg/jmxfetch/jmxfetch.go:249 in func1) | 2019-12-20 15:29:06,127 | INFO | App | Started instance initialization...
Dec 20 15:29:06 ip-172-18-2-142.eu-west-1.compute.internal agent[4224]: 2019-12-20 15:29:06 UTC | CORE | INFO | (pkg/jmxfetch/jmxfetch.go:249 in func1) | 2019-12-20 15:29:06,127 | INFO | Instance | Trying to connect to JMX Server at null:7199
Dec 20 15:29:06 ip-172-18-2-142.eu-west-1.compute.internal agent[4224]: 2019-12-20 15:29:06 UTC | CORE | INFO | (pkg/jmxfetch/jmxfetch.go:249 in func1) | 2019-12-20 15:29:06,127 | INFO | Instance | Connection closed or does not exist. Attempting to cre...new connection...
Dec 20 15:29:06 ip-172-18-2-142.eu-west-1.compute.internal agent[4224]: 2019-12-20 15:29:06 UTC | CORE | INFO | (pkg/jmxfetch/jmxfetch.go:249 in func1) | 2019-12-20 15:29:06,127 | INFO | ConnectionFactory | Connecting using JMX Remote
Dec 20 15:29:06 ip-172-18-2-142.eu-west-1.compute.internal agent[4224]: 2019-12-20 15:29:06 UTC | CORE | INFO | (pkg/jmxfetch/jmxfetch.go:249 in func1) | 2019-12-20 15:29:06,127 | INFO | Connection | Connecting to: service:jmx:rmi:///jndi/rmi://null:7199/jmxrmi
Dec 20 15:29:06 ip-172-18-2-142.eu-west-1.compute.internal agent[4224]: 2019-12-20 15:29:06 UTC | CORE | INFO | (pkg/jmxfetch/jmxfetch.go:249 in func1) | 2019-12-20 15:29:06,128 | INFO | App | Completed instance initialization...
Dec 20 15:29:06 ip-172-18-2-142.eu-west-1.compute.internal agent[4224]: 2019-12-20 15:29:06 UTC | CORE | INFO | (pkg/jmxfetch/jmxfetch.go:249 in func1) | 2019-12-20 15:29:06,128 | INFO | App | Could not initialize instance: jmx: java.util.concurrent.ExecutionException:...
Dec 20 15:29:06 ip-172-18-2-142.eu-west-1.compute.internal agent[4224]: 2019-12-20 15:29:06 UTC | CORE | INFO | (pkg/jmxfetch/jmxfetch.go:249 in func1) |         java.net.UnknownHostException: null]
Dec 20 15:29:06 ip-172-18-2-142.eu-west-1.compute.internal agent[4224]: 2019-12-20 15:29:06 UTC | CORE | INFO | (pkg/jmxfetch/jmxfetch.go:249 in func1) | 2019-12-20 15:29:06,135 | WARN | App | Unable to instantiate or initialize instance null:7199. Is the target JMX Se...
Dec 20 15:29:06 ip-172-18-2-142.eu-west-1.compute.internal agent[4224]: 2019-12-20 15:29:06 UTC | CORE | INFO | (pkg/jmxfetch/jmxfetch.go:249 in func1) |         java.net.UnknownHostException: null]

Describe what happened: JMX plugin default configuration says that default host is localhost. When this field is left , like in the default configuration, the agent tries to connect to null host.

  -

    ## @param host - string - optional - default: localhost
    ## JMX host to connect to.
    #
    # host: localhost

    ## @param port - integer - required
    ## JMX port to connect to.
    #
    port: 7199

Describe what you expected: Agent is unable to initialize due to connecting to null host.

Steps to reproduce the issue: Copy the default configuration file /etc/datadog-agent/conf.d/jmx.d/conf.yaml.example to /etc/datadog-agent/conf.d/jmx.d/conf.yaml

Additional environment details (Operating System, Cloud provider, etc): Amazon Linux 2 AWS eu-west-1 Agent version 6.16.0 installed from Yum repository

Was tempted to fix this with an alias 127.0.0.1 null in /etc/hosts

steinybot commented 3 years ago

I wasn't paying attention to this and it filled up /var/log/messages and my server ground to a halt.

hvithval commented 3 years ago

any news on this?

djmitche commented 3 years ago

Hi folks -- if one (or more!) of you can contact datadog support regarding this issue, that would help to get this prioritized. Thanks!

tjj225 commented 2 years ago

+1