Cacti / cacti

Cacti ™
http://www.cacti.net
GNU General Public License v2.0
1.64k stars 405 forks source link

Poller warning due to SNMP timeout connecting to localhost, with no corresponding SNMP device set up #296

Closed mhoran closed 7 years ago

mhoran commented 7 years ago

Since upgrading to Cacti 1.0.1, I've seen the following message in my logs:

2017-02-12 18:20:02 - CMDPHP WARNING: SNMP Error:'Timeout (500 ms)\', Device:'udp6:[::1]', OID:'.1.3.6.1.2.1.1.3.0'

I do not have any devices set up to poll SNMP on localhost. I do, however, have a "Local Linux Machine" device set up. This device is set up with SNMP Version "Not In Use", yet when I disable this device the warning is no longer printed.

It's interesting to note the following:

2017-02-12 18:20:09 - SYSTEM STATS: Time:9.4668 Method:cmd.php Processes:1 Threads:N/A Hosts:5 HostsPerProcess:5 DataSources:56 RRDsProcessed:37

Which shows that there are 5 hosts being scanned. However, my device list shows only 4. When I disable the "Local Linux Machine" device, the hosts count drops to 4.

cigamit commented 7 years ago

That's quite sneaky. We have not included ipv6 localhost detection into Cacti based upon that ipv6 address. It's a minor oversight. For that reason, I'll leave this ticket open.

With regard to the device count being off by one, that is due to some legacy behavior with a special device of id 0. There are some people out there who create data sources for non-existent devices (aka device_id=0).

The reason that you are getting this warning is that you are likely using some data source that leverages snmp.

mhoran commented 7 years ago

Thanks @cigamit. When I comment out the ::1 entry in /etc/hosts, I now get the following in my log:

2017-02-12 19:30:02 - CMDPHP WARNING: SNMP Error:'Timeout (500 ms)\', Device:'127.0.0.1', OID:'.1.3.6.1.2.1.1.3.0'

... which I guess is to be expected.

I've double checked my data sources for localhost, and none of them look to be leveraging SNMP. They've all got "Unix" in their name, which I believe means they're simply local scripts.

I'm not sure what could have happened, because I've never seen this warning before in years of running Cacti. Any additional help would be much appreciated!

cigamit commented 7 years ago

What is the availability method set to? The only path to expose this would be with an availability version set to something of the 'snmp' type. Was the device created programmatically using add_device or some other method?

mhoran commented 7 years ago

The availability method is set to "Ping". The device was created via the UI in 2015.

mhoran commented 7 years ago

I disabled downed device detection and the issue persists.

cigamit commented 7 years ago

Please, enable the device, and then goto the Utilities -> View Poller Cache, filter on this device, and show the entries.

Also, from the MySQL command prompt, execute the following query:

select hostname, snmp_version, availability_method, ping_method from host where id = ?

mhoran commented 7 years ago

Here's what I see in the poller cache:

localhost - Free Space - /dev/ad0s1a    Script: perl /usr/local/share/cacti/scripts/query_unix_partitions.pl get used '/dev/ad0s1a'
RRD: /var/db/cacti/rra/localhost_hdd_free_98.rrd
localhost - Free Space - /dev/ad0s1a    Script: perl /usr/local/share/cacti/scripts/query_unix_partitions.pl get available '/dev/ad0s1a'
RRD: /var/db/cacti/rra/localhost_hdd_free_98.rrd
localhost - Free Space - /dev/ad0s1d    Script: perl /usr/local/share/cacti/scripts/query_unix_partitions.pl get used '/dev/ad0s1d'
RRD: /var/db/cacti/rra/localhost_hdd_free_99.rrd
localhost - Free Space - /dev/ad0s1d    Script: perl /usr/local/share/cacti/scripts/query_unix_partitions.pl get available '/dev/ad0s1d'
RRD: /var/db/cacti/rra/localhost_hdd_free_99.rrd
localhost - Free Space - /dev/ad0s1e    Script: perl /usr/local/share/cacti/scripts/query_unix_partitions.pl get available '/dev/ad0s1e'
RRD: /var/db/cacti/rra/localhost_hdd_free_100.rrd
localhost - Free Space - /dev/ad0s1e    Script: perl /usr/local/share/cacti/scripts/query_unix_partitions.pl get used '/dev/ad0s1e'
RRD: /var/db/cacti/rra/localhost_hdd_free_100.rrd
localhost - Free Space - /dev/ad0s1f    Script: perl /usr/local/share/cacti/scripts/query_unix_partitions.pl get available '/dev/ad0s1f'
RRD: /var/db/cacti/rra/localhost_hdd_free_101.rrd
localhost - Free Space - /dev/ad0s1f    Script: perl /usr/local/share/cacti/scripts/query_unix_partitions.pl get used '/dev/ad0s1f'
RRD: /var/db/cacti/rra/localhost_hdd_free_101.rrd
localhost - Load Average    Script: perl /usr/local/share/cacti/scripts/loadavg_multi.pl
RRD: /var/db/cacti/rra/localhost_load_1min_95.rrd
localhost - Logged in Users Script: perl /usr/local/share/cacti/scripts/unix_users.pl ''
RRD: /var/db/cacti/rra/localhost_users_96.rrd
localhost - Processes   Script: perl /usr/local/share/cacti/scripts/unix_processes.pl
RRD: /var/db/cacti/rra/localhost_proc_97.rrd

... and here's the result of the SQL query:

mysql> select hostname, snmp_version, availability_method, ping_method from host where
    -> id = 9;
+-----------+--------------+---------------------+-------------+
| hostname  | snmp_version | availability_method | ping_method |
+-----------+--------------+---------------------+-------------+
| localhost |            0 |                   0 |           1 |
+-----------+--------------+---------------------+-------------+
1 row in set (0.00 sec)
cigamit commented 7 years ago

It weird, I did the same setup and could not reproduce.

pautiina commented 7 years ago

If you use IPv6 host you must activate this function on net-snmp daemon: add line to yo snmpd.conf file: rocommunity6 public Maybe it's help you.

cigamit commented 7 years ago

The issue is device is being checked for SNMP when it's not even an SNMP device. It could be due to the wrong re-index method I'm guessing. Hmm. That might be it. For the Linux Partitions, the re-index method should be 'none'. Oh, damn, that is the issue ;) I'll keep this open for a while while I figure our the right course of action. To resolve the issue, drop the Data Query, and re-add using the 'None' reindex method.

mhoran commented 7 years ago

Yup, that did it. I had tried changing the method in the database, but that didn't take. Deleting and re-adding worked fine.

cigamit commented 7 years ago

Resolved, thanks for reporting!