Cacti / cacti

Cacti ™
http://www.cacti.net
GNU General Public License v2.0
1.61k stars 399 forks source link

NaN in rrd files and no graph #548

Closed mbpakalin closed 7 years ago

mbpakalin commented 7 years ago

Hello, I had a fresh install (this one is fourth one), all graphs is NaN. And i searched a lot and didnt find any solution to this problem.

But interesting problem is when i click real time mode, its shows correct graphics. But when real time mode disabled its not showing anything.

2017-04-10_13-41-31

Some of graphics still gives error, but this is fresh install too. 10-15 minutes later all graphs shows empty.

And this is when i click real time mode, all graphs are working. realtimemode

cigamit commented 7 years ago

What's in your cacti log? Any error_log entries. It's also a good thing to test setting the error_log in php.ini to something like /tmp/php_errors.log, and then restarting apache.

cigamit commented 7 years ago

On the re-install front, though we are released, we keep making progress on a fully stable branch. Thanks for keeping on top of it.

mbpakalin commented 7 years ago

Thanks for your reply, There is not so much in error.log but in cacti log there are some entries

2017-04-10 14:35:01 - POLLER: Poller[1] NOTE: Poller Int: '10', Cron Int: '60', Time Since Last: '300.47', Max Runtime '58', Poller Runs: '6' 2017-04-10 14:35:01 - POLLER: Poller[1] WARNING: Cron is out of sync with the Poller Interval! The Poller Interval is '10' seconds, with a maximum of a '60' second Cron, but 300.5 seconds have passed since the last poll! 2017-04-10 14:35:01 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:35:01 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:35:01 - SYSTEM STATS: Time:0.2266 Method:cmd.php Processes:20 Threads:N/A Hosts:2 HostsPerProcess:1 DataSources:0 RRDsProcessed:0 2017-04-10 14:35:11 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:35:11 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:35:11 - SYSTEM STATS: Time:0.4112 Method:cmd.php Processes:20 Threads:N/A Hosts:2 HostsPerProcess:1 DataSources:0 RRDsProcessed:0 2017-04-10 14:35:21 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:35:21 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:35:21 - SYSTEM STATS: Time:0.2133 Method:cmd.php Processes:20 Threads:N/A Hosts:2 HostsPerProcess:1 DataSources:0 RRDsProcessed:0 2017-04-10 14:35:31 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:35:31 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:35:31 - SYSTEM STATS: Time:0.2122 Method:cmd.php Processes:20 Threads:N/A Hosts:2 HostsPerProcess:1 DataSources:0 RRDsProcessed:0 2017-04-10 14:35:41 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:35:41 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:35:42 - SYSTEM STATS: Time:1.3285 Method:cmd.php Processes:20 Threads:N/A Hosts:2 HostsPerProcess:1 DataSources:0 RRDsProcessed:0 2017-04-10 14:35:51 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:35:51 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:35:51 - SYSTEM STATS: Time:0.3673 Method:cmd.php Processes:20 Threads:N/A Hosts:2 HostsPerProcess:1 DataSources:0 RRDsProcessed:0 2017-04-10 14:40:01 - POLLER: Poller[1] NOTE: Poller Int: '10', Cron Int: '60', Time Since Last: '300.91', Max Runtime '58', Poller Runs: '6' 2017-04-10 14:40:01 - POLLER: Poller[1] WARNING: Cron is out of sync with the Poller Interval! The Poller Interval is '10' seconds, with a maximum of a '60' second Cron, but 300.9 seconds have passed since the last poll! 2017-04-10 14:40:01 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:40:02 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:40:02 - SYSTEM STATS: Time:0.2126 Method:cmd.php Processes:20 Threads:N/A Hosts:2 HostsPerProcess:1 DataSources:0 RRDsProcessed:0 2017-04-10 14:40:12 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:40:12 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:40:12 - SYSTEM STATS: Time:0.2917 Method:cmd.php Processes:20 Threads:N/A Hosts:2 HostsPerProcess:1 DataSources:0 RRDsProcessed:0 2017-04-10 14:40:21 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:40:22 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:40:22 - SYSTEM STATS: Time:0.2116 Method:cmd.php Processes:20 Threads:N/A Hosts:2 HostsPerProcess:1 DataSources:0 RRDsProcessed:0 2017-04-10 14:40:31 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:40:31 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:40:31 - SYSTEM STATS: Time:0.2119 Method:cmd.php Processes:20 Threads:N/A Hosts:2 HostsPerProcess:1 DataSources:0 RRDsProcessed:0 2017-04-10 14:40:41 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:40:41 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:40:42 - SYSTEM STATS: Time:0.3792 Method:cmd.php Processes:20 Threads:N/A Hosts:2 HostsPerProcess:1 DataSources:0 RRDsProcessed:0 2017-04-10 14:40:51 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:40:51 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:40:51 - SYSTEM STATS: Time:0.2136 Method:cmd.php Processes:20 Threads:N/A Hosts:2 HostsPerProcess:1 DataSources:0 RRDsProcessed:0 2017-04-10 14:45:12 - POLLER: Poller[1] Device[4] RECACHE DQ[2] OID: .1.3.6.1.2.1.1.3.0, output: 326699 2017-04-10 14:45:12 - POLLER: Poller[1] Device[4] RECACHE DQ[4] OID: .1.3.6.1.2.1.1.3.0, output: 326699 2017-04-10 14:45:12 - POLLER: Poller[1] Device[4] RECACHE DQ[5] OID: .1.3.6.1.2.1.1.3.0, output: 326699 2017-04-10 14:45:12 - POLLER: Poller[1] Device[4] DS[75] CMD: perl /var/www/html/cacti2/scripts/linux_memory.pl 'SwapFree:', output: 8384508 2017-04-10 14:45:12 - POLLER: Poller[1] Time: 0.0265 s, Poller: , Theads: N/A, Devices: 1 2017-04-10 14:45:12 - SYSTEM STATS: Time:0.2211 Method:cmd.php Processes:20 Threads:N/A Hosts:2 HostsPerProcess:1 DataSources:1 RRDsProcessed:1 2017-04-10 14:45:21 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:45:22 - POLLER: Poller[1] Device[4] STATUS: Device '172.16.136.112' is UP. 2017-04-10 14:45:22 - POLLER: Poller[1] Device[4] RECACHE: Processing 4 items in the auto reindex cache for '172.16.136.112'. 2017-04-10 14:45:22 - POLLER: Poller[1] Device[4] RECACHE DQ[1] OID: .1.3.6.1.2.1.1.3.0, output: 327697 2017-04-10 14:45:22 - POLLER: Poller[1] Device[4] RECACHE DQ[2] OID: .1.3.6.1.2.1.1.3.0, output: 327697 2017-04-10 14:45:22 - POLLER: Poller[1] Device[4] RECACHE DQ[4] OID: .1.3.6.1.2.1.1.3.0, output: 327697 2017-04-10 14:45:22 - POLLER: Poller[1] Device[4] RECACHE DQ[5] OID: .1.3.6.1.2.1.1.3.0, output: 327697 2017-04-10 14:45:22 - POLLER: Poller[1] Device[4] DS[76] CMD: perl /var/www/html/cacti2/scripts/loadavg_multi.pl, output: 1min:0.14 5min:0.08 10min:0.02 2017-04-10 14:45:22 - POLLER: Poller[1] Time: 0.0174 s, Poller: , Theads: N/A, Devices: 1 2017-04-10 14:45:22 - POLLER: Poller[1] Parsed MULTI output field '1min:0.14' [map 1min->load_1min] 2017-04-10 14:45:22 - POLLER: Poller[1] Parsed MULTI output field '5min:0.08' [map 5min->load_5min] 2017-04-10 14:45:22 - POLLER: Poller[1] Parsed MULTI output field '10min:0.02' [map 10min->load_15min] 2017-04-10 14:45:22 - SYSTEM STATS: Time:0.2147 Method:cmd.php Processes:20 Threads:N/A Hosts:2 HostsPerProcess:1 DataSources:1 RRDsProcessed:1 2017-04-10 14:45:31 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:45:32 - POLLER: Poller[1] Device[4] STATUS: Device '172.16.136.112' is UP. 2017-04-10 14:45:32 - POLLER: Poller[1] Device[4] RECACHE: Processing 4 items in the auto reindex cache for '172.16.136.112'. 2017-04-10 14:45:32 - POLLER: Poller[1] Device[4] RECACHE DQ[1] OID: .1.3.6.1.2.1.1.3.0, output: 328706 2017-04-10 14:45:32 - POLLER: Poller[1] Device[4] RECACHE DQ[2] OID: .1.3.6.1.2.1.1.3.0, output: 328707 2017-04-10 14:45:32 - POLLER: Poller[1] Device[4] RECACHE DQ[4] OID: .1.3.6.1.2.1.1.3.0, output: 328707 2017-04-10 14:45:32 - POLLER: Poller[1] Device[4] RECACHE DQ[5] OID: .1.3.6.1.2.1.1.3.0, output: 328707 2017-04-10 14:45:32 - POLLER: Poller[1] Device[4] DS[77] CMD: perl /var/www/html/cacti2/scripts/unix_users.pl '', output: 2 2017-04-10 14:45:32 - POLLER: Poller[1] Time: 0.0450 s, Poller: , Theads: N/A, Devices: 1 2017-04-10 14:45:33 - SYSTEM STATS: Time:1.3755 Method:cmd.php Processes:20 Threads:N/A Hosts:2 HostsPerProcess:1 DataSources:1 RRDsProcessed:1 2017-04-10 14:45:41 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:45:42 - POLLER: Poller[1] Device[4] STATUS: Device '172.16.136.112' is UP. 2017-04-10 14:45:42 - POLLER: Poller[1] Device[4] RECACHE: Processing 4 items in the auto reindex cache for '172.16.136.112'. 2017-04-10 14:45:42 - POLLER: Poller[1] Device[4] RECACHE DQ[1] OID: .1.3.6.1.2.1.1.3.0, output: 329694 2017-04-10 14:45:42 - POLLER: Poller[1] Device[4] RECACHE DQ[2] OID: .1.3.6.1.2.1.1.3.0, output: 329694 2017-04-10 14:45:42 - POLLER: Poller[1] Device[4] RECACHE DQ[4] OID: .1.3.6.1.2.1.1.3.0, output: 329694 2017-04-10 14:45:42 - POLLER: Poller[1] Device[4] RECACHE DQ[5] OID: .1.3.6.1.2.1.1.3.0, output: 329694 2017-04-10 14:45:42 - POLLER: Poller[1] Device[4] DS[78] CMD: perl /var/www/html/cacti2/scripts/unix_processes.pl, output: 241 2017-04-10 14:45:42 - POLLER: Poller[1] Time: 0.0235 s, Poller: , Theads: N/A, Devices: 1 2017-04-10 14:45:42 - SYSTEM STATS: Time:0.2151 Method:cmd.php Processes:20 Threads:N/A Hosts:2 HostsPerProcess:1 DataSources:1 RRDsProcessed:1 2017-04-10 14:45:51 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:45:52 - POLLER: Poller[1] NOTE: There are no items in your poller for this polling cycle! 2017-04-10 14:45:53 - SYSTEM STATS: Time:1.3098 Method:cmd.php Processes:20 Threads:N/A Hosts:2 HostsPerProcess:1 DataSources:1 RRDsProcessed:0

But i dont understand why realtime works but the normal one doesnt work :/

mbpakalin commented 7 years ago

Btw, i'm using ubuntu 16.04. On ubuntu repos, cacti is so old. We must download from cacti.net or github release.

cigamit commented 7 years ago

Well, you are running with a poller interval of 10 seconds, but I doubt your Data Source Profile is setup properly for that. I would use both a 5 minute cron process and the standard 5 minute update frequency until you are comfortable with it. Any time you change these settings on the poller tab, you need to re-populate the poller cache.

mbpakalin commented 7 years ago

I'll try that right away, but also i'm trying to manual run scripts. For example;

`root@mbpServer:/var/www/html/cacti2/log# php -q /var/www/html/cacti2/scripts/ss_host_cpu.php ss_host_cpu 172.16.136.112 4 2:161:500:1:10:secret:::::: get usage '0'

PHP Notice: Undefined offset: 1 in /var/www/html/cacti2/scripts/ss_host_cpu.php on line 29 PHP Notice: Undefined offset: 2 in /var/www/html/cacti2/scripts/ss_host_cpu.php on line 30 PHP Notice: Undefined offset: 3 in /var/www/html/cacti2/scripts/ss_host_cpu.php on line 31 PHP Notice: Undefined offset: 4 in /var/www/html/cacti2/scripts/ss_host_cpu.php on line 32 PHP Notice: Undefined offset: 5 in /var/www/html/cacti2/scripts/ss_host_cpu.php on line 50

root@mbpServer:/var/www/html/cacti2/log# ` it gives that errors.

mbpakalin commented 7 years ago

It worked like you said, i changed it to 5 minutes interval. Thanks so much for that. But why didnt worked on 10 seconds interval ? And how can i configure for that ?

cigamit commented 7 years ago

You need to first setup a Data Source profile for Data Sources that are to be polled at 10 seconds, and then assign them to the Data Templates. Right now everything should work at 10 seconds, but I have not seen your system, and frankly, we are trying to focus on development here and not so much support except when someone finds some oddity and can explain it clearly enough. You should really spend some time on the Data Source profiles. Cacti fully support having 10 seconds for some things 1 minute for others and even 5 and 10 minutes for others, all in the same system. But only when performed correctly. There is a new Packet publishing book in the works that will cover this and other topics. The author is Thomas Urban and he's pretty a real formidable Cacti user and he also sells support services.

mbpakalin commented 7 years ago

Thanks so much for all of your efforts, really i appreciate it and thanks for your help.

fcocquyt commented 5 years ago

I have a similar issue where my normal graphs are blank, but real time shows the data fine.

I see the poller retrieving the data fine

2018/11/14 15:25:02 - POLLER: Poller[1] Device[17] DS[276] SNMP: v2: 10.57.5.136, dsname: traffic_out, oid: .1.3.6.1.2.1.31.1.1.1.10.436256768, output: 189058770286670 2018/11/14 15:30:02 - POLLER: Poller[1] Device[17] DS[276] SNMP: v2: 10.57.5.136, dsname: traffic_out, oid: .1.3.6.1.2.1.31.1.1.1.10.436256768, output: 189073297906930 2018/11/14 15:35:02 - POLLER: Poller[1] Device[17] DS[276] SNMP: v2: 10.57.5.136, dsname: traffic_out, oid: .1.3.6.1.2.1.31.1.1.1.10.436256768, output: 189080322243512 2018/11/14 15:40:02 - POLLER: Poller[1] Device[17] DS[276] SNMP: v2: 10.57.5.136, dsname: traffic_out, oid: .1.3.6.1.2.1.31.1.1.1.10.436256768, output: 189090233545921 2018/11/14 15:45:02 - POLLER: Poller[1] Device[17] DS[276] SNMP: v2: 10.57.5.136, dsname: traffic_out, oid: .1.3.6.1.2.1.31.1.1.1.10.436256768, output: 189097728796119

but the RRD tool is recording NaN:

: root@sjcinf7201vmw415:rra # ; rrdtool fetch /usr/share/cacti/rra/sw2_eng_sjc_traffic_in_276.rrd AVERAGE | tail 1542236700: -nan -nan 1542237000: -nan -nan 1542237300: -nan -nan 1542237600: -nan -nan 1542237900: -nan -nan 1542238200: -nan -nan 1542238500: -nan -nan 1542238800: -nan -nan 1542239100: -nan -nan 1542239400: -nan -nan

Data Source info shows "Last DS" as a valid number for the counter, but Value == "NaN" - the Max is 10,000,000,000 - this should be fine for a counter which deals with the delta values - right?:

Data Source Items Type Minimal Heartbeat Min Max Last DS Value Unknown Sec
traffic_in COUNTER 600 0 10,000,000,000 151,718,057,458,296 NaN 4
traffic_out COUNTER 600 0 10,000,000,000 188,866,669,140,439 NaN 4
netniV commented 5 years ago

Check your permissions. You can also run the debug plugin if you are in 1.1.38 to check individual data sources. From 1.2 that plugin is integrated.

fcocquyt commented 5 years ago

Ugh - this seemed to be the issue: https://forums.cacti.net/viewtopic.php?f=2&t=47858

system timezone was GMT, while cacti (php.ini) is Pacific I made the system TZ Pacific and the data is showing up in the graphs now

thanks

netniV commented 5 years ago

Glad you found the cause.

sunthra commented 5 years ago

Hi, I'm facing the similar issue, graph has no data

image

but it works on real time

image

*using CentOS 7, Cacti 1.1.38

Please assist, thanks

netniV commented 5 years ago

Permissions, either file/folder or SELinux. If you search on here there are references to the things that will correct the SELinux problems and also many posts on the correct permissions.

sunthra commented 5 years ago

SELinux is disabled but no data