Unable to create graphs due to Data Source verification failure

[ssvenn]

I ran into the problem with "Damaged Graph" on 1.2.16 after editing my new templates for Brocade FC switches, and tried to fix it by upgrading to 1.2.17 - this didn't help so i erased the rrd files, reset the database and started from scratch with my exported template

I am seeing strange behaviour where only some of my queried interfaces are being created, most fail with the text "NOTE: Graph not for Data Query and index due to Data Source verification failure.

Rolling back to 1.2.16 and resetting the database again I was able to use the template to create graphs without getting this error message, and spine is able to collect values.

cacti_data_query_brocade_fcmgmt_switch.zip

Steps to reproduce the behavior:

1. Import the included templates

2. Create a Brocade FC Switch device

3. Create graphs from the the Brocade FCMGMT Switch query

[netniV]

Hmm, that should have used dsv_log which was left in to allow anyone to quickly log these things should we need it. I will patch.

[netniV]

So 8f6637c29 should resolve that now.

[netniV]

For reference, there is another fix for the other source types.

[shkmuradhssn]

lib/function.php

Can you please help me to know path of lib/function.php as even I encountered the same issue and deployed on Ubuntu 20

[netniV]

I've updated your ticket, there has been another update to the test_data_source function which should have resolved it before I knew someone else had the problem.

[Lejooohn]

Hello,

Sorry to reopen but yesterday i have updated my cacti to the latest version but actually it seems the issue is still there :

Maybe it's not related but in fact the message is the same :(

EDIT : to recover the good behaviour i have manually applied the fix purposed by @xmacan in lib/functions.php

[netadmin101]

I just updated here as well, also seeing this issue return in the latest version 1.2.18. We rarely saw the bad data problem, the common one for us is "due to Data Source verification failure"

[eschoeller]

Upgraded to 1.2.18 over here and this is still really broken for me. I can't make any new graphs. I might need to downgrade, since I'm dead in the water now because of this data source validation issue.

2021/07/16 22:07:15 - ERROR PHP NOTICE: Undefined index: cg in file: /cacti/cacti-1.2.18-prod/lib/template.php  on line: 1723
2021/07/16 22:07:15 - CMDPHP PHP ERROR NOTICE Backtrace:  (/graphs_new.php[42]:form_save(), /graphs_new.php[150]:html_graph_new_graphs(), /lib/html_graph.php[465]:host_new_graphs_save(), /graphs_new.php[243]:create_save_graph(), /lib/template.php[1723]:CactiErrorHandler())
2021/07/16 22:07:15 - DSV test_data_source = {"data_template_id":"366","host_id":861,"snmp_query_id":0,"snmp_index":"","suggested_vals":null}
2021/07/16 22:07:15 - DSV data_input = {"id":"58","type_id":"1","data_template_data_id":"13185","data_template_id":"366","active":"on","rrd_step":"60","name":"System Activity Reporter"}
2021/07/16 22:07:15 - DSV host = {"id":"861","poller_id":"1","site_id":"0","host_template_id":"50","description":"test","hostname":"test.colorado.edu","location":"","notes":"","external_id":"","snmp_community":"test","snmp_version":"2","snmp_username":"","snmp_password":"","snmp_auth_protocol":"","snmp_priv_passphrase":"","snmp_priv_protocol":"","snmp_context":"","snmp_engine_id":"","snmp_port":"161","snmp_timeout":"1500","snmp_sysDescr":"Linux test 4.19.0-17-amd64 #1 SMP Debian 4.19.194-2 (2021-06-21) x86_64","snmp_sysObjectID":"iso.3.6.1.4.1.8072.3.2.10","snmp_sysUpTimeInstance":"113964523","snmp_sysContact":"test","snmp_sysName":"test","snmp_sysLocation":"Boulder","availability_method":"2","ping_method":"1","ping_port":"23","ping_timeout":"400","ping_retries":"1","max_oids":"60","bulk_walk_size":"-1","device_threads":"6","deleted":"","disabled":"","monitor":"on","monitor_text":"","status":"3","status_event_count":"0","status_fail_date":"0000-00-00 00:00:00","status_rec_date":"2021-07-15 20:01:11","status_last_error":"Device did not respond to SNMP","min_time":"0.76103","max_time":"29.11401","cur_time":"1.10793","avg_time":"1.13460","polling_time":"0.153","total_polls":"2117","failed_polls":"575","availability":"72.83890","last_updated":"2021-07-16 22:06:36"}
2021/07/16 22:07:15 - DSV script_path = perl /cacti/cacti-1.2.18-prod/scripts/sarfetch.pl '' '2' 'public' '161' '1500'
2021/07/16 22:07:15 - DSV num_output_fields_sql = SELECT SQL_NO_CACHE id FROM data_input_fields WHERE data_input_id = ? AND input_output = "out" AND update_rra="on"
2021/07/16 22:07:15 - DSV num_output_fields = 34
2021/07/16 22:07:15 - DSV data_source_item_name = 
2021/07/16 22:07:15 - DSV script_path = perl /cacti/cacti-1.2.18-prod/scripts/sarfetch.pl '' '2' 'public' '161' '1500'
2021/07/16 22:07:16 - DSV output =

[bmfmancini]

I believe there was a work around suggested on this thread and I thought there was a commit that fixed it

On Sat., Jul. 17, 2021, 00:11 Eric Schoeller, @.***> wrote:

Upgraded to 1.2.18 over here and this is still really broken for me. I can't make any new graphs. I might need to downgrade, since I'm dead in the water now because of this data source validation issue.

2021/07/16 22:07:15 - ERROR PHP NOTICE: Undefined index: cg in file: /cacti/cacti-1.2.18-prod/lib/template.php on line: 1723 2021/07/16 22:07:15 - CMDPHP PHP ERROR NOTICE Backtrace: (/graphs_new.php[42]:form_save(), /graphs_new.php[150]:html_graph_new_graphs(), /lib/html_graph.php[465]:host_new_graphs_save(), /graphs_new.php[243]:create_save_graph(), /lib/template.php[1723]:CactiErrorHandler()) 2021/07/16 22:07:15 - DSV test_data_source = {"data_template_id":"366","host_id":861,"snmp_query_id":0,"snmp_index":"","suggested_vals":null} 2021/07/16 22:07:15 - DSV data_input = {"id":"58","type_id":"1","data_template_data_id":"13185","data_template_id":"366","active":"on","rrd_step":"60","name":"System Activity Reporter"} 2021/07/16 22:07:15 - DSV host = {"id":"861","poller_id":"1","site_id":"0","host_template_id":"50","description":"test","hostname":"test.colorado.edu","location":"","notes":"","external_id":"","snmp_community":"test","snmp_version":"2","snmp_username":"","snmp_password":"","snmp_auth_protocol":"","snmp_priv_passphrase":"","snmp_priv_protocol":"","snmp_context":"","snmp_engine_id":"","snmp_port":"161","snmp_timeout":"1500","snmp_sysDescr":"Linux test 4.19.0-17-amd64 #1 SMP Debian 4.19.194-2 (2021-06-21) x86_64","snmp_sysObjectID":"iso.3.6.1.4.1.8072.3.2.10","snmp_sysUpTimeInstance":"113964523","snmp_sysContact":"test","snmp_sysName":"test","snmp_sysLocation":"Boulder","availability_method":"2","ping_method":"1","ping_port":"23","ping_timeout":"400","ping_retries":"1","max_oids":"60","bulk_walk_size":"-1","device_threads":"6","deleted":"","disabled":"","monitor":"on","monitor_text":"","status":"3","status_event_count":"0","status_fail_date":"0000-00-00 00:00:00","status_rec_date":"2021-07-15 20:01:11","status_last_error":"Device did not respond to SNMP","min_time":"0.76103","max_time":"29.11401","cur_time":"1.10793","avg_time":"1.13460","polling_time":"0.153","total_polls":"2117","failed_polls":"575","availability":"72.83890","last_updated":"2021-07-16 22:06:36"} 2021/07/16 22:07:15 - DSV script_path = perl /cacti/cacti-1.2.18-prod/scripts/sarfetch.pl '' '2' 'public' '161' '1500' 2021/07/16 22:07:15 - DSV num_output_fields_sql = SELECT SQL_NO_CACHE id FROM data_input_fields WHERE data_input_id = ? AND input_output = "out" AND update_rra="on" 2021/07/16 22:07:15 - DSV num_output_fields = 34 2021/07/16 22:07:15 - DSV data_source_item_name = 2021/07/16 22:07:15 - DSV script_path = perl /cacti/cacti-1.2.18-prod/scripts/sarfetch.pl '' '2' 'public' '161' '1500' 2021/07/16 22:07:16 - DSV output =

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Cacti/cacti/issues/4250#issuecomment-881814855, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADGEXTB6UIXFWFOC5Z2PU33TYD7H5ANCNFSM44FSRYVQ .

[netadmin101]

@eschoeller Try this: in lib/functions.php, change line 1728 from the test_data_sources function from return false to return true. That's the workaround I'm currently using.

[eschoeller]

Yup I found that fix and meant to update my comment but got sidetracked onto other things. The work-around is really simple ... it essentially disables the entire data source verification function. I think I was involved in the discussion(s) where this function originated from - due in part to automation creating graphs that end up having no data - but I think a LOT more testing needs to occur to get this working right. Perhaps it needs to be marked experimental for now and manually enabled in the settings until all the issues can be tested and worked out. Having any errors with this function has a pretty negative impact on the overall functionality of the software, especially when it's included in the last two major releases.

[gj00354347]

Hi Team ,

we also upgraded cacti to 1.2.18 in Test and observed similar issues. Although there is no hard and fast rule like when it works and when not . so we enabled the log verbosity to Debug in cacti and found below lines .

2021-07-21 15:54:46 - ERROR PHP NOTICE: Undefined index: cg in file: /opt/SP/cacti/cacti/lib/template.php on line: 1723 2021-07-21 15:54:46 - CMDPHP PHP ERROR NOTICE Backtrace: (/graphs_new.php[42]:form_save(), /graphs_new.php[150]:html_graph_new_graphs(), /lib/html_graph.php[465]:host_new_graphs_save(), /graphs_new.php[243]:create_save_graph(), /lib/template.php[1723]:CactiErrorHandler())

PFB snap

Best Regards, Gopal

[bernisys]

As my colleague just wrote, we also see the "Bad data" error here, and some graphs are not created. I've looked a bit deeper into it while he was writing the comment and I'm not sure about the source, but as he points out, we're seeing this behavior:

In cacti/lib/template.php in function create_save_graph() on line 1723 we get an error for an undefined index "cg" in the "values" array - this could be a hint.

The function is called by cacti/graphs_new.php in line 243 after the $values array is filled. But filling $values depends on the loop thru the "$_POST" variable, which seems to be a built-in in PHP, containing the form as given by the "POST" HTML transfer method. If that POST was empty, the array would not have been created.

Haven't looked too deeply into it, but it could be a hint. @netniV does that ring a bell maybe?

We will go for the workaround now, as this function is critical to us. Maybe a bit more data-sanitizing code would be nice in cacti .. for example, checking if the variables contain valid data, and if not, give us a bit of side-info in the logs, so that it is easier to relate things to each other.

[netniV]

Basically, the form that creates the selection list, sets various field names and when selected, these are POST'd to back. Those are then split apart to generate the various elements. If you are using v1.2.18, make sure you have the latest lib/functions.php from the 1.2.x branch and see if you get exactly the same error.

[gj00354347]

Hi Team ,

we also see one strange behavior like the graph template associated with a host templates should be drawn the moment we add the device under the right template . but what we observed is like no graph are created even for the ones whose graph template is associated with host template and we see in device edit page like "NOT BEING GRAPHED" .

we tried to sync the device to device template but that did not work .

yes we tried a few work around like we removed the association of device with existing Host template and assigned a generic template and then again assigned the right (previous template) template and Bingo all were there !!

and I enabled debug log as well as developer log but could not get anything in the logs . PFB snap I see in DEVICE EDIT PAGE

yes we have tried the work around suggested below but this failure of graphs associated with device template even doesnt work after applying the workaround. The only work around at our disposal is like break the existing association of device with present template and associate with some other host template and again associate it with previous template and all the graphs are created associated with that host template . this is really strange behaviour. like in the first go , all the graphs associated with that host template should have been created when we add a device under a host template but this do not happen in our case.

we are on cacti 1.2.18 including the change in lib/functions.php ,"change line 1728 from the test_data_sources function from return false to return true".

Best Regards, Gopal

[bernisys]

If you are using v1.2.18, make sure you have the latest lib/functions.php from the 1.2.x branch and see if you get exactly the same error.

Ok we will do so in our test environment, while we are pushing out 1.2.18 already in prod, using the workaround to skip the data source validation. This gives us a bit of time to observe in an uncritical environment.

[skyjou]

This issue still exist.

[chrcoluk]

Hi guys just a "me too" post.

I think this needs to be treated as experimental and in a new version of cacti changed to off by default, it for sure needs to be made optional, instead of havng to hack the source code.

[will60113]

my cacti is Version 1.2.18. I've already solved this problem by editing the config, and finally the graph template can show up.

But another problem comes up, some of the source cannot be graphed, the graphs are still empty. (all of them are Windows server 2012 R2)

[ARC1450]

Not sure this is closed and fixed.

I have some graph templates that are supposed to be empty. They're graph templates that have up to 4 items that can be graphed (specifically, I'm pulling IPMI sensor data, and when you have a couple of different motherboards with OOB, you can have different numbers of fans on each).

I create each individual fan sensor graph, then go back and delete the graph, leaving the data source. Then I tie the data source to the empty graph template and presto, rinse and repeat 3 to 4 times, and presto. . .I can either have 3 or 4 fan IPMI systems that use the same graph template.

The fix that bypassed this error:

The Operation was successful.  Details are below.

Graph Not created for IPMI Fans (4 Sensors) due to bad data

was to hop into vim, search for "test_data_sources", set it to always return "true", and I was able to create my empty graph template.

On a side note, I've noticed that what users used to be able to do (this, for example) are being "checked" to make sure they're not bad. Just because a graph template doesn't have an existing data source doesn't mean it's a bad graph template. I've ran into other issues where I've had to edit the underlying PHP to create graphs I've been able to create in the past (the ucd/net Memory graph is one such graph, I believe) that now Cacti refuses to create for some reason.

[netniV]

The issue raised in this post was identified and fixed. There have been a few other corner cases that have also been fixed so the test_data_sources should be working a lot better when 1.2.19 is released. That is still undergoing testing to make sure we are happy with it as one or two more issues have been raised in other areas that require some testing prior to release.

[TheWitness]

In 1.2.19, you will be able to mark the Graph Templates to be tested first prior to creation.

This way, if you have a Data Source that takes 20 seconds to test and create 1000 graphs based upon it, you won't have to wait 20,000 seconds for the GUI to respond. That is an extreme to prove the point of adding the setting.

[ARC1450]

Awesome! Thanks Witness!

I know y'all are basically making things idiot-proof. It's just a little frustrating when things work for 2, 3, or 4 years then suddenly poof. . they don't work and you have to go digging on Dr. Google only to find someone say "well, it was decided that we should be checking this, that, and another; you are a corner case".

I realize there are corner cases, but maybe a toggle on the configuration/settings page for "development" and "production" could be implemented, where the "development" removes all the sanity checking and "idiot-proofing" being done, and "production" only allows for extremely safe, expected conditions. That's the way I've seen it done for a lot of systems I work on (I'm in IT) because, well, some admins are power users that can make their own templates/queries.

[TheWitness]

I'm notorious for breaking things and that release broke things bigly.

[ctrowat]

We are also stuck on this issue (running 1.2.18) and as a result cannot create any graphs for any device. I am not sure what to think there, at the crux of it (for us at least) is that the interfaces.xml snmp query file contains all the possible OIDs to query, but not every device supports every OID, so this is failing to create graphs for one thing because the device does not support another. Would it be possible to make the test specific to the graph being created?

[jchambers777]

Hi Team ,

we also see one strange behavior like the graph template associated with a host templates should be drawn the moment we add the device under the right template . but what we observed is like no graph are created even for the ones whose graph template is associated with host template and we see in device edit page like "NOT BEING GRAPHED" .

we tried to sync the device to device template but that did not work .

yes we tried a few work around like we removed the association of device with existing Host template and assigned a generic template and then again assigned the right (previous template) template and Bingo all were there !!

and I enabled debug log as well as developer log but could not get anything in the logs . PFB snap I see in DEVICE EDIT PAGE

yes we have tried the work around suggested below but this failure of graphs associated with device template even doesnt work after applying the workaround. The only work around at our disposal is like break the existing association of device with present template and associate with some other host template and again associate it with previous template and all the graphs are created associated with that host template . this is really strange behaviour. like in the first go , all the graphs associated with that host template should have been created when we add a device under a host template but this do not happen in our case.

we are on cacti 1.2.18 including the change in lib/functions.php ,"change line 1728 from the test_data_sources function from return false to return true".

Best Regards, Gopal

Thank you so much this worked for me

[TheWitness]

@jchambers777,

This should all be fixed up on the 1.2.20 release that we are nearing the release of. Might even be able to get it out ahead of time depending on all our holiday schedules. Have to play wait and see. If you are daring, you can just update to the 1.2.x branch.

cd /var/www/html
git clone -b 1.2.x https://github.com/cacti/cacti.git cacti.new
/bin/cp -rpf cacti.new/* cacti
chown -R apache:apache cacti

Then, refresh your browser (F5 on Windows). If there are any updates, they will come from people like yourself testing.

TheWitness

[jchambers777]

@jchambers777,

This should all be fixed up on the 1.2.20 release that we are nearing the release of. Might even be able to get it out ahead of time depending on all our holiday schedules. Have to play wait and see. If you are daring, you can just update to the 1.2.x branch.

cd /var/www/html
git clone -b 1.2.x https://github.com/cacti/cacti.git cacti.new
/bin/cp -rpf cacti.new/* cacti
chown -R apache:apache cacti

Then, refresh your browser (F5 on Windows). If there are any updates, they will come from people like yourself testing.

TheWitness

I was wondering do you think upgrading to this specific one will correct my issue? Here is what I am facing now: ERROR: opening C:/Apache24/htdocs/cacti/rra/cacti_server_boost_peak_memory_4.rrd the system cannot find the file.

It just seems like it's not creating the graphs in the specified area so cacti is looking for them, but they don't appear in that area.

Thanks for replying to me I do appreciate it.

[TheWitness]

I suspect it's a permission issue. I've not used Windows in so long that I dread even working on it again. Our windows guy is busy bouncing around his young children too. From experience, there is not much time for Cacti when they are young.

Cacti "calculates" the base_path in the poller (or used to). Go into the database and run this command, then check permissions:

SELECT * FROM settings WHERE name LIKE '%path%';

You'll be looking for path_webroot, I'm pretty sure.

[TheWitness]

The repair script will fix the damaged graph. It's easier for me to say take it all vs. cherry picking files. It's real stable now.

[jchambers777]

sorry for my ignorance, where can I find this repair script? Github?

On Sat, Dec 11, 2021 at 6:13 AM TheWitness @.***> wrote:

The repair script will fix the damaged graph. It's easier for me to say take it all vs. cherry picking files. It's real stable now.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Cacti/cacti/issues/4250#issuecomment-991621206, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANOCCK3753KBBMQUYAVYYJLUQM55TANCNFSM44FSRYVQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

[bmfmancini]

/cli/repair_graphs.php On Mon, Dec 13, 2021 at 11:36 AM jchambers777 ***@***.***> wrote: > sorry for my ignorance, where can I find this repair script? Github? > > On Sat, Dec 11, 2021 at 6:13 AM TheWitness ***@***.***> wrote: > > > The repair script will fix the damaged graph. It's easier for me to say > > take it all vs. cherry picking files. It's real stable now. > > > > — > > You are receiving this because you were mentioned. > > Reply to this email directly, view it on GitHub > > , or > > unsubscribe > > < > https://github.com/notifications/unsubscribe-auth/ANOCCK3753KBBMQUYAVYYJLUQM55TANCNFSM44FSRYVQ > > > > . > > Triage notifications on the go with GitHub Mobile for iOS > > < > https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 > > > > or Android > > < > https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub > >. > > > > > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > , or > unsubscribe > > . > -- Thank you Sean Mancini,(Six Sigma LBBIT®, ITIL,CEA-IT®,SCRUM SMPC®) Owner/Principal Engineer www.seanmancini.com “Companies spend millions of dollars on firewalls, encryption, and secure access devices, and it’s money wasted because none of these measures address the weakest link in the security chain.” *– Kevin Mitnick*

[TheWitness]

Actually, you should consider the 1.2.x branch. It's just about ready for release. After that, it should work well.

[jchambers777]

FINALLY! I figured how to add the graphs and not get the error message. It's working now.

Thank you for your help

https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=icon Virus-free. www.avast.com https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=link <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Mon, Dec 13, 2021 at 10:40 AM Sean Mancini @.***> wrote:

/cli/repair_graphs.php On Mon, Dec 13, 2021 at 11:36 AM jchambers777 ***@***.***> wrote: > sorry for my ignorance, where can I find this repair script? Github? > > On Sat, Dec 11, 2021 at 6:13 AM TheWitness ***@***.***> wrote: > > > The repair script will fix the damaged graph. It's easier for me to say > > take it all vs. cherry picking files. It's real stable now. > > > > — > > You are receiving this because you were mentioned. > > Reply to this email directly, view it on GitHub > > , or > > unsubscribe > > < > https://github.com/notifications/unsubscribe-auth/ANOCCK3753KBBMQUYAVYYJLUQM55TANCNFSM44FSRYVQ > > > > . > > Triage notifications on the go with GitHub Mobile for iOS > > < > https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 > > > > or Android > > < > https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub > >. > > > > > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > , or > unsubscribe > < https://github.com/notifications/unsubscribe-auth/ADGEXTC2SKVSJ6A4NUAW2MDUQYOIRANCNFSM44FSRYVQ > > . > -- Thank you Sean Mancini,(Six Sigma LBBIT®, ITIL,CEA-IT®,SCRUM SMPC®) Owner/Principal Engineer www.seanmancini.com “Companies spend millions of dollars on firewalls, encryption, and secure access devices, and it’s money wasted because none of these measures address the weakest link in the security chain.” *– Kevin Mitnick* — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub , or unsubscribe . Triage notifications on the go with GitHub Mobile for iOS or Android .