Option to add external DS from CLI

[jpobeda]

Hi there,

It would be nice to have the ability to add custom external data sources and graphs for these via CLI.

My case is, thousands of RRD files from a different monitoring tool I'd like to bring to Cacti to put the whole Cacti + plugins functionality to work.

I'm doing this right now by manually adding them via web gui but it's a pain.. I know it might no be very useful to others but it might, who knows :D

Ref: http://forums.cacti.net/viewtopic.php?t=12202

If you need any further details, don't hesitate to ask!

Thanks!

[cigamit]

What system are you mapping from?

[jpobeda]

It's Munin.

[cigamit]

And what type of things are you doing with those data sources in Cacti? Manual Graph creation or something else?

[cigamit]

Next question, what does the host and data source structures look like can you dump the database as follows (assuming MySQL)

mysqldump -d munin_database > output_for_this_ticket.sql

[cigamit]

Ope, it's old school. No database. I had to rub my eye and say "who does that anymore".

[jpobeda]

So far I'm only graphing and using them in weathermaps BUT weathermap has also an issue with picking up the data sources I manually put into cacti, don't know why o what's different so it can causes that issue. So I have also to do the weathermap stuff manually as well :(

I'm not doing much in Cacti because of that, can't see how automate this easily.

That's right, no database hehehe.

The reason why Munin is...I'm not the admin and the way they are polling those specific devices is by agent because they don't want to run snmp or any other stuff, anyway..

Does that answer your question?

[cigamit]

So, the agent sends the data to the Munin master and then write to the correct RRDfiles. How does templating work? What protocol does Munin use? Is it SSH? Let me know.

[jpobeda]

The poller connects to each node on tcp/4949 ( by default), fetches the data and update rrd files. You could specify ssh or cmd but by default it's using 'munin' so I guess it's a custom socket made by munin because it sort of allows you to run some 'commands' like list, version, nodes, quit, etc.

You've said it, it's old school so config is by manually editing files......

You basically define node ip/hostname in separated files on your poller server. Every node runs locally the scripts you have set up and as far as I can see they read data from /proc and store it in...memory?

I also noticed that it uses 1 DS per RRD file, always called the same (42) so for a normal interface traffic thera are at least 2 files, 1 for incoming traffic and 1 for outgoing.

Templates are more for the web interface I think, to organise and define the graph structure. to group devices and/or graphs categories but that's all I think.

I'm not really an expert on munin I find it very confusive.

If you want me to attach and rrdtool info output, let me know.

I'll read a little bit more about it because I don't know much as you may noticed.

Thanks

[cigamit]

This whole thing begs for a munin plugin to have the Munin master be the Cacti server, but I'm not sure that follows the Munin paradigm. This way you could add cacti devices and make them a special class of hosts and mark them as such, and then replace the Munin master with Cacti.

[jpobeda]

If you leave this opened I'll post my findings later on. It may be better to focus it as 'externally updated' that way could be also useful for other software such as mrtg, zabbix, etc.

[ronytomen]

Closed due to inactivity. @jpobeda let us know if you are still interested in discussing this.

[cigamit]

I am going to re-open this one. Having a cli for such a thing is a minor change. However, we are looking for a volunteer to help with this.

[jpobeda]

The answer to how Munin collect the interface data is from SNAPSTATS so it sort of gets this output

snapstats: SNAPSTATS 14192378.31 53743674.32 ; eth0 23078181652929 35374101814 22852418299649 54789602973 , eth1 9250581720047 21916508720 14073472241307 19790953832 , eth2 0 0 0 0 , eth3 0 0 0 0 , eth4 1906107390900 4487000271 3852432405704 6819817011 , eth5 2737867878031 4356776112 215608576 1684442 , bond0 25816049530960 39730877926 22852633908225 54791287415 , bond1 11156689110947 26403508991 17925904647011 26610770843 , bond1.2 156130921868 296002472 19053623829 161921941 , bond1.3 235565513117 2930719530 5404089228779 9383815272 , bond1.9 14864396625 149568310 339438467375 253361337 , ;

On the node, the munin-node agent creates a symbolic link to the if_ plugin for every interface up. If you telnet to 1 of the nodes on port 4949 (default) you can actually fetch the current stats, that emulates what the master munin poller gets.

munin node at somehost.somedomain.com fetch if_bond0 down.value 25866658083644 up.value 22864678069228

Then the master writes those values to each RRD file. A single interface has 2 RRD files, 1 for incoming and 1 for outgoing traffic. cacti_#557_down.txt cacti_#557_up.txt

I attached the rrdtool info output.

[cigamit]

Can you see if the add_datasource.php that was contributed is close enough?

[cigamit]

Since we've added the add_datasource.php to Cacti, I think this can be closed.

[jpobeda]

I'll have a look and provide feedback

[cigamit]

Keep us posted.