Cacti 1.2.2: Multi poller: Central Cacti cannot communicate with devices

[stefanbrudny]

Describe the bug Central Cacti cannot access the remote pollers hosts

When accessing Management -> Devices -> a device user insted of SNMP / ICMP details gets:

404 Not Found nginx/1.14.1

Same valid for apache. Surprisingly, all Remote Pollers display the SNMP / ICMP details correctly., even if there is no direct routing between the Remote Poller and a device on different Remote Poller.

To Reproduce Install multi poller and ask for unreacheable devices from Central Cacti. Expected behavior SNMP / ICMP details should be executed by correct Spine or API from Remote Poller should be leveraged to retreive the data.

[netniV]

If a device is associated to a remote poller, it calls the remote_agent.php on the remote server. You likely have a misconfiguration of the location of cacti setup against the poller.

[stefanbrudny]

Well, I do not know whether it is configurable somewhere for the poller location, so I won't close it.

But I streamlined all contexts and it finally worked so for me it is solved.

[netniV]

If you have resolved your issue, then it can be closed.

[max6491]

@stefanbrudny - can you please share me the steps to resolve the issue ? .. I am in the same situation with multi poller configuration.

[stefanbrudny]

sure, to some extent, as I am not sure how this works in detail.

In my case it was necessary to use same $url_path in config.php for all pollers. I can afford to have all cactis in root (/) context, I am not sure if going to some subcontext wouldn't break the things, I just did not test.

For me it works. I currently have 14 pollers on single host (to query IPv4 addresses in same networks), and all I can say is Cacti developers have never tested anything at this scale :-) Otherwise, they would at least try to wonder why this password for local (in my case its still remote) db is to be typed TWICE for remote pollers? :-) not to mention other improvements so one can differentiate the pollers. It is way too easy to lost to which Cacti you are currently logged in.

[TheWitness]

@stefanbruny, even though you are running in an unsupported configuration, it would be good to know how well things are going so far now that you have played with it for a while.

[stefanbrudny]

@TheWitness

Yeah, dust has gone away already and I am running some Cactis again. You know, I love Cacti, but its a mix of love and hate. Sorry for too many rants.

• I tried twice to go at scale, way above 200k of devices, but I failed for numerous reasons already mentioned in my tickets and in forum. No working device search or bulk management and spine unable to cope with thousands of offline devices were among of critical reasons.
• now I run cacti for approximately 60 devices, with several hundreds of graphs. This is small but extreme deployment, and actually it was the architecture what made perfect match for me.

My setup is as follows:

• single server with a 14 routing namespaces (man ip-netns) and one global namespace
• each ns has its routing table and connected GRE tunnels towards some networks. These networks use same IPv4 network (eg 192.168.0.0/16) so its no good way to access the hosts located there from global namespace
• each namespace contais complete nginx with cacti (actually it is not really needed once all runs fine). Again, each of these nginx runs in separate routing ns;
• in global namespace there is a cron which runs a poller into a namespace, eg.

*/1 * * * *     root ip netns exec dm10 /usr/bin/php /var/www/cacti-dm10/poller.php.modified
*/1 * * * *     root ip netns exec dm20 /usr/bin/php /var/www/cacti-dm20/poller.php.modified
[...]

• Poller had to be modified, as each Spine needs different configuration (different database, right) and it had to be different file, as it gets overwritten between servers.

• obviously, each spine runs in different ns, but as a target it has same ip addresses, from overlapping ipv4 networks

• this approach could be different. I could have remap using NETMAP target with iptables, and actually I did it as well in tests. Tricky part is that NETMAP (in short) would support only ~6.8M of IPv4 in all private ipv4 networks. Netmap at scale to ipv6 was also considered, but linux doeas not support that in networking stack (!!) and I would have to go to OpenBSD or some Tayga userspace, both of which I hate. I am trying yet Jool, but this is a side project.

This is my network config: for i in seq 10 34; do ip netns add dm$i ip link add veth$i type veth peer name br-veth$i ip link set veth$i netns dm$i ip -n dm$i address add 192.168.$i.100/24 dev veth$i ip -n dm$i link set veth$i up ip link set br-veth$i up ip link add name br$i type bridge ip link set br$i up ip link set br-veth$i master br$i ip address add 192.168.$i.1/24 dev br$i ip -n dm$i route add 0.0.0.0/0 via 192.168.$i.1 ip -n dm$i address add 10.0.1.1/24 dev lo done

To sum up: I was counting somehow again that I could go to scale. Instead, I had to limit my apetite to some gateways only. This is second time this chance is inevitably gone, and I doubt I'll have real occassion to go to such scale again in next years or ever again.