search

Cacti / cacti

Cacti ™
http://www.cacti.net
GNU General Public License v2.0
1.65k stars 406 forks source link

Remote pooler stuck in Heartbeat #5670

Closed enrikio closed 8 months ago

enrikio commented 9 months ago

I deployed a new remote pooler on remote location and now have trouble when doing full sync of both data collectors of getting the remote pooler into "Idle" (it always stays in Heartbeat). All database GRANTS are correct from both ends (Both are on cacti 1.2.26 and spine 1.3.0, spine.conf and config.php were also edited accordingly on both ends with correct db values of both). Anyone knows the root cause of this? We have site to site vpn to remote cacti which is in completely other network segment. Initially we just opened ports 3306, 443 and 80, and now also all ports (I did all tests and both ends listen on ports 3306 and 80. On 443 it can't listen because I'm only running on 80 currently, but that is okay). I also did db connection test on linux in mysql and from cacti and also everything is ok. I commented out #bind-address = 127.0.0.1 (in mariadb.conf.d) on both ends also so both can listen to each other on their ports.

Now the strange thing is, I tried installing the cacti pooler in our network segment locally with completely same settings and it works (right away I can see status idle on main cacti)! Only difference here is, that this test pooler had same network segment as main cacti.

Is there something else i should configure in some config in linux maybe in case these poolers are in other networks? What I see on main cacti in logs when doing full sync (checked main cacti and remote cacti): REPLICATE ERROR: Unable to connect to Remote Data Collector *** WEBUI WARNING: Some selected Remote Data Collectors in [7] failed synchronization by user admin, Successful/Failed[0/1]. See log for details.

I also posted this on cacti forums with no luck so far: https://forums.cacti.net/viewtopic.php?p=293897

bmfmancini commented 9 months ago

This is an interesting one

May I ask why you are running spine 1.3 with 1.2.26?

1.3 is developmental and should not be used in prod

When the poller is in heartbeat mode have you tried running spine manually to see the output?

enrikio commented 9 months ago

This is an interesting one

May I ask why you are running spine 1.3 with 1.2.26?

1.3 is developmental and should not be used in prod

When the poller is in heartbeat mode have you tried running spine manually to see the output?

No specific reason..I just chose the latest version. It works fine though on main cacti and remote pooler which is in same network. As for running spine manually, you will need to give me more information please. Update: I completely forgot to add spine config path on remote pooler (/usr/local/spine/etc/spine.conf)... Now I get success in logs when doing full sync of both poolers (but still have trouble getting remote pooler into Idle - Still in heartbeat mode.): WEBUI NOTE: All selected Remote Data Collectors in [8] synchronized correctly by user admin REPLICATE Synchronization of Poller 8 completed

enrikio commented 9 months ago

Same issue with spine 1.2.26 on cacti 1.2.26 (on main and remote cacti pooler)...

TheWitness commented 8 months ago

You have to ensure that there are two way communications. That the main server can connect to the remote, and vice versa. Your remote likely can not talk to the main server over the MariaDB/MySQL port.

Also, also make sure the TimeZones are correct on both servers and the remote poller configuration and do a Full Sync afterwards.

TheWitness commented 8 months ago

Finish this discussion on the forums if you don't mind.