search

LINBIT / linstor-server

High Performance Software-Defined Block Storage for container, cloud and virtualisation. Fully integrated with Docker, Kubernetes, Openstack, Proxmox etc.
https://docs.linbit.com/docs/linstor-guide/
GNU General Public License v3.0
953 stars 75 forks source link

Satellite recovery after controller failover #403

Open dniasoff opened 4 months ago

dniasoff commented 4 months ago

Hi

Just testing controller HA (using drbd-reactor) and simulating failures.

Drbd-reactor is working nicely but satellite nodes aren't recovering automatically.

Here is what I see after a failure

image

However, if I restart the satellite daemon on both nodes - here is what I see

image

I could somehow work out how to create a process to restart all satellite nodes after controller failure but I guess somehow there should be a way to allow the satellite service to "self-heal" without a restart.

Thanks

Daniel

ghernadi commented 4 months ago

Yes, once the satellite is up again, the controller should automatically reconnect to the satellite and get the resource UpToDate again. Can you verify if the satellite 1) did start and 2) is shown as Online in linstor node list?

If that looks fine, please check for possible ErrorReports (via err list) to see if something happened that could help us why LINSTOR did not properly restore the state again.