Open nobuto-m opened 1 month ago
UtkarshBhatthere
commented
1 month ago The way it is done with MicroCeph is that Mon deployment is automatic (upto 3 nodes). So 1,2 and 3 node deployments will always have (mon, mgr, mds) daemons running. And OSD deployment can be done on any node. So upto 3 node (MON and MON/OSD are possible) and for more than 3 nodes (MON + MON/OSD + OSD are possible).
nobuto-m
commented
1 month ago What happens if the initial deployment has 7 nodes? 3 out of 7 are randomly selected as MON?
What happens if 1 or 2 out of the 3 initial MONs get down? Do we have an ability with the charm to move MON service back to the original set of physical machines to over come an issue like the following? https://bugs.launchpad.net/nova/+bug/1452641
UtkarshBhatthere
commented
1 month ago What happens if the initial deployment has 7 nodes? 3 out of 7 are randomly selected as MON?
Technically yes, The first 3 nodes to join the microcluster will spawn MON service.
What happens if 1 or 2 out of the 3 initial MONs get down? Do we have an ability with the charm to move MON service back to the original set of physical machines to over come an issue like the following?
MicroCeph supports migrating "auto services" (mon, mgr, mds) to another node. The charm operator can juju run the migrate command to move services across the cluster.
I get the zeal of this bug, and I agree that the ability to plan services beforehand would be an awesome addition to charm-microceph but we lack 1.) A clear view of a juju native way to do it (I am asking around in the juju matrix channel) 2.) The bandwidth to implement this in the current development cycle.
UtkarshBhatthere
commented
1 month ago Ongoing discussion for REF
https://bugs.launchpad.net/charm-microceph/+bug/2067282
Let's say a design is with 3 minimum hosts to have 3 replicas of Ceph MON+OSD, and other nodes will have OSD only to add more capacity to the cluster. As of today, there is no way to achieve that kind of designs since there is no way to differentiate MON, MON+OSD, OSD in Sunbeam deployments.
Why does having 3 MON statically matter? OpenStack itself doesn't react well to an event of adding/removing MONs e.g. https://bugs.launchpad.net/nova/+bug/1452641 And clients outside of Sunbeam managed needs static entry points to access Ceph data (It's pretty common to share a Ceph cluster between OpenStack workload such as VMs and other such as S3).