Open pasztorl opened 1 month ago
By default pools should have 3 replicas. Did you change some setting so it would default to 1 replica?
To ensure the replication of the .mgr
pool is set, you need to specify its CephBlockPool under cephBlockPools:
in the helm values. An example of the spec is in pool-builtin-mgr.yaml.
@travisn thanks for the quick answer. I have the replicated: 3 in the values and all pools got replicaX3 but .mgr not.
@pasztorl How are you deploying the cluster? Which rook version is it? By default, .mgr pool is created for 3 replicas.
sh-4.4$ ceph osd pool ls detail
pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode on last_change 19 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 3.03
Have you deployed using the [cluster-test.yaml](https://github.com/rook/rook/blob/master/deploy/examples/cluster-test.yaml#L65)
@sp98 am I missed something from the values?
I've created a cluster with 3 nodes. The blockpool replicated to 3 nodes. By default .mgr only have one replica, so the operator can't finish the cluster installation. This log entry repeating:
2024-07-05 13:08:07.357184 I | op-osd: OSD 0 is not ok-to-stop. will try updating it again later
After I do:
ceph osd pool set .mgr size 3
The operator finishes the installation:
I not found an option for .mgr to set the replication level in the helm chart. What is the best practice about this situation?