search

canonical / microk8s-core-addons

Core MicroK8s addons
Apache License 2.0
43 stars 34 forks source link

Only a single diskpool created #219

Closed meatballs closed 1 year ago

meatballs commented 1 year ago

Only a single diskpool created

Similar to #133, only a single diskpool is created in my 3 node cluster (3 x Raspberry Pi 4 running Ubuntu 22.04 server and microk8s 1.27)

kubectl get diskpool -n mayastor
NAME                   NODE     STATUS   CAPACITY      USED   AVAILABLE
microk8s-glados-pool   glados   Online   21449670656   0      21449670656
kubectl get all -n mayastor
NAME                                              READY   STATUS    RESTARTS   AGE
pod/mayastor-io-engine-l4lzm                      0/1     Pending   0          5m23s
pod/mayastor-io-engine-979kg                      0/1     Pending   0          5m23s
pod/mayastor-csi-node-mgtcq                       2/2     Running   0          5m23s
pod/mayastor-csi-node-f8nkw                       2/2     Running   0          5m23s
pod/etcd-operator-mayastor-8574f998bc-clbsf       1/1     Running   0          5m23s
pod/mayastor-csi-node-fzv42                       2/2     Running   0          5m23s
pod/etcd-kp7s728fz9                               1/1     Running   0          5m
pod/mayastor-agent-core-f7ccf485-zxht9            1/1     Running   0          5m23s
pod/mayastor-io-engine-h82xp                      1/1     Running   0          5m23s
pod/mayastor-operator-diskpool-5b4cfb555b-mk5j9   1/1     Running   0          5m23s
pod/etcd-7xr4c62vrp                               1/1     Running   0          4m18s
pod/etcd-p7bfjw7gq6                               1/1     Running   0          4m43s
pod/mayastor-api-rest-bcb58d479-bp5w4             1/1     Running   0          5m23s
pod/mayastor-csi-controller-6b867dd474-7mbls      3/3     Running   0          5m23s

NAME                          TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)               AGE
service/mayastor-agent-core   ClusterIP   None             <none>        50051/TCP,50052/TCP   5m24s
service/mayastor-api-rest     ClusterIP   10.152.183.71    <none>        8080/TCP,8081/TCP     5m24s
service/etcd-client           ClusterIP   10.152.183.254   <none>        2379/TCP              5m
service/etcd                  ClusterIP   None             <none>        2379/TCP,2380/TCP     4m59s

NAME                                DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR   AGE
daemonset.apps/mayastor-csi-node    3         3         3       3            3           <none>          5m24s
daemonset.apps/mayastor-io-engine   3         3         1       3            1           <none>          5m24s

NAME                                         READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/etcd-operator-mayastor       1/1     1            1           5m24s
deployment.apps/mayastor-agent-core          1/1     1            1           5m24s
deployment.apps/mayastor-operator-diskpool   1/1     1            1           5m24s
deployment.apps/mayastor-api-rest            1/1     1            1           5m24s
deployment.apps/mayastor-csi-controller      1/1     1            1           5m24s

NAME                                                    DESIRED   CURRENT   READY   AGE
replicaset.apps/etcd-operator-mayastor-8574f998bc       1         1         1       5m24s
replicaset.apps/mayastor-agent-core-f7ccf485            1         1         1       5m24s
replicaset.apps/mayastor-operator-diskpool-5b4cfb555b   1         1         1       5m24s
replicaset.apps/mayastor-api-rest-bcb58d479             1         1         1       5m23s
replicaset.apps/mayastor-csi-controller-6b867dd474      1         1         1       5m24s

I tried the patch suggested at https://github.com/canonical/microk8s-core-addons/issues/133#issuecomment-1387366772 but there was no change even after disabling and re-enabling the addon

What Should Happen Instead?

As per the docs:

In a 3-node cluster, the output should look like this:

NAME NODE STATUS CAPACITY USED AVAILABLE microk8s-m2-pool m2 Online 21449670656 0 21449670656 microk8s-m1-pool m1 Online 21449670656 0 21449670656 microk8s-m3-pool m3 Online 21449670656 0 21449670656

Reproduction Steps

Follow the instructions at https://microk8s.io/docs/addon-mayastor

Introspection Report

inspection-report-20230807_145905.tar.gz

Can you suggest a fix?

If only I had a suggestion :disappointed:

Are you interested in contributing with a fix?

Sure, if I knew what to do!

meatballs commented 1 year ago

This looks like it's a hugepages issue. Only one of the three nodes has retained its vm.nr_hugepages setting. Looking into it...

meatballs commented 1 year ago

Looks like I must have done something wrong on one of the nodes - I rebuilt it from scratch and all is fine. For the other, it's a 2GB pi and that's not enough to handle 1024 Huge Pages!