Closed rochecompaan closed 12 months ago
rochecompaan
commented
12 months ago It looks like the Mayastor operator is trying to provision a disk pool using the openebs dynamic-localpv-provisioner/ as per this comment. I imagine all the node prep required for OpenEBS volumes would be required before you can install Mayastor in a Talos cluster.
smira
commented
12 months ago Talos by default follows Kubernetes best security practices, so it enables Pod Security. The Mayastor deployment doesn't mark itself with a proper policy, so this is not a bug in Talos, but rather might be seen as a bug in the Helm chart.
Long story short, you can label the namespace with the proper label as Mayastor is definitely privileged:
$ kubectl label ns mayastor pod-security.kubernetes.io/enforce=privilegedThis is not a Talos bug per sé, but the Talos documentation does not have sufficient node prep instructions for the current release of Mayastor working. As per the comment I linked to, there is a hard dependency on the OpenEBS localpv provisioner, and this requires additional node prep. It would be helpful to add this to the docs.
smira
commented
12 months ago @rochecompaan please submit a PR with fixes, documentation always lags behind!
rochecompaan
commented
12 months ago @smira I'm more than happy to. I'll confirm the exact requirements locally and update the docs accordingly.
Mohitsharma44
commented
11 months ago @rochecompaan, did you get a chance to test out mayastor's Dynamic Local Persistent Volume (LocalPV) provisioner?
rochecompaan
commented
11 months ago I ran into more issues when I continued testing and ran out of time. I mean to return to it once I have time again. Once I managed to install the openebs dependency I realized that openebs has a more than adequate local storage solution, so I wasn't pressed to make Mayastor work anymore. Unfortunately, I can't remember what issues I ran into, they might not even be Mayastor specific.
Mohitsharma44
commented
11 months ago Fair enough. I had sometime this evening so tried to make Mayastor work. The following is what I did (in case its helpful to others):
hostpathClass.basePath. This should be the same path that we pass to Kubelet via extraMounts.
helm upgrade --install --create-namespace --namespace mayastor --version 2.4.0 mayastor mayastor/mayastor \
# --set etcd.persistence.storageClass="mayastor-etcd-localpv" \
# --set loki-stack.loki.persistence.storageClassName="mayastor-loki-localpv" \
--set etcd.localpvScConfig.basePath="/var/openebs/local/{{ .Release.Name }}/etcd" \
--set loki-stack.localpvScConfig.basePath="/var/openebs/local/{{ .Release.Name }}/loki"
phiilu
commented
11 months ago Thanks for the steps @Mohitsharma44
However, it seems like Mayastor is not supporting arm64, which is a bummer.
I tried to install the 2.5.0 helm chart release, but the etcd version they depend on is outdated and uses an older version of bitnami-shell which is not built for arm64:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 16m default-scheduler Successfully assigned mayastor/mayastor-etcd-0 to n2-storage
Normal Pulled 15m (x5 over 16m) kubelet Container image "docker.io/bitnami/bitnami-shell:11-debian-11-r63" already present on machine
Normal Created 15m (x5 over 16m) kubelet Created container volume-permissions
Normal Started 15m (x5 over 16m) kubelet Started container volume-permissions
Warning BackOff 103s (x70 over 16m) kubelet Back-off restarting failed container volume-permissions in pod mayastor-etcd-0_mayastor(52798499-a0d7-42ef-acdd-5519341ed07f) ➜ ~ kubectl logs pods/mayastor-etcd-0 -c volume-permissions
exec /bin/bash: exec format error
Bug report
Mayastor fails to install, presumably because it violates the PodSecurity policy.
Description
Logs
Response after helm install
Pods status after installation:
Environment