search

openebs-archive / cstor-operators

Collection of OpenEBS cStor Data Engine Operators
https://openebs.io
Apache License 2.0
93 stars 69 forks source link

Cannot install cstor on Microk8s #394

Open RafaelMoreira1180778 opened 2 years ago

RafaelMoreira1180778 commented 2 years ago

Microk8s information:

microk8s is running
high-availability: no
  datastore master nodes: 127.0.0.1:19001
  datastore standby nodes: none
addons:
  enabled:
    dashboard            # The Kubernetes dashboard
    dns                  # CoreDNS
    ha-cluster           # Configure high availability on the current node
    helm                 # Helm 2 - the package manager for Kubernetes
    helm3                # Helm 3 - Kubernetes package manager
    ingress              # Ingress controller for external access
    metrics-server       # K8s Metrics Server for API access to service metrics
    portainer            # Portainer UI for your Kubernetes cluster
    storage              # Storage class; allocates storage from host directory
    openebs              # OpenEBS is the open-source storage solution for Kubernetes
  disabled:
    ambassador           # Ambassador API Gateway and Ingress
    cilium               # SDN, fast with full network policy
    fluentd              # Elasticsearch-Fluentd-Kibana logging and monitoring
    gpu                  # Automatic enablement of Nvidia CUDA
    host-access          # Allow Pods connecting to Host services smoothly
    istio                # Core Istio service mesh services
    jaeger               # Kubernetes Jaeger operator with its simple config
    keda                 # Kubernetes-based Event Driven Autoscaling
    knative              # The Knative framework on Kubernetes.
    kubeflow             # Kubeflow for easy ML deployments
    linkerd              # Linkerd is a service mesh for Kubernetes and other frameworks
    metallb              # Loadbalancer for your Kubernetes cluster
    multus               # Multus CNI enables attaching multiple network interfaces to pods
    openfaas             # openfaas serverless framework
    prometheus           # Prometheus operator for monitoring and logging
    rbac                 # Role-Based Access Control for authorisation
    registry             # Private image registry exposed on localhost:32000
    traefik              # traefik Ingress controller for external access

The command I am running:

microk8s helm3 install openebs-cstor openebs-cstor/cstor -n openebs --set-string csiNode.kubeletDir="/var/snap/microk8s/common/var/lib/kubelet/" --set openebsNDM.enabled=false

WARNING: Kubernetes configuration file is group-readable. This is insecure. Location: /var/snap/microk8s/2487/credentials/client.config
Error: failed to download "openebs-cstor/cstor" (hint: running `helm repo update` may help)

Is this an error on my side? I am following this documentation.

RafaelMoreira1180778 commented 2 years ago

I just tried to install using Operator and git this output:

microk8s kubectl apply -f https://openebs.github.io/charts/microk8s-cstor-operator.yaml

namespace/openebs unchanged
serviceaccount/openebs-cstor-operator unchanged
clusterrole.rbac.authorization.k8s.io/openebs-cstor-operator unchanged
clusterrolebinding.rbac.authorization.k8s.io/openebs-cstor-operator unchanged
clusterrole.rbac.authorization.k8s.io/openebs-cstor-migration unchanged
clusterrolebinding.rbac.authorization.k8s.io/openebs-cstor-migration unchanged
customresourcedefinition.apiextensions.k8s.io/cstorbackups.cstor.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/cstorcompletedbackups.cstor.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/cstorpoolclusters.cstor.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/cstorpoolinstances.cstor.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/cstorrestores.cstor.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/cstorvolumeconfigs.cstor.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/cstorvolumepolicies.cstor.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/cstorvolumereplicas.cstor.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/cstorvolumes.cstor.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/upgradetasks.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/migrationtasks.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/cstorvolumeattachments.cstor.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/volumesnapshotclasses.snapshot.storage.k8s.io configured
customresourcedefinition.apiextensions.k8s.io/volumesnapshotcontents.snapshot.storage.k8s.io configured
customresourcedefinition.apiextensions.k8s.io/volumesnapshots.snapshot.storage.k8s.io configured
csidriver.storage.k8s.io/cstor.csi.openebs.io unchanged
clusterrolebinding.rbac.authorization.k8s.io/openebs-cstor-csi-snapshotter-binding unchanged
clusterrole.rbac.authorization.k8s.io/openebs-cstor-csi-snapshotter-role unchanged
serviceaccount/openebs-cstor-csi-controller-sa unchanged
clusterrole.rbac.authorization.k8s.io/openebs-cstor-csi-provisioner-role unchanged
clusterrolebinding.rbac.authorization.k8s.io/openebs-cstor-csi-provisioner-binding unchanged
priorityclass.scheduling.k8s.io/openebs-cstor-csi-controller-critical configured
priorityclass.scheduling.k8s.io/openebs-cstor-csi-node-critical configured
statefulset.apps/openebs-cstor-csi-controller unchanged
clusterrole.rbac.authorization.k8s.io/openebs-cstor-csi-attacher-role unchanged
clusterrolebinding.rbac.authorization.k8s.io/openebs-cstor-csi-attacher-binding unchanged
clusterrole.rbac.authorization.k8s.io/openebs-cstor-csi-cluster-registrar-role unchanged
clusterrolebinding.rbac.authorization.k8s.io/openebs-cstor-csi-cluster-registrar-binding unchanged
serviceaccount/openebs-cstor-csi-node-sa unchanged
clusterrole.rbac.authorization.k8s.io/openebs-cstor-csi-registrar-role unchanged
clusterrolebinding.rbac.authorization.k8s.io/openebs-cstor-csi-registrar-binding unchanged
configmap/openebs-cstor-csi-iscsiadm unchanged
daemonset.apps/openebs-cstor-csi-node unchanged
deployment.apps/cspc-operator unchanged
deployment.apps/cvc-operator unchanged
service/cvc-operator-service unchanged
deployment.apps/openebs-cstor-admission-server configured
customresourcedefinition.apiextensions.k8s.io/blockdevices.openebs.io configured
customresourcedefinition.apiextensions.k8s.io/blockdeviceclaims.openebs.io configured
configmap/openebs-ndm-config unchanged
Warning: resource daemonsets/openebs-ndm is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
Warning: resource deployments/openebs-ndm-operator is missing the kubectl.kubernetes.io/last-applied-configuration annotation which is required by kubectl apply. kubectl apply should only be used on resources created declaratively by either kubectl create --save-config or kubectl apply. The missing annotation will be patched automatically.
deployment.apps/openebs-ndm-cluster-exporter unchanged
service/openebs-ndm-cluster-exporter-service unchanged
daemonset.apps/openebs-ndm-node-exporter unchanged
service/openebs-ndm-node-exporter-service unchanged
Error from server (Invalid): error when applying patch:
{"metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apps/v1\",\"kind\":\"DaemonSet\",\"metadata\":{\"annotations\":{},\"labels\":{\"name\":\"openebs-ndm\",\"openebs.io/component-name\":\"ndm\",\"openebs.io/version\":\"3.0.0\"},\"name\":\"openebs-ndm\",\"namespace\":\"openebs\"},\"spec\":{\"selector\":{\"matchLabels\":{\"name\":\"openebs-ndm\",\"openebs.io/component-name\":\"ndm\"}},\"template\":{\"metadata\":{\"labels\":{\"name\":\"openebs-ndm\",\"openebs.io/component-name\":\"ndm\",\"openebs.io/version\":\"3.0.0\"}},\"spec\":{\"containers\":[{\"args\":[\"-v=4\",\"--feature-gates=\\\"GPTBasedUUID\\\"\"],\"env\":[{\"name\":\"NAMESPACE\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"metadata.namespace\"}}},{\"name\":\"NODE_NAME\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"spec.nodeName\"}}},{\"name\":\"SPARSE_FILE_DIR\",\"value\":\"/var/openebs/sparse\"},{\"name\":\"SPARSE_FILE_SIZE\",\"value\":\"10737418240\"},{\"name\":\"SPARSE_FILE_COUNT\",\"value\":\"0\"}],\"image\":\"openebs/node-disk-manager:1.7.0\",\"imagePullPolicy\":\"IfNotPresent\",\"livenessProbe\":{\"exec\":{\"command\":[\"pgrep\",\"ndm\"]},\"initialDelaySeconds\":30,\"periodSeconds\":60},\"name\":\"node-disk-manager\",\"securityContext\":{\"privileged\":true},\"volumeMounts\":[{\"mountPath\":\"/host/node-disk-manager.config\",\"name\":\"config\",\"readOnly\":true,\"subPath\":\"node-disk-manager.config\"},{\"mountPath\":\"/run/udev\",\"name\":\"udev\"},{\"mountPath\":\"/host/proc\",\"name\":\"procmount\",\"readOnly\":true},{\"mountPath\":\"/dev\",\"name\":\"devmount\"},{\"mountPath\":\"/var/openebs/ndm\",\"name\":\"basepath\"},{\"mountPath\":\"/var/openebs/sparse\",\"name\":\"sparsepath\"}]}],\"hostNetwork\":true,\"serviceAccountName\":\"openebs-cstor-operator\",\"volumes\":[{\"configMap\":{\"name\":\"openebs-ndm-config\"},\"name\":\"config\"},{\"hostPath\":{\"path\":\"/run/udev\",\"type\":\"Directory\"},\"name\":\"udev\"},{\"hostPath\":{\"path\":\"/proc\",\"type\":\"Directory\"},\"name\":\"procmount\"},{\"hostPath\":{\"path\":\"/dev\",\"type\":\"Directory\"},\"name\":\"devmount\"},{\"hostPath\":{\"path\":\"/var/openebs/ndm\",\"type\":\"DirectoryOrCreate\"},\"name\":\"basepath\"},{\"hostPath\":{\"path\":\"/var/openebs/sparse\"},\"name\":\"sparsepath\"}]}},\"updateStrategy\":{\"type\":\"RollingUpdate\"}}}\n"},"labels":{"name":"openebs-ndm"}},"spec":{"selector":{"matchLabels":{"name":"openebs-ndm","openebs.io/component-name":"ndm"}},"template":{"spec":{"$setElementOrder/containers":[{"name":"node-disk-manager"}],"$setElementOrder/volumes":[{"name":"config"},{"name":"udev"},{"name":"procmount"},{"name":"devmount"},{"name":"basepath"},{"name":"sparsepath"}],"containers":[{"args":["-v=4","--feature-gates=\"GPTBasedUUID\""],"env":[{"name":"NAMESPACE","valueFrom":{"fieldRef":{"fieldPath":"metadata.namespace"}}},{"name":"NODE_NAME","valueFrom":{"fieldRef":{"fieldPath":"spec.nodeName"}}},{"name":"SPARSE_FILE_DIR","value":"/var/openebs/sparse"},{"name":"SPARSE_FILE_SIZE","value":"10737418240"},{"name":"SPARSE_FILE_COUNT","value":"0"}],"image":"openebs/node-disk-manager:1.7.0","imagePullPolicy":"IfNotPresent","livenessProbe":{"exec":{"command":["pgrep","ndm"]},"initialDelaySeconds":30,"periodSeconds":60},"name":"node-disk-manager","securityContext":{"privileged":true},"volumeMounts":[{"mountPath":"/host/node-disk-manager.config","name":"config","readOnly":true,"subPath":"node-disk-manager.config"},{"mountPath":"/run/udev","name":"udev"},{"mountPath":"/host/proc","name":"procmount","readOnly":true},{"mountPath":"/dev","name":"devmount"},{"mountPath":"/var/openebs/ndm","name":"basepath"},{"mountPath":"/var/openebs/sparse","name":"sparsepath"}]}],"serviceAccountName":"openebs-cstor-operator","volumes":[{"$retainKeys":["hostPath","name"],"hostPath":{"path":"/var/openebs/ndm"},"name":"basepath"},{"$retainKeys":["hostPath","name"],"hostPath":{"path":"/var/openebs/sparse"},"name":"sparsepath"}]}}}}
to:
Resource: "apps/v1, Resource=daemonsets", GroupVersionKind: "apps/v1, Kind=DaemonSet"
Name: "openebs-ndm", Namespace: "openebs"
for: "https://openebs.github.io/charts/microk8s-cstor-operator.yaml": DaemonSet.apps "openebs-ndm" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app":"openebs", "component":"ndm", "name":"openebs-ndm", "openebs.io/component-name":"ndm", "release":"openebs"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
Error from server (Invalid): error when applying patch:
{"metadata":{"annotations":{"kubectl.kubernetes.io/last-applied-configuration":"{\"apiVersion\":\"apps/v1\",\"kind\":\"Deployment\",\"metadata\":{\"annotations\":{},\"labels\":{\"name\":\"openebs-ndm-operator\",\"openebs.io/component-name\":\"ndm-operator\",\"openebs.io/version\":\"3.0.0\"},\"name\":\"openebs-ndm-operator\",\"namespace\":\"openebs\"},\"spec\":{\"replicas\":1,\"selector\":{\"matchLabels\":{\"name\":\"openebs-ndm-operator\",\"openebs.io/component-name\":\"ndm-operator\"}},\"strategy\":{\"type\":\"Recreate\"},\"template\":{\"metadata\":{\"labels\":{\"name\":\"openebs-ndm-operator\",\"openebs.io/component-name\":\"ndm-operator\",\"openebs.io/version\":\"3.0.0\"}},\"spec\":{\"containers\":[{\"env\":[{\"name\":\"WATCH_NAMESPACE\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"metadata.namespace\"}}},{\"name\":\"POD_NAME\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"metadata.name\"}}},{\"name\":\"SERVICE_ACCOUNT\",\"valueFrom\":{\"fieldRef\":{\"fieldPath\":\"spec.serviceAccountName\"}}},{\"name\":\"OPERATOR_NAME\",\"value\":\"node-disk-operator\"},{\"name\":\"CLEANUP_JOB_IMAGE\",\"value\":\"openebs/linux-utils:3.0.0\"}],\"image\":\"openebs/node-disk-operator:1.7.0\",\"imagePullPolicy\":\"IfNotPresent\",\"livenessProbe\":{\"httpGet\":{\"path\":\"/healthz\",\"port\":8585},\"initialDelaySeconds\":15,\"periodSeconds\":20},\"name\":\"node-disk-operator\",\"readinessProbe\":{\"httpGet\":{\"path\":\"/readyz\",\"port\":8585},\"initialDelaySeconds\":5,\"periodSeconds\":10}}],\"serviceAccountName\":\"openebs-cstor-operator\"}}}}\n"},"labels":{"name":"openebs-ndm-operator"}},"spec":{"selector":{"matchLabels":{"name":"openebs-ndm-operator","openebs.io/component-name":"ndm-operator"}},"template":{"metadata":{"labels":{"name":"openebs-ndm-operator"}},"spec":{"$setElementOrder/containers":[{"name":"node-disk-operator"}],"containers":[{"env":[{"name":"WATCH_NAMESPACE","valueFrom":{"fieldRef":{"fieldPath":"metadata.namespace"}}},{"name":"POD_NAME","valueFrom":{"fieldRef":{"fieldPath":"metadata.name"}}},{"name":"SERVICE_ACCOUNT","valueFrom":{"fieldRef":{"fieldPath":"spec.serviceAccountName"}}},{"name":"OPERATOR_NAME","value":"node-disk-operator"},{"name":"CLEANUP_JOB_IMAGE","value":"openebs/linux-utils:3.0.0"}],"image":"openebs/node-disk-operator:1.7.0","imagePullPolicy":"IfNotPresent","livenessProbe":{"httpGet":{"path":"/healthz","port":8585},"initialDelaySeconds":15,"periodSeconds":20},"name":"node-disk-operator","readinessProbe":{"httpGet":{"path":"/readyz","port":8585},"initialDelaySeconds":5,"periodSeconds":10}}],"serviceAccountName":"openebs-cstor-operator"}}}}
to:
Resource: "apps/v1, Resource=deployments", GroupVersionKind: "apps/v1, Kind=Deployment"
Name: "openebs-ndm-operator", Namespace: "openebs"
for: "https://openebs.github.io/charts/microk8s-cstor-operator.yaml": Deployment.apps "openebs-ndm-operator" is invalid: spec.selector: Invalid value: v1.LabelSelector{MatchLabels:map[string]string{"app":"openebs", "name":"openebs-ndm-operator", "openebs.io/component-name":"ndm-operator", "release":"openebs"}, MatchExpressions:[]v1.LabelSelectorRequirement(nil)}: field is immutable
prateekpandey14 commented 2 years ago

@RafaelMoreira1180778 did you already installed the openebs operator ? looks like you already have NDM components running in cluster, and you have to skip the installation of NDM deployments in cstor-operator.yaml file.