Kubesphere install on existed kubernetes failed

[rupang790]

I've been trying to install KubeSphere on existed kubernetes. My Kubernetes information below:

• kubernetes version: v1.22.8
• Container Runtime: CRI-O
• Cluster Environment: Bare-Metal (3Masters VMs / 3 HA ETCD VMs / 1 Bare-Metal Worker)
• Kubesphere Version: v3.2.1

On my environment, I tried to install KubeSphere by minimal package for managing multi-clusters as process below:

$ kubectl apply -f https://github.com/kubesphere/ks-installer/releases/download/v3.2.1/kubesphere-installer.yaml

$ kubectl apply -f https://github.com/kubesphere/ks-installer/releases/download/v3.2.1/cluster-configuration.yaml

After applying, error occurred.( Logging KubeSphere pod)

$ kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l app=ks-install -o jsonpath='{.items[0].metadata.name}') -f
...

failed: [localhost] (item={'ns': 'kubesphere-system', 'kind': 'users.iam.kubesphere.io', 'resource': 'admin', 'release': 'ks-core'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "/usr/local/bin/kubectl -n kubesphere-system annotate --overwrite users.iam.kubesphere.io admin meta.helm.sh/release-name=ks-core && /usr/local/bin/kubectl -n kubesphere-system annotate --overwrite users.iam.kubesphere.io admin meta.helm.sh/release-namespace=kubesphere-system && /usr/local/bin/kubectl -n kubesphere-system label --overwrite users.iam.kubesphere.io admin app.kubernetes.io/managed-by=Helm\n", "delta": "0:00:00.250663", "end": "2022-05-31 15:30:35.524006", "failed_when_result": true, "item": {"kind": "users.iam.kubesphere.io", "ns": "kubesphere-system", "release": "ks-core", "resource": "admin"}, "msg": "non-zero return code", "rc": 1, "start": "2022-05-31 15:30:35.273343", "stderr": "Error from server (InternalError): Internal error occurred: failed calling webhook \"users.iam.kubesphere.io\": Post \"https://ks-controller-manager.kubesphere-system.svc:443/validate-email-iam-kubesphere-io-v1alpha2?timeout=30s\": service \"ks-controller-manager\" not found", "stderr_lines": ["Error from server (InternalError): Internal error occurred: failed calling webhook \"users.iam.kubesphere.io\": Post \"https://ks-controller-manager.kubesphere-system.svc:443/validate-email-iam-kubesphere-io-v1alpha2?timeout=30s\": service \"ks-controller-manager\" not found"], "stdout": "", "stdout_lines": []}

PLAY RECAP *********************************************************************
localhost                  : ok=15   changed=9    unreachable=0    failed=1    skipped=6    rescued=0    ignored=0

I tried to search this issue, but I could not find. Could you please check this issue?

[emoxam]

got the same error. it worked, but i need the devops addon consider the manual https://kubesphere.io/docs/v3.3/pluggable-components/devops/ editing CDR didn't helped me, so i download wget https://github.com/kubesphere/ks-installer/releases/download/v3.3.1/cluster-configuration.yaml edited it and apply. I found it of lack of PVs - created needed, minio and prometeus started, but no DEVOPS tabs anyway. After a while web stopped worked and i got the same error as rupang790 How ti fix it ?

[emoxam]

Thats the fix i think kubectl delete validatingwebhookconfigurations.admissionregistration.k8s.io users.iam.kubesphere.io

But then i get another problem

k logs ks-apiserver-567fcb899d-p6gbp -n kubesphere-system

W1209 10:26:16.687725       1 client_config.go:615] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
W1209 10:26:16.689698       1 client_config.go:615] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
W1209 10:26:16.701470       1 options.go:191] ks-apiserver starts without redis provided, it will use in memory cache. This may cause inconsistencies when running ks-apiserver with multiple replicas.
I1209 10:26:16.701523       1 interface.go:50] start helm repo informer
I1209 10:26:17.272025       1 apiserver.go:417] Start cache objects
Error: failed to fetch group version resources devops.kubesphere.io/v1alpha3: the server could not find the requested resource
2022/12/09 10:26:17 failed to fetch group version resources devops.kubesphere.io/v1alpha3: the server could not find the requested resource

[halinwu]

I have same issue when I deployed ks at kubernetes (v1.21.6) .
failed: [localhost] (item={'ns': 'kubesphere-system', 'kind': 'users.iam.kubesphere.io', 'resource': 'admin', 'release': 'ks-core'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "/usr/local/bin/kubectl -n kubesphere-system annotate --overwrite users.iam.kubesphere. io admin meta.helm.sh/release-name=ks-core && /usr/local/bin/kubectl -n kubesphere-system annotate --overwrite users.iam.kubesphere.io admin meta.helm.sh/release-namespace=kubesphere-system && /usr/local/bin/kubectl -n kubesphere-system label --overwrite users.iam.kubesphere. io admin app.kubernetes.io/managed-by=Helm\n", "delta": "0:00:00.802839", "end": "2023-01-12 06:04:22.532670", "failed_when_result": true, "item": {"kind": "users.iam.kubesphere.io", "ns": "kubesphere-system", "release": "ks-core", "resource": "admin"}, "msg": "non-zero retur n code", "rc": 1, "start": "2023-01-12 06:04:21.729831", "stderr": "Error from server (InternalError): Internal error occurred: failed calling webhook \"users.iam.kubesphere.io\": Post \"https://ks-controller-manager.kubesphere-system.svc:443/validate-email-iam-kubesphere-io- v1alpha2?timeout=30s\": Service Unavailable", "stderr_lines": ["Error from server (InternalError): Internal error occurred: failed calling webhook \"users.iam.kubesphere.io\": Post \"https://ks-controller-manager.kubesphere-system.svc:443/validate-email-iam-kubesphere-io-v1al pha2?timeout=30s\": Service Unavailable"], "stdout": "", "stdout_lines": []}