search

eclipse-che / che

Kubernetes based Cloud Development Environments for Enterprise Teams
http://eclipse.org/che
Eclipse Public License 2.0
6.96k stars 1.19k forks source link

ERR_CONNECTION_TIMED_OUT after successfull deployment on Azure #22883

Open pankan opened 6 months ago

pankan commented 6 months ago

Summary

Followed the installation guide https://eclipse.dev/che/docs/stable/administration-guide/installing-che-on-microsoft-azure/ to install on Azure, no error was faced during installation. But when I open dashboard it saysERR_CONNECTION_TIMED_OUT

Relevant information

NAME                               READY   STATUS      RESTARTS   AGE
che-dashboard-56c6fd6657-dp9qv     1/1     Running     0          5m56s
che-f5747f7fc-v66rt                1/1     Running     0          5m56s
che-gateway-bd75d65c8-hnpt4        4/4     Running     0          5m56s
che-operator-d5f7df49-k2x9b        1/1     Running     0          6m30s
che-tls-job-tps7d                  0/1     Completed   0          6m9s
plugin-registry-6468759d7d-zqhp2   1/1     Running     0          5m56s

Only thing I noticed is _acme-challenge in DNS is getting disappearing

maheshrajrp commented 6 months ago

Hi @pankan , In my case, I didn't try with Certs and Azure DNS. I used self-signed certs. Could be a problem unsure. To better assist, can we try to install with self-signed certs?

Also, are you able to access the Kubernetes Cluster with the kube-login command?

i.e Are you able to run kubectl get pods --all-namespaces ?

pankan commented 6 months ago 
$ kubectl get pods --all-namespaces
NAMESPACE                 NAME                                               READY   STATUS      RESTARTS   AGE
cert-manager              cert-manager-5f8646db6b-qm6fp                      1/1     Running     0          56m
cert-manager              cert-manager-cainjector-5cf5f57dd7-pnzpr           1/1     Running     0          56m
cert-manager              cert-manager-webhook-687b7f8b97-dbxms              1/1     Running     0          56m
devworkspace-controller   devworkspace-controller-manager-5844b45c55-cqp6z   2/2     Running     0          38m
devworkspace-controller   devworkspace-webhook-server-5689485d8d-hqvvm       2/2     Running     0          37m
eclipse-che               che-9c7977964-bwvhz                                1/1     Running     0          28m
eclipse-che               che-dashboard-56c6fd6657-dp9qv                     1/1     Running     0          36m
eclipse-che               che-gateway-bd75d65c8-hnpt4                        4/4     Running     0          36m
eclipse-che               che-operator-d5f7df49-k2x9b                        1/1     Running     0          37m
eclipse-che               che-tls-job-tps7d                                  0/1     Completed   0          36m
eclipse-che               plugin-registry-6468759d7d-zqhp2                   1/1     Running     0          36m
ingress-nginx             ingress-nginx-controller-7854678f64-dsrq9          1/1     Running     0          59m
kube-system               azure-ip-masq-agent-bqxlh                          1/1     Running     0          64m
kube-system               azure-ip-masq-agent-pff42                          1/1     Running     0          64m
kube-system               azure-ip-masq-agent-s8qzt                          1/1     Running     0          64m
kube-system               cloud-node-manager-czbzt                           1/1     Running     0          64m
kube-system               cloud-node-manager-dd2ld                           1/1     Running     0          64m
kube-system               cloud-node-manager-rh6x5                           1/1     Running     0          64m
kube-system               coredns-789789675-ccnt7                            1/1     Running     0          63m
kube-system               coredns-789789675-wpdgq                            1/1     Running     0          64m
kube-system               coredns-autoscaler-649b947bbd-p5t9j                1/1     Running     0          64m
kube-system               csi-azuredisk-node-b9nbv                           3/3     Running     0          64m
kube-system               csi-azuredisk-node-lct9p                           3/3     Running     0          64m
kube-system               csi-azuredisk-node-lthhg                           3/3     Running     0          64m
kube-system               csi-azurefile-node-5snx7                           3/3     Running     0          64m
kube-system               csi-azurefile-node-6gcpj                           3/3     Running     0          64m
kube-system               csi-azurefile-node-vvqsv                           3/3     Running     0          64m
kube-system               konnectivity-agent-64fc5d449d-59jr7                1/1     Running     0          20m
kube-system               konnectivity-agent-64fc5d449d-wbkdx                1/1     Running     0          20m
kube-system               kube-proxy-fnprg                                   1/1     Running     0          64m
kube-system               kube-proxy-nsktx                                   1/1     Running     0          64m
kube-system               kube-proxy-wkgss                                   1/1     Running     0          64m
kube-system               metrics-server-5897c748b8-cmdj4                    2/2     Running     0          63m
kube-system               metrics-server-5897c748b8-sl8bj                    2/2     Running     0          63m

Looks like issue is with certificates, Can you provide me documentation to install it with self-signed certs?

maheshrajrp commented 6 months ago

Try this.

chectl server:deploy --platform=k8s --che-operator-cr-patch-yaml=che-cluster-patch.yaml --skip-oidc-provider-check --domain=$DOMAIN_NAME

pankan commented 6 months ago 
Error: Command server:deploy failed with the error: Command failed with exit code 1: kubectl apply -f C:\ProgramData\chectl\chectl\templates\devworkspace-operator/kubernetes/combined.yaml
resource mapping not found for name: "devworkspace-controller-serving-cert" namespace: "devworkspace-controller" from "C:\\ProgramData\\chectl\\chectl\\templates\\devworkspace-operator/kubernetes/combined.yaml": no matches for kind "Certificate" in version "cert-manager.io/v1"
ensure CRDs are installed first
resource mapping not found for name: "devworkspace-controller-selfsigned-issuer" namespace: "devworkspace-controller" from "C:\\ProgramData\\chectl\\chectl\\templates\\devworkspace-operator/kubernetes/combined.yaml": no matches for kind "Issuer" in version "cert-manager.io/v1"
ensure CRDs are installed first

was getting this error for self-signed certificates, tried chectl server:deploy without --skip-cert-manager then I'm getting error

[19:01:56] Wait for Dev Workspace operator ready [started]
[19:03:08] Wait for Dev Workspace operator ready [failed]
[19:03:08] → ERR_TIMEOUT: Timeout set to pod ready timeout 60000
[19:03:08] Install Dev Workspace operator [failed]
[19:03:08] → ERR_TIMEOUT: Timeout set to pod ready timeout 60000
[19:03:08] Deploy Eclipse Che operator [failed]
[19:03:08] → ERR_TIMEOUT: Timeout set to pod ready timeout 60000
tolusha commented 6 months ago

@pankan Install cert-manager first https://cert-manager.io/docs/installation/kubectl/

pankan commented 6 months ago

@tolusha

Installation is completing but the dashboard says ERR_CONNECTION_TIMED_OUT , nothing to be seen in eclipse che pod logs

In ingress-nginx log, I see

Service "eclipse-che/che-gateway" does not have any active Endpoint.

$ kubectl logs -n ingress-nginx -l app.kubernetes.io/name=ingress-nginx

W0320 13:15:00.953181       7 controller.go:1214] Service "eclipse-che/che-gateway" does not have any active Endpoint.
I0320 13:15:00.953250       7 controller.go:190] "Configuration changes detected, backend reload required"
I0320 13:15:01.019995       7 controller.go:210] "Backend successfully reloaded"
I0320 13:15:01.020234       7 event.go:364] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-7854678f64-wmzq8", UID:"dbf8500d-1346-4f23-a3f8-e163eb0e512b", APIVersion:"v1", ResourceVersion:"8312", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration
W0320 13:15:04.286641       7 controller.go:1214] Service "eclipse-che/che-gateway" does not have any active Endpoint.
W0320 13:15:07.620104       7 controller.go:1214] Service "eclipse-che/che-gateway" does not have any active Endpoint.
W0320 13:15:10.953777       7 controller.go:1214] Service "eclipse-che/che-gateway" does not have any active Endpoint.
W0320 13:15:14.287775       7 controller.go:1214] Service "eclipse-che/che-gateway" does not have any active Endpoint.
I0320 13:15:33.368873       7 status.go:304] "updating Ingress status" namespace="eclipse-che" ingress="che" currentValue=null newValue=[{"ip":"20.235.233.85"}]
I0320 13:15:33.376455       7 event.go:364] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"eclipse-che", Name:"che", UID:"f0ca34e8-80fb-455f-a2ee-ab09cda28a99", APIVersion:"networking.k8s.io/v1", ResourceVersion:"56572", FieldPath:""}): type: 'Normal' reason: 'Sync' Scheduled for sync