Open tullo opened 4 years ago
The svclb-gateway-proxy pods are still Pending They never made it to Running even with firewall off
sudo k3s kubectl get all -n rio-system
NAME READY STATUS RESTARTS AGE
pod/svclb-gateway-proxy-hh8rj 0/2 Pending 0 5h59m
pod/gateway-6c496445d7-c2nxn 1/1 Running 0 5h59m
pod/gateway-proxy-7bdfc54996-tjxb6 1/1 Running 0 5h59m
pod/cert-manager-759c4847bc-fttkm 1/1 Running 0 5h59m
pod/gloo-7f984f76cc-j9lzn 1/1 Running 0 5h59m
pod/socat-rnr5m 1/1 Running 0 5h55m
pod/webhook-678fc6f47c-p6zn5 1/1 Running 0 5h55m
pod/buildkitd-798d9df44d-8c4zf 2/2 Running 0 5h55m
pod/dashboard-8698cbd7b7-8b6ws 1/1 Running 1 5h38m
pod/rio-controller-597fb9d959-9nwrm 0/1 CrashLoopBackOff 41 5h59m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/rio-api-validator ClusterIP 10.43.105.101 <none> 443/TCP 6h5m
service/gateway-proxy LoadBalancer 10.43.3.200 <pending> 80:31700/TCP,443:32442/TCP 5h59m
service/gloo ClusterIP 10.43.158.254 <none> 9977/TCP,9988/TCP,9966/TCP 5h59m
service/cert-manager ClusterIP 10.43.7.29 <none> 80/TCP 5h59m
service/cert-manager-v0 ClusterIP 10.43.87.233 <none> 80/TCP 5h59m
service/webhook ClusterIP 10.43.1.141 <none> 8090/TCP 5h56m
service/buildkitd ClusterIP 10.43.204.134 <none> 8080/TCP,80/TCP 5h55m
service/webhook-v0 ClusterIP 10.43.243.251 <none> 8090/TCP 5h55m
service/buildkitd-v0 ClusterIP 10.43.98.4 <none> 8080/TCP,80/TCP 5h55m
service/dashboard ClusterIP 10.43.124.238 <none> 80/TCP 5h38m
service/dashboard-v0 ClusterIP 10.43.33.23 <none> 80/TCP 5h38m
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/svclb-gateway-proxy 1 1 0 1 0 <none> 5h59m
daemonset.apps/socat 1 1 1 1 1 <none> 5h55m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/gateway 1/1 1 1 5h59m
deployment.apps/gateway-proxy 1/1 1 1 5h59m
deployment.apps/cert-manager 1/1 1 1 5h59m
deployment.apps/gloo 1/1 1 1 5h59m
deployment.apps/webhook 1/1 1 1 5h55m
deployment.apps/buildkitd 1/1 1 1 5h55m
deployment.apps/dashboard 1/1 1 1 5h38m
deployment.apps/rio-controller 0/1 1 0 6h5m
NAME DESIRED CURRENT READY AGE
replicaset.apps/gateway-6c496445d7 1 1 1 5h59m
replicaset.apps/gateway-proxy-7bdfc54996 1 1 1 5h59m
replicaset.apps/cert-manager-759c4847bc 1 1 1 5h59m
replicaset.apps/gloo-7f984f76cc 1 1 1 5h59m
replicaset.apps/webhook-678fc6f47c 1 1 1 5h55m
replicaset.apps/buildkitd-798d9df44d 1 1 1 5h55m
replicaset.apps/dashboard-8698cbd7b7 1 1 1 5h38m
replicaset.apps/rio-controller-597fb9d959 1 1 0 5h59m
@tullo svlb-gateway pending seems to be another issue(since you are using k3s, it by default use svclb for its own traefik ingress controller thus port will conflict). Try kubectl delete svc traefik -n kube-system
to see if this solves your issue. The real issue behind is the error Get https://10.43.0.1:443/apis/apiextensions.k8s.io/v1beta1/customresourcedefinitions: dial tcp 10.43.0.1:443: i/o timeout
, which appears when you turn on firewall. This is the kubernetes api endpoint controller will need to talk to which is blocked by firewall.
Running into the same issue, k3d/k3s on mac, attempted to run with k3d c --publish 8081:80 --publish 8082:443 --server-arg "--no-deploy=traefik"
also attempted with a "regular" install, so with traefik and deleting the service as suggested but workloads still aren't available.
Complete cluster looks like this:
rio-controller | time="2020-02-07T14:06:31Z" level=info msg="Starting rio-controller, version: v0.7.0, git commit: 4afd4901"
rio-controller | time="2020-02-07T14:06:31Z" level=info msg="Updating CRD services.rio.cattle.io"
rio-controller | time="2020-02-07T14:06:31Z" level=info msg="Updating CRD stacks.rio.cattle.io"
rio-controller | I0207 14:06:32.923118 1 leaderelection.go:241] attempting to acquire leader lease rio-system/rio...
rio-controller | time="2020-02-07T14:06:32Z" level=info msg="listening at :443"
rio-controller | I0207 14:06:32.952197 1 leaderelection.go:251] successfully acquired lease rio-system/rio
rio-controller | time="2020-02-07T14:06:34Z" level=info msg="Starting /v1, Kind=ConfigMap controller"
rio-controller | time="2020-02-07T14:06:35Z" level=info msg="Starting /v1, Kind=Endpoints controller"
rio-controller | time="2020-02-07T14:06:35Z" level=info msg="Starting apps/v1, Kind=StatefulSet controller"
rio-controller | time="2020-02-07T14:06:35Z" level=info msg="Starting apps/v1, Kind=Deployment controller"
rio-controller | time="2020-02-07T14:06:35Z" level=info msg="Starting apps/v1, Kind=DaemonSet controller"
rio-controller | time="2020-02-07T14:06:35Z" level=info msg="Starting cert-manager.io/v1alpha2, Kind=Certificate controller"
rio-controller | time="2020-02-07T14:06:35Z" level=info msg="Starting /v1, Kind=Service controller"
rio-controller | time="2020-02-07T14:06:35Z" level=info msg="Starting tekton.dev/v1alpha1, Kind=TaskRun controller"
rio-controller | time="2020-02-07T14:06:35Z" level=info msg="Starting extensions/v1beta1, Kind=Ingress controller"
rio-controller | time="2020-02-07T14:06:35Z" level=info msg="Starting /v1, Kind=Secret controller"
rio-controller | time="2020-02-07T14:06:35Z" level=info msg="Starting rbac.authorization.k8s.io/v1, Kind=ClusterRole controller"
rio-controller | time="2020-02-07T14:06:35Z" level=info msg="Starting rio.cattle.io/v1, Kind=Service controller"
rio-controller | time="2020-02-07T14:06:35Z" level=info msg="Starting rio.cattle.io/v1, Kind=Stack controller"
rio-controller | time="2020-02-07T14:06:35Z" level=info msg="Starting rio.cattle.io/v1, Kind=ExternalService controller"
rio-controller | time="2020-02-07T14:06:35Z" level=info msg="Starting admin.rio.cattle.io/v1, Kind=ClusterDomain controller"
rio-controller | time="2020-02-07T14:06:35Z" level=info msg="Starting admin.rio.cattle.io/v1, Kind=PublicDomain controller"
rio-controller | time="2020-02-07T14:06:35Z" level=info msg="Starting rbac.authorization.k8s.io/v1, Kind=ClusterRoleBinding controller"
rio-controller | time="2020-02-07T14:06:35Z" level=info msg="Starting rio.cattle.io/v1, Kind=Router controller"
rio-controller | time="2020-02-07T14:06:35Z" level=info msg="Starting gitwatcher.cattle.io/v1, Kind=GitCommit controller"
rio-controller | time="2020-02-07T14:06:35Z" level=info msg="Starting gloo.solo.io/v1, Kind=Settings controller"
rio-controller | E0207 14:06:45.202967 1 controller.go:135] error syncing 'default/competent-johnson-v0gcx87': handler service-build: failed to update default/competent-johnson-v0gcx87-ee709-4e40c tekton.dev/v1alpha1, Kind=TaskRun for service-build default/competent-johnson-v0gcx87: Internal error occurred: failed calling webhook "webhook.tekton.dev": Post https://tekton-pipelines-webhook.tekton-pipelines.svc:443/?timeout=27s: dial tcp 10.43.64.249:443: connect: no route to host, handler template: skip processing, requeuing
rio info
Rio Version: v0.7.0 (4afd4901)
Rio CLI Version: v0.7.0 (4afd4901)
Cluster Domain: aiaai1.on-rio.io
Cluster Domain IPs: 172.19.0.2
System Namespace: rio-system
Wildcard certificates: aiaai1.on-rio.io(true)
@krishofmans looks like you need to push 80 and 443 if you are using k3d.
@StrongMonkey the trick that worked for me was installing rio like this on my k3s docker rio install --ip-address=127.0.0.1
before it tried to resolve something in the docker ip, which it could not reach from inside the docker container. If it would have been a native binary k3s it would have worked probably.
yes, if you are on k3d for mac that's the only workaround you can use because of networking problems. K3s binary on linux won't have such issue
Describe the bug
rio install
gets stuck with active firewall (ufw)To Reproduce Steps to reproduce the behavior:
rio install
on fresh ubuntu box (19.10)Expected behavior Installation does not hang and k3s deployments do not crash
Kubernetes version & type (GKE, on-prem):
kubectl version
(cloud server)Type: Rio version:
rio info
Additional context
rio system logs
output: