Closed satishweb closed 10 months ago
Hello @satishweb ! Thanks for creating this issue, when using an ingress without SSL-passthrough, you need to make sure the ingress is signing the certificate not only for itself but also for the vcluster, in your case it seems it returns the default ingress certificate which is insecure by default. Also you will need to use a service account token as described here: https://www.vcluster.com/docs/operator/external-access#ingress-without-ssl-passthrough
hi @FabianKramm , Thanks for the quick response. I am using SSL Passthrough. Below is the error that I get when I use kubectl:
Unable to connect to the server: x509: certificate is valid for ingress.local, not my.domain.com
If you think that this error happens only when SSL passthrough is not enabled, I can redo the test to confirm it. Please let me know.
@satishweb there is probably a problem with the SSL passthrough then, what ingress controller are you using? This error message shows thats the ingress certificate (hence it says ingress.local) which means that the ingress controller is doing TLS termination and does not passthrough.
Heyaa, @satishweb 👋
Have you, by any chance, managed to check if your ingress certificate is validated on your desired domain?
Thanks! Waiting on your confirmation before closing this issue 🌞
I haven't got the time to test this yet. I have added this to my personal to-do list. I will respond as soon as I am done with the testing. For now, I am using the --insecure
flag for vcluster cli.
I got the same issue
Deploy:
$ helm upgrade --install my-vcluster vcluster \
--values tenant/virtual_cluster_provider/vcluster/chart/values.yaml \
--repo https://charts.loft.sh/ \
--namespace my-vcluster \
--repository-config=''
values.yaml
ingress:
enabled: true
host: my-vcluster.account.local
isolation:
enabled: true
syncer:
extraArgs:
- --out-kube-config-server=https://my-vcluster.account.local/
I used the following secret
$ kubectl get secret vc-my-vcluster -n my-vcluster --template={{.data.config}} | base64 --decode
Error
$ kubectl get pods
Unable to connect to the server: x509: certificate is valid for ingress.local, not my-vcluster.account.local
Here is the command that works right now (not ideal but works): vcluster connect --insecure my-vcluster --server my-tls-san-fqdn-address.domain.com --service-account=kube-system/developer --cluster-role=cluster-admin
Hi, could everyone please add the details of the ingress controller they are using? Thanks! @satishweb @eldad87
Just to be super clear: "Enabling SSL Passthrough" also means passing a custom flag to the nginx ingress controller. This may differ for other controllers.
Hi, could everyone please add the details of the ingress controller they are using? Thanks! @satishweb @eldad87
Nginx ingress controller with pass through enabled.
hi All,
I try to use SSL Passthrough in my ingress, but still got this _Unable to connect to the server: tls: failed to verify certificate: x509: certificate is valid for ingress.local, not cluster1.co.id_If I disable insecure then this is working, i can access my cluster. But this is something that my internal dont want.
Therefore, i want to change ingress.local certificate, and replace with cluster1.ebays.co.id. Is it possible?
Therefore, i want to change ingress.local certificate, and replace with cluster1.ebays.co.id. Is it possible?
please see the --tls-sans
flag here: https://www.vcluster.com/docs/using-vclusters/access
Hi @rohantmp
I've tried to put following values into values.yml.
syncer:
extraArgs:
- --tls-san=cluster1.co.id
Then i create cluster using following script
vcluster create $NAME -n $NS --connect=false --upgrade -f values.yml
but still say the same....still lookup for ingress.local
Hey, this might mean that you haven't successfully enabled ingress passthrough. Could explain the steps you tried for this?
if I disable insecure then this is working, i can access my cluster.
What does it mean to "disable insecure"?
Hey, this might mean that you haven't successfully enabled ingress passthrough. Could explain the steps you tried for this?
if I disable insecure then this is working, i can access my cluster.
What does it mean to "disable insecure"?
Sorry, i mean enable insecure.
I faced the same issue and I am still using ssl passthrough with insecure flag set as false. I did not test the new version though.
I faced the same issue and I am still using ssl passthrough with insecure flag set as false. I did not test the new version though.
Which version are you using right now? I'm using
root@test-for-bq:~# vcluster version
vcluster version 0.17.1
I am still on a very old version, based on your comments, for the ssl-passthrough option, there seems to be no much difference between current version and this version: 0.13.0
To follow up on what @rohantmp is saying, this means modifying the nginx configuration and adding a flag to the container args:
spec:
containers:
- args:
- /nginx-ingress-controller
- --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
- --election-id=ingress-nginx-leader
- --controller-class=k8s.io/ingress-nginx
- --ingress-class=nginx
- --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
- --validating-webhook=:8443
- --validating-webhook-certificate=/usr/local/certificates/cert
- --validating-webhook-key=/usr/local/certificates/key
- --enable-ssl-passthrough <<<<<<
This seems to be a configuration issue, so I'm closing this for now. Feel free to re-open if you believe that was a mistake :)
This is still happening on 0.19.7
, I had a single syncer v0.10.2
cluster, which I torn down, then recreated with the latest vcluster binary, but using a syncer.replicas=3
on the values.yaml, and now using the same ingress to reach the vcluster k8s admin returns this error if I try to connect via kubeconfig or plain vcluster connect
, both fail, excelt that the kubectl --kubeconfig
works one out of every three times which leads me to belive the HA mode, has a problem.
I have ssl-passthrough, I guarantee it as this cluster was working fine on v0.10.2
Here are three subsequent tries, only 2 succeeded
➜ kubectl --kubeconfig ./kubeconfig.yaml get namespaces
NAME STATUS AGE
kube-system Active 18m
kube-public Active 18m
kube-node-lease Active 18m
default Active 18m
dev-vcluster git:(master) ✗
➜ kubectl --kubeconfig ./kubeconfig.yaml get namespaces
Unable to connect to the server: tls: failed to verify certificate: x509: certificate signed by unknown authority
dev-vcluster git:(master) ✗
➜ kubectl --kubeconfig ./kubeconfig.yaml get namespaces
NAME STATUS AGE
kube-system Active 18m
kube-public Active 18m
kube-node-lease Active 18m
default Active 18m
again, nothing changed in my host cluster between version changes
What happened?
I am facing an issue with vcluster helm chart version
0.13.0
and Argo CD. I tried to add cluster to ArgoCD using kubeconfig and it errored out.I spent some time debugging this. So far, I think that vcluster/k3s is not adding custom domain in the certificate SAN list.
What did you expect to happen?
ArgoCD certificate verification should be successful with custom domain name that is known to vcluster via --tls-san argument
How can we reproduce it (as minimally and precisely as possible)?
Vcluster helm chart values.yaml
vcluster args
syncer args
vcluster cli error
sts logs
Anything else we need to know?
Argocd secret for cluster
Note: This was generated based on the vcluster secret that contains the kubeconfig
Host cluster Kubernetes version
Host cluster Kubernetes distribution
vlcuster version
Vcluster Kubernetes distribution(k3s(default)), k8s, k0s)
OS and Arch