cirruslabs / orchard

Orchestrator for running Tart Virtual Machines on a cluster of Apple Silicon devices
Other
189 stars 13 forks source link

Certificate not valid for orchard-controller #148

Closed dennispost closed 8 months ago

dennispost commented 8 months ago

Hey,

when trying to create a new context I'm getting the following error:

API client failed to make a request: Get "https://MY_ORCHARD_DOMAIN:443/v1//": tls: failed to verify certificate: x509: certificate is valid for MY_ORCHARD_DOMAIN, not orchard-controller

I launch the controller with custom certificate and key: orchard controller run --controller-cert /certs/controller-cert.pem --controller-key /certs/controller-key.pem

https://github.com/cirruslabs/orchard/blob/main/internal/netconstants/netconstants.go#L11 references this hostname. In #86 the hard-coded SNI was also mentioned. How can I deploy orchard controller with my own hostname?

dennispost commented 8 months ago

Got it working 🎉

The solution in OpenShift/kubernetes was to use a reencrypt ingress, annotate the service with service.beta.openshift.io/serving-cert-secret-name to get the auto generated certificates and mount these into the controller pod.

edigaryev commented 8 months ago
API client failed to make a request: Get "https://MY_ORCHARD_DOMAIN:443/v1//": tls: failed to verify certificate: x509: certificate is valid for MY_ORCHARD_DOMAIN, not orchard-controller

This shouldn't be normally the case when using custom X.509 certificates.

Which command-line arguments did you use to invoke the orchard context create?

dennispost commented 8 months ago

I used this command: orchard context create --name test --service-account-name bootstrap-admin --service-account-token $ORCHARD_BOOTSTRAP_ADMIN_TOKEN https://MY_ORCHARD_DOMAIN:443.

It then asks to establish trust using the certificate's fingerprint of my custom certificate. After I confirm this it fail with this error.

API client failed to make a request: Get "https://MY_ORCHARD_DOMAIN:443/v1//": tls: failed to verify certificate: x509: certificate is valid for [all the wildcards of our cluster] not orchard-controller
edigaryev commented 8 months ago

It then asks to establish trust using the certificate's fingerprint of my custom certificate.

Interesting, which environment are you running this in?

I've checked the code and the only way this fallback may happen is when the host's root CA set is empty, which might be the case when running in containers, for example, in debian:latest you need to install ca-certificates package.

I agree that the fallback method should respect the certificate details, though. And we should probably let the user know that the fallback is commencing.

dennispost commented 8 months ago

It's an alpine based docker container which runs in OpenShift. ca-certificates are installed.

I tried it with https://github.com/cirruslabs/orchard/blob/main/internal/netconstants/netconstants.go#L11 changed to my hostname. Now I'm getting: tls: failed to verify certificate: x509: certificate signed by unknown authority

edigaryev commented 8 months ago

Please check out the new 0.15.0 Orchard release, it fixes the most of the issues you've reported in this thread 🚀

Now I'm getting: tls: failed to verify certificate: x509: certificate signed by unknown authority

I've just tried running orchard context create https://example.com:443/ on alpine:latest and it validated the certificate nicely, but failed to interact with the API, which is expected:

trying to connect to the controller using PKI and host's root CA set...
2023/11/22 09:00:09 API client encountered an API error to make a request: 404 Not Found

Regarding your Orchard installation, does curl https://MY_ORCHARD_DOMAIN:443/ when running in alpine based docker container work?

And does openssl s_client -connect MY_ORCHARD_DOMAIN:443 (apk add openssl) validate the certificate properly, and which certificate chain does it report?

dennispost commented 8 months ago

We have a working passthrough ingress now. Thanks for the fix!