Redesign client initialization and requests flow for CA-signed certificates

[sigalsax]

Is your feature request related to a problem? Please describe.

The CLI supports 2 certificate flows 1. signed 2. self-signed certificates. Self-signed cert flow - The current flow we have for initializing the client is ok and referred to as certificate pinning or trust on first use. In this flow, we save the certificate on the machine after accepting the fingerprint and each follow-up request requires us to send the pem.

Signed cert flow - For all requests, we must validate the certificate chain, the certificate hostname, etc and this is done in Python terms via verify=True . In this flow, we don’t need to save the certificate on the machine because the server sends the cert on each client request and we immediately verify it against the CA bundles that exist on the machine. Even though we don't need to save the pem on the machine in this flow we still do to abstract away the implementation details from the user and keep the UX uniform.

We should reevaluate this because currently for both these flows, we save the certificate on the machine and ask them to verify the fingerprint (pining) even though for the CA signed cert, we don't use it.

Proposed flow

1. Open socket with Conjur on init
2. Fetch the certificate
3. Evaluate if self-signed or signed by comparing if issuer == subject
4. If so, self-signed. If not, signed a. If self-signed save the pem on the machine and update the conjurrc with the path b. If signed don't save the pem on the machine and update the cert_file to '' / signed

.conjurrc for self-signed

---
account: cucumber
appliance_url: https://conjur-server
cert_file: /Users/ssax/conjur-server.pem
plugins: []

.conjurrc for signed

---
account: cucumber
appliance_url: https://conjur-server
cert_file: ''
plugins: []

Implementation details

cert_file should only tell you the path to a CA certificate to verify peer against (SSL)

1. cert_file has a value. verify=cert_file
2. cert_file has no value. verify=True . You want verify=False and so you run conjur --insecure …

[doodlesbykumbi]

I agree with the gist of the proposed flow. I'm not sure if this part is enough

Evaluate if self-signed or signed by comparing if issuer == subject ? self-signed : signed

My reasoning is that there are 3 different cases for signing, and I think the condition mentioned above doesn't take into account all 3.

1. A certificate that is signed by a trusted certificate authority (e.g. google, amazon, godaddy etc.)
2. ~Altogether self signed certificate. A certificate that is not signed by a certificate authority~ Self signed certificate
3. ~A certificate that is signed by an untrusted certificate authority (i.e. an organisation's own CA)~ Self signed CA

I think we'd want to check for (1) which maps to cert_file: '', and if not (1) then "fetch"/"get from user" the cert and set cert_file: "/path/to/cert".

The tricky part about fetching the cert is that it's possible that the server is (mis)configured to not advertise the whole chain, and so trying to use the fetched certificate for verification results in an error like SSL Error: unable to get local issuer certificate. I believe the way to circumvent this is to get the missing part of the chain to the root CA certificate (likely from a person or some other service) and use that for cert_file: "/path/to/cert". The cool thing is we can check for this after fetching the certificate by trying to verify the endpoint using the fetched certificate.

Got a reference implementation of this in Go over at https://gist.github.com/doodlesbykumbi/33130af98c94f28aabab60206206cc70.

[whip113]

@doodlesbykumbi , all certificates must be signed by a CA. All root CA certificates are, by definition, self-signed. All intermediate CA certificates are signed by either the root CA or another intermediate CA. A self-signing CA is a bit of a misnomer IMO, and in this context means only that the CA used to sign the certificate is not one that is already trusted within an org.

While true that not all web servers are configured to provide the full certificate chain, in this case since it's our web server I think we can be a little more confident in how it's configured. Now, when it comes to getting the certificate for an OCP cluster...

[sigalsax]

@doodlesbykumbi thanks for your feedback! What about the above proposal do you think does not fully answer those 3 use cases? If it's because of your third point "A certificate that is signed by an untrusted certificate authority", I grouped it together (mentally at least 🤔 ) with the self-signed flow because the flow will ultimately be the same. Because we don't have a way to verify the CA who issued the cert, we will need to perform certificate pinning, having the user verify the fingerprint, saving the certificate on the machine, and passing it with each request (just like self-signed certs). I will need to see what the proper way to validate if this is the case on initialization of the client somewhere here but the overall flow remains the same and is almost identical to the self-signed cert flow.

Regarding the suggestion to fetch the remaining part of the certificate. We ask users to dedicate a separate machine for their Conjur deployments so I agree with @whip113 on this. We can maybe return a dedicate error for this so it is more human readable but sort of think this might be not such a common usecase. Is there a reason a user would go against our recommendations for how to configure their Conjur servers?

Let me know your thoughts!

[doodlesbykumbi]

What about the above proposal do you think does not fully answer those 3 use cases?

I wasn't, and am still not, sure if issuer == subject detects (3). That was motivation for my suggestion to flip the condition and instead check for (1).

I also read somewhere (sorry, don't have the link to hand) that issuer == subject might have cases where it would be truthy for (1).

Is there a reason a user would go against our recommendations for how to configure their Conjur servers?

I think users will always surprise us. My suggestion is to be preemptive in detecting a misconfigured server when auto fetching the certificate.

[whip113]

As a general rule, @doodlesbykumbi is correct and we should expect the customer to surprise us. That said, my experience with evoke is that its not accepting of anything that doesn't follow happy path. I've seen a wide range of ways customers have botched this and things typically fall down well before we get to the point where we'd be using a CLI. That said, there can be two ways to accomplish a more predictable outcome, either by documenting a very particular way of doing this and hoping you catch all of the edge cases, or by putting up guardrails to prevent edge cases to begin with. I'd prefer we invest in the 2nd and make evoke reliably give us the environment we're expecting.

[whip113]

Also, there are edge cases where we could be using the internal signing CA and have a leaf certificate that issuer != subject. For example, if I configure the master with evoke configure master -h nate.lab, the initial conjur.pem and ca.pem will have nate.lab as both the issuer and subject. However, if I then run evoke ca issue write.nate.lab, then set that certificate as my conjur.pem, then the issuer is still nate.lab, but the subject is write.nate.lab. I might do this so that my followers can get a seed from the seed service that points to the correct master_url (another implicit thing we do which isn't a great UX). This goes back to evoke doing things behind the scenes without user input, such as creating a CA and seemingly arbitrarily using the --hostname argument as the name for the CA.

[InbalZilberman]

This bug need architect attention and guidance.