New TLS cert needed

[c0c0n3]

Describe the bug

The current TLS certificate we've got for kitt4sme.collab-cloud.eu is expiring on May the 2nd, 2023. Since review day is May 12th, we've got to replace it with a fresh one ASAP. Also, the current cert is for *.collab-cloud.eu and it might be better to have one more specific, i.e. for kitt4sme.collab-cloud.eu, if other unrelated sites run on collab cloud. Finally, the CA that signed it (SSL.com RSA SSL subCA) is an intermediate CA that's not always included in popular trust stores. This means some clients might fail to connect to kitt4sme.collab-cloud.eu over HTTPs.

To Reproduce

Steps to reproduce the behavior:

1. Browse to GeoCerts online certificate validator: https://www.geocerts.com/ssl-checker
2. Enter kitt4sme.collab-cloud.eu and 443 in the URL and port fields, respectively.
3. Hit the "Check SSL" button.
4. Eyeball the report. In particular, you should see an expiry date of 02 May 2023 and a warning about the validator not being able to chain the certificate to a valid, trusted CA root.
5. Run the same validation at https://www.digicert.com/help/ and https://www.sslshopper.com/SSL-CHECKER.HTML#hostname=kitt4sme.collab-cloud.eu. You should get similar reports.

To see how this could be a problem for clients trying to connect to kitt4sme.collab-cloud.eu, get a shell on the Argo CD's server pod. It's an Ubuntu 22.04.2 LTS image which doesn't include "SSL.com RSA SSL subCA" in its trust store. Then use openssl to connect to kitt4sme.collab-cloud.eu:443 and see validation fail.

$ kubectl -n argocd exec -it svc/argocd-server -- sh

$ cat /etc/issue
Ubuntu 22.04.2 LTS

$ openssl s_client -connect kitt4sme.collab-cloud.eu:443 -showcerts
CONNECTED(00000003)
depth=0 CN = *.collab-cloud.eu
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 CN = *.collab-cloud.eu
verify error:num=21:unable to verify the first certificate
verify return:1
depth=0 CN = *.collab-cloud.eu
verify return:1
---
Certificate chain
 0 s:CN = *.collab-cloud.eu
   i:C = US, ST = Texas, L = Houston, O = SSL Corporation, CN = SSL.com RSA SSL subCA
   a:PKEY: rsaEncryption, 2048 (bit); sigalg: RSA-SHA256
   v:NotBefore: Apr  1 12:54:12 2022 GMT; NotAfter: May  2 12:54:12 2023 GMT
-----BEGIN CERTIFICATE-----
MIIHUDCCBTigAwIBAgIQfIvT1jBxv1Uu2svR29e24zANBgkqhkiG9w0BAQsFADBp
MQswCQYDVQQGEwJVUzEOMAwGA1UECAwFVGV4YXMxEDAOBgNVBAcMB0hvdXN0b24x
GDAWBgNVBAoMD1NTTCBDb3Jwb3JhdGlvbjEeMBwGA1UEAwwVU1NMLmNvbSBSU0Eg
U1NMIHN1YkNBMB4XDTIyMDQwMTEyNTQxMloXDTIzMDUwMjEyNTQxMlowHDEaMBgG
A1UEAwwRKi5jb2xsYWItY2xvdWQuZXUwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAw
ggEKAoIBAQCnkRTo8argvn/D03/9zB6kJnWM9GaDU8I1mgXxkVZrDAn8oAWX24m4
x1x3JBW8YjGs2ct+Oi7rATPuKlX6xYccvmswv6yxdbz8wSPcKUT7qTAatZ/iX9rV
AP6yNDXBUbqKcxQq6PkpeqntxLOJgc4YsO+4qUMLQlQC3McXwsRUl90W/zBgjQdx
vXC1x+9DpPSS9CkTU0gZAEhysIF5VoXwfBJtPY6xO8tfBxk/Ry0lQD6hPSyjinW2
kZr6dvOjaqaGPTdzJRtJ8wWkIhTBtwO80qLHt3mI+0ASKH00gd1sRD7gQElKDT9e
ZEPcoxRuBiHqG5UZzkbxiWEeEg4IzXa9AgMBAAGjggM/MIIDOzAMBgNVHRMBAf8E
AjAAMB8GA1UdIwQYMBaAFCYUfuDc16b34tQEJ99h8cLs5zLKMHIGCCsGAQUFBwEB
BGYwZDBABggrBgEFBQcwAoY0aHR0cDovL2NlcnQuc3NsLmNvbS9TU0xjb20tU3Vi
Q0EtU1NMLVJTQS00MDk2LVIxLmNlcjAgBggrBgEFBQcwAYYUaHR0cDovL29jc3Bz
LnNzbC5jb20wLQYDVR0RBCYwJIIRKi5jb2xsYWItY2xvdWQuZXWCD2NvbGxhYi1j
bG91ZC5ldTBRBgNVHSAESjBIMAgGBmeBDAECATA8BgwrBgEEAYKpMAEDAQEwLDAq
BggrBgEFBQcCARYeaHR0cHM6Ly93d3cuc3NsLmNvbS9yZXBvc2l0b3J5MB0GA1Ud
JQQWMBQGCCsGAQUFBwMCBggrBgEFBQcDATBFBgNVHR8EPjA8MDqgOKA2hjRodHRw
Oi8vY3Jscy5zc2wuY29tL1NTTGNvbS1TdWJDQS1TU0wtUlNBLTQwOTYtUjEuY3Js
MB0GA1UdDgQWBBTvGjAmWGDjKrKxZjs0njVqunV0LTAOBgNVHQ8BAf8EBAMCBaAw
ggF9BgorBgEEAdZ5AgQCBIIBbQSCAWkBZwB1ALNzdwfhhFD4Y4bWBancEQlKeS2x
ZwwLh9zwAw55NqWaAAABf+U58rIAAAQDAEYwRAIgcIj+sTM8j3Ttad9wx0qrNvTC
UvjXJ/UWGGnHacUFbWYCIDKmkTjYENW/t8LW1kJhvM5bHOqXF//zAwRLRC0ZqoPC
AHcAejKMVNi3LbYg6jjgUh7phBZwMhOFTTvSK8E6V6NS61IAAAF/5TnzHwAABAMA
SDBGAiEAsBgkzhNCN/rIGCDCt9Oj9kI1tBs7k3n2B6DknsuY9j4CIQDhGESh4GaF
DjiGqjwsN9BMlyJJBg6TJwGzHYF2m4vQWAB1AOg+0No+9QY1MudXKLyJa8kD08vR
EWvs62nhd31tBr1uAAABf+U58rAAAAQDAEYwRAIgWXQ9d5xfBjrQWgPMYokVoBkD
HwB/fBxPgcV55Ts89fgCIHMFmJnCb5ZTXDKbwfu18MTr3NfLKo3IZGw7K58abV3f
MA0GCSqGSIb3DQEBCwUAA4ICAQCA/Vr3XetQ6l+xGKSTyG1PJA0SLcYxI6QsFEw7
2PsHPbGigTFHPVBbcIMY1o5ouwDhaVAW6uToWZryXnKTF7I+BgaNxtM7ZjYWjCSq
2tYaRaBPZXf+KNt0MNvZRsvEyAzLQzKiALEhNDj6byZ+8Q6+FW/n0mu+Kz1uEACX
PjyoQYe8ljSom0diFbGogAew3Uo9L5/whbsNUKV2IvSzMiInbSe129ctP2KzzXPk
DlJH+vQccHlKBLxGAoZKOdPC1phyPxOZut4XkID4gsGNsXxfQmGv3lEgrBvffWKN
r4fbo7xKQ9TZq12iIxb1pO/FbFuJlGT6sQnxxkJKh+135nlDvYXLJm2+T1LcQjTW
gGxP4bzbY4+jimpuX9DYrS4xiF8pqaZMg7nSHENXe7VL8fhpQYHfRynWUx65P5hN
5GqM5z6Mn5XrM0YhwyUcgfhfBinePPzgOFfAP6AeDYPCem+uh1wH+ecTunlHumDb
4ozny8Fal3KT9ACvAZTbwr4xbx3wCB4pzzoCr3r4c86DL8fUnXrm9vYoP/USYlqM
fHkqWo2ccbYqhFp2GCAvwzECTcaLE6U9k0yCL/h6FmopEI2IspAGNAKBixnHU/7H
SqPJQIKda7XDoyhB4P+JwApbMN0B3R6JXVyUD905V+o7OHJ1KcsC82jEsgKu7D9b
oPaBNg==
-----END CERTIFICATE-----
---
Server certificate
subject=CN = *.collab-cloud.eu
issuer=C = US, ST = Texas, L = Houston, O = SSL Corporation, CN = SSL.com RSA SSL subCA
---
No client certificate CA names sent
Peer signing digest: SHA256
Peer signature type: RSA-PSS
Server Temp Key: X25519, 253 bits
---
SSL handshake has read 2366 bytes and written 406 bytes
Verification error: unable to verify the first certificate
---
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384
Server public key is 2048 bit
Secure Renegotiation IS NOT supported
Compression: NONE
Expansion: NONE
No ALPN negotiated
Early data was not sent
Verify return code: 21 (unable to verify the first certificate)

Expected behavior

We get a new TLS certificate that's valid from 01 May 2023 to at least 30 Jul 2023. Replace the old cert and key on kitt4sme.collab-cloud.eu (/etc/certs) with the new ones, reinstall the new cert in the Istio gateway and add it to the Argo CD config too.

Additional context

• 207

• 230

[c0c0n3]

@karikolehmainen instead of getting a new cert ourselves and pay for it, we could instead deploy Certificate Manager and configure it to automatically get free certs from Let's Encrypt and automatically renew them on expiry.

[Alireza-x]

@karikolehmainen: Hard deadline. Please follow up the ticket

[karikolehmainen]

New certs have been uploaded to the server to folder /etc/cert/cert_2023

I will update the kubernetes certs ASAP, but I will try to test those first on an other machine.

[karikolehmainen]

certs have to be recreated by deleting the old istio-gw-cert and creating a new oen with command:

kubectl create -n istio-system secret tls istio-gw-cert --key=/etc/certs/server.key --cert=/etc/certs/server.crt

[karikolehmainen]

Finally new certs have been created to the platform. Problem was the user was not added correctly to microk8s group and kubectl config was off. Old certs deleted with command kubectl delete -n istio-system secret istio-gw-cert and new ones created with: kubectl create -n istio-system secret tls istio-gw-cert --key=/etc/certs/server.key --cert=/etc/certs/server.crt

[c0c0n3]

@karikolehmainen awesome job!

[c0c0n3]

@karikolehmainen so I've recreated the Istio GW secret with this command

$ kubectl create -n istio-system secret tls istio-gw-cert --key=server.key --cert=fullchain.crt

where the key and cert files are the new ones you put together with the full cert CA chain in /etc/certs/.

Everything seems to work fine now and if I validate our TLS setup with

• https://www.geocerts.com/ssl-checker

it tells me everything is hunky-dory and that the certificate chain is complete.

happy days!