Certificate data with newlines parsed incorrectly from kubeconfig (NO_CERTIFICATE_OR_CRL_FOUND)

[ridangotaavi]

Which project are you reporting a bug for?

kr8s

What happened?

Kr8s version: 0.17.2 Python version: 3.11.2

I have a kubeconfig, which contains the CA certificate in the following format:

- name: my-cluster
  cluster:
    server: https://my-cluster-hostname
    certificate-authority-data: |
      LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURCVENDQWUyZ0F3SUJBZ0lJ
      U05hNGY0RFRlRWd3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhN
      S2EzVmlaWEp1WlhSbGN6QWVGdzB5TkRBME1Ua3dPVE01TWpsYUZ3MHpOREEwTVRj
      d09UUTBNamxhTUJVeApFekFSQmdOVkJBTVRDbXQxWW1WeWJtVjBaWE13Z2dFaU1B
      MEdDU3FHU0liM0RRRUJBUVVBQTRJQkR3QXdnZ0VLCkFvSUJBUUNjSWZ0YS9NeU5i
      OEZwWjdySVR4ZCtnZHRMRFVwbjViaUpZS05CSVk0WHV1WWh6ZElNbkZjclFsM24K
      RVRuWFFpNGx6bnRiT3JhMFdMb0oxUkRrVnlVeEE3R0Jiem5GTk83dE8vM0xnWG9E
      bHQ0dEF6RzFrYzRkMWtpMwpKN3QwS2xuUStLYWdxdDFrd1RwUHkzbjVkdldjYUZQ
      aXBSWmFWWFNUM21RanBQWmtZaERmVGlJVlZGbEFHYnlNCnlPNHIxQ1J2aFNDOTU0
      T1FiYVZiMTBhdVNEc0x3QzgwTW01MWU5emdTT1o2YWs4SU52TDluaVd2ZHNTeXlV
      clUKSDRoRFB5WUxjSGorMGtwU0NpU1ltRGdDbkY0OGZnYWR0bUZWblcrcWRkUTdB
      MERURkxWc0dUWlQ2cm1kdHZQdAplb0JvMjlkRnJVdmlHL3ppV2lEaFZNUUwzL2dO
      QWdNQkFBR2pXVEJYTUE0R0ExVWREd0VCL3dRRUF3SUNwREFQCkJnTlZIUk1CQWY4
      RUJUQURBUUgvTUIwR0ExVWREZ1FXQkJRMi80a2lXQ3ZPUm5Qa2dEV0Y4Z0xvek9D
      L0hUQVYKQmdOVkhSRUVEakFNZ2dwcmRXSmxjbTVsZEdWek1BMEdDU3FHU0liM0RR
      RUJDd1VBQTRJQkFRQmZhQlZPV2Z4dApjYjdXby85UHM4WnNwQzdOUDlCSHc2ZmFR
      Q2N1QU9UekVYdzN4ZTB3elBaRU9zdUdVbHBPTGtsZC9BMmpGak1NCkxYRWIyeHlY
      QUJGKytGRHpnajNzMUJuZStIeFpVelJHSWhCN0NtbFNxYmQ5RjRkc1B4YTN2eXAx
      c2hITkhVdlgKYjIxb2luS1JyVmxUUnFNc2diVW91MjNyaDJoQkpIREpGaU5veGt5
      ejd3K0Jld3hIV0EyUk5MeEphWDI1SmdUeAoxRTRQZWlkZjBnUkJHR3hDcHJTM2N6
      U0ZPSXBVL1pWZzRySTRSYnRFZWptUnhZb09xWDE2OEEwQjNReDR6RVB1Cms3NzdE
      TXR6OEJuK3d6Zno4T1RYT2pPZWU3N294VHB2anpzZTEycm9iWVBNQXVQNXArVkRw
      TlMwcE1VTjgrb3EKNWhoTk5Odkt4V04rCi0tLS0tRU5EIENFUlRJRklDQVRFLS0t
      LS0K

When trying to use this kubeconfig with kr8s, it fails with the following traceback:

Traceback (most recent call last):
  File "/home/taavi/projects/kr8s-testing/main.py", line 7, in <module>
    print(client.whoami())
          ^^^^^^^^^^^^^^^
  File "/home/taavi/projects/kr8s-testing/.venv/lib/python3.11/site-packages/kr8s/_async_utils.py", line 121, in run_sync_inner
    return portal.call(wrapped)
           ^^^^^^^^^^^^^^^^^^^^
  File "/home/taavi/projects/kr8s-testing/.venv/lib/python3.11/site-packages/kr8s/_async_utils.py", line 92, in call
    return self._portal.call(func, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/taavi/projects/kr8s-testing/.venv/lib/python3.11/site-packages/anyio/from_thread.py", line 287, in call
    return cast(T_Retval, self.start_task_soon(func, *args).result())
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/_base.py", line 456, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/home/taavi/projects/kr8s-testing/.venv/lib/python3.11/site-packages/anyio/from_thread.py", line 218, in _call_func
    retval = await retval_or_awaitable
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/taavi/projects/kr8s-testing/.venv/lib/python3.11/site-packages/kr8s/_api.py", line 268, in whoami
    return await self.async_whoami()
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/taavi/projects/kr8s-testing/.venv/lib/python3.11/site-packages/kr8s/_api.py", line 277, in async_whoami
    async with self.call_api(
  File "/usr/lib/python3.11/contextlib.py", line 204, in __aenter__
    return await anext(self.gen)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/taavi/projects/kr8s-testing/.venv/lib/python3.11/site-packages/kr8s/_api.py", line 147, in call_api
    await self._create_session()
  File "/home/taavi/projects/kr8s-testing/.venv/lib/python3.11/site-packages/kr8s/_api.py", line 106, in _create_session
    verify=await self.auth.ssl_context(),
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/taavi/projects/kr8s-testing/.venv/lib/python3.11/site-packages/kr8s/_auth.py", line 100, in ssl_context
    sslcontext.load_verify_locations(cafile=self.server_ca_file)
ssl.SSLError: [X509: NO_CERTIFICATE_OR_CRL_FOUND] no certificate or crl found (_ssl.c:4123)

I have used this same kubeconfig pretty much everywhere else (including kubectl) and this is the first time I run into this kind of problem.

Anything else?

Replacing line 236 in _auth.py with this:

self._cluster["certificate-authority-data"].replace(os.linesep, ""), validate=True

seems to fix the issue, but I am not sure whether this is a proper fix.

[jacobtomlinson]

So it looks like the problem here is the validate=True, which raises an exception if the base64 encoded certificate contains invalid characters (like newlines). If we remove this then it goes back to the default behaviour of stripping the newlines automatically and things work as expected with the example you provided.

It looks like the validate=True was introduced in #324 to handle situations where the cert is in plaintext and not base64 encoded, but is the correct length and therefore gets mistakenly decoded.

I think the fix here is to remove the validate=True in favour of some other way to check if the string is base64 encoded or not.

[jacobtomlinson]

Hmm unfortunately it's not straight forward to check if a string is base64 encoded. Any string where len(s) % 4 == 0 will successfully decode. And if you encode it again you get the same string out that you started with.

So I think we are going to need to first check whether the input data is a valid certificate instead, and if it isn't then attempt to base64 decode it.

We could either use something like cryptography.x509.load_pem_x509_certificates() to verify the cert, or maybe it would be simpler to just check if it starts with -----BEGIN CERTIFICATE-----.