400 invalid base64 encoding ERROR: issue, analysis and a possible solution (Microsoft Azure IoT Edge)

[arlotito]

Hi, If I use curl to request a certificate over the /simpleenroll endpoint, I get the "400 invalid base64 encoding" response from the EST server.

I guess the problem is caused by the BEGIN/END CERTIFICATE REQUEST header and footer in the CSR. The EST server indeed tries to base64 decode the entire incoming CSR, header&footer included... and that causes the 'invalid base64 encoding' error.

If I remove the header/footer from the CSR, everything works.

Here are the steps to reproduce the issue:

# create a CSR
openssl req -new -nodes \
    -newkey rsa:4096 -keyout key.pem \
    -subj "/C=US/ST=Somewhere/L=Somewhere/O=company x/OU=iot/CN=device1" \
    -out csr.pem

...and you will get a csr.pem like this:

-----BEGIN CERTIFICATE REQUEST-----
MIIErjCCApYCAQAwaTELMAkGA1UEBhMCVVMxEjAQBgNVBAgMCVNvbWV3aGVyZTES
MBAGA1UEBwwJU29tZXdoZXJlMRIwEAYDVQQKDAljb21wYW55IHgxDDAKBgNVBAsM
...
t8jXqWb7efx/YiP7OFQL3DvM7+3gbgkjMoTD4mJu7GQzsMwZl5r8ecy0TXuputR7
WjM=
-----END CERTIFICATE REQUEST-----

Let's submit the CSR over the simpleenroll endpoint to request for the certificate:

# submit CSR (-k to trust the self-signed server certificate)
curl -X POST --data-binary "@csr.pem" \
      -H "Content-Type: application/pkcs10" \
      -H "Content-Transfer-Encoding: base64" \
      -k \
      https://est.contoso.com:8449/.well-known/est/simpleenroll

...and I get the 400 invalid base64 encoding error.

If I remove the header/footer from the CSR and re-submit it to the EST server, it works and I get a response with a proper certificate.

This is btw what happens if you use the EST server to issue certificates for the Azure IoT Edge 1.2 via the built-in EST integration. The IoT Edge will issue a CSR including the header and footer, and it will get a "400 invalid base64 encoding ERROR" as a response.

As a workaround, I patched the EST server code to remove the header/footer (if any) from the incoming CSR before performing the base64 decoding. Patch is here: https://github.com/arlotito/est/commit/502dbf8e4489f57774c66516aa6d13ac78135a09

With that patch applied, both curl and iotedge work without issues.

I also created a containerized est server here

I'd be happy if you could have a look at it and tell me what you think. I'm also happy to open a PR if it helps.

Thanks!

[paulgriffiths]

Hi Arturo -

I guess the problem is caused by the BEGIN/END CERTIFICATE REQUEST header and footer in the CSR. The EST server indeed tries to base64 decode the entire incoming CSR, header&footer included... and that causes the 'invalid base64 encoding' error.

If I remove the header/footer from the CSR, everything works.

This is the correct behavior.

If you take a look at RFC 5967 section 2.1 which defines the application/pkcs10 media type, or RFC 7030 appendix A.3 which shows an example of valid /simpleenroll message, you can see that the CSR should be a base64-encoded DER representation of the CSR, and not a PEM-encoded CSR. This is made more explicit by RFC 8951 section 3, which updates RFC 7030 and says:

[for] all endpoints using base64 encoding ... the Distinguished Encoding Rules (DER) ... are used to produce the input for the base64 encoding routine ... Senders are not required to insert any kind of white space.

Note that the same applies to the certificates returned from /cacerts and the enroll endpoints, as well as to the PKCS#7 and PKCS#8 structures returned by /serverkeygen - none of them are PEM-encoded, by design. The estclient executable in this package does accept PEM-encoded input and gives PEM-encoded output for convenience, but it's translating those to/from the proper formats for the actual EST communications with the server.

This is btw what happens if you use the EST server to issue certificates for the Azure IoT Edge 1.2 via the built-in EST integration. The IoT Edge will issue a CSR including the header and footer, and it will get a "400 invalid base64 encoding ERROR" as a response.

This is interesting, and sounds like a bug if it's accurate. Let me check with our integration team, and I'll see if we can verify this.

Thanks!

[paulgriffiths]

Hi @arlotito - we've been able to confirm this behavior, so it does indeed look like there'a bug in Microsoft's client. We'll notify them, but in the meantime I'll close out this issue, since there's no modification required on our side.

Thanks for reporting this!