Support for PEM-Encoded CA Certificates with Rustls Backend in reqwest Problem

[davidklein147]

Description

In my Rust project, I am using the reqwest library to make server calls over mTls, which requires adding client certificates. These certificates are in PEM format. To add them to the client builder's identity, I included the "rustls-tls" feature, which supports PEM format certificates and configured the client builder to use the rustls backend.

However, when attempting to load build-in CA certificates from the computer, it does not work as expected. The issue seems to be related to the way reqwest handles build-in CA certificates when using the rustls backend. Specifically, reqwest relies on the "rustls-tls-webpki-roots" feature to load build-in root CA certificates, but this feature is documented as supporting only DER formats, which contradicts the ability to use PEM format certificates with the rustls backend.

This inconsistency poses a challenge when working with PEM-encoded CA certificates and using the rustls backend.

Expected Behavior

I expect reqwest to support loading build-in CA certificates in PEM format when using the rustls backend, as this is a common use case for mTls.

Additional Information

The issue seems to be related to the "rustls-tls-webpki-roots" feature, which only officially supports DER formats.

Below is a code snippet of reqwesr::clientBuilde::build that used OwnedTrustAnchor::from_subject_spki_name_constraints

#[cfg(feature = "rustls-tls-webpki-roots")]
      if config.tls_built_in_root_certs {
          use rustls::OwnedTrustAnchor;
          let trust_anchors =
              webpki_roots::TLS_SERVER_ROOTS.iter().map(|trust_anchor| {
                  OwnedTrustAnchor::from_subject_spki_name_constraints(
                      trust_anchor.subject,
                      trust_anchor.spki,
                      trust_anchor.name_constraints,
                  )
              });
          root_cert_store.add_trust_anchors(trust_anchors);
      }

and blow is the docuention if this function

    /// Constructs an `OwnedTrustAnchor` from its components.
    ///
    /// All inputs are DER-encoded.
    ///
    ...

Loading CA certificates in PEM format is essential for many real-world scenarios and should be supported by reqwest.

reqwest = { version = "0.11.4", features = ["blocking", "json", "stream", "rustls-tls"] } Rust version: rustc 1.73.0 (cc66ad468 2023-10-03) Operating system: windows

Example

In my case for local development, I created my own CA certificate and signed the server certificate with it, I added my CA certificate for me in the list of trusted root certificates. I launched node server with ssl with certificates issued by my CA. blow is the two options I tried to do:

1.

#[tokio::main]
async fn main() {
    let client: Client = ClientBuilder::new().build().unwrap();
   let res = client.get("https://localhost:3010/api/device/descoer").send().await;
   match res {
      Ok(val) => {println!("{:?}", val)}
      Err(err) => {println!("{:?}", err)}
   }
}

the result:

Response { url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("localhost")), port: Some(3010), path: "/api/device/descoer", query: None, fragment: None }, status: 401, headers: {"x-powered-by": "Express", "access-control-allow-origin": "*", "content-type": "application/json; charset=utf-8", "content-length": "129", "etag": "W/\"81-4ZNFF8L66kWCN400Lh0XtKqj9W0\"", "date": "Thu, 26 Oct 2023 13:38:25 GMT", "connection": "keep-alive", "keep-alive": "timeout=5"} }

it is unauthorized because I don't attected the client certificate, but it works, is arrive to the server and get back an error

2.

#[tokio::main]
async fn main() {
   let client: Client = ClientBuilder::new().use_rustls_tls().tls_built_in_root_certs(true).build().unwrap();
   let res = client.get("https://localhost:3010/api/device/descoer").send().await;
   match res {
      Ok(val) => {println!("{:?}", val)}
      Err(err) => {println!("{:?}", err)}
   }
}

and in this case is the result:

reqwest::Error { kind: Request, url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("localhost")), port: Some(3010), path: "/api/device/descoer", query: None, fragment: None }, source: hyper::Error(Connect, Custom { kind: Other, error: Custom { kind: InvalidData, error: InvalidCertificate(UnknownIssuer) } }) }

UnknownIssuer

Thank you for your attention to this matter. Resolving this issue would enhance the usability of reqwest for a broader range of use cases and improve the overall developer experience.

[seanmonstar]

Thanks for the report! I'd welcome any contribution trying to fix this. Is it something reqwest needs to handle internally? Or is something that could be done by rustls or webpki?

[davidklein147]

It seems that this should be handled either in webpki which will allow certificates to be loaded in PEM format, or in rustls which will use something else that supports PEN format.

In any case, there is some conflict here, on the one hand, to use PEM format I must use the "rustlt backend", and the "rustle backend" itself does not support the PEM format (in the section of loading CA certificates)

[djc]

This issue seems to confuse the server roots used to verify the server certificate with the client certificates (apparently stored in PEM format) that are sent to the server to verify the client's identity. The rustls project provides the rustls-pemfile crate to help parse DER out of PEM files, so you should probably consider using it.

(That is, you should probably be using some of the Identity constructors here, which do already support PEM.)