Define 'CURL_CA_BUNDLE' with default values

[viscropst]

Inserting default certification bundle file to avoiding no certification info error for normal use 'static-curl' feature on Unix(-like) systems.

[alexcrichton]

Can you elabortate amore on why this is necessary? It looks like this is probing at build-time where the certificates are but the bundled version is frequently used for cross-compiled builds meaning the target system may not look like the system that's building the project.

[viscropst]

Because when I using 'static-curl' option to build and run a example like below on alpine linux with libressl, It's panics with Error { description: "SSL peer certificate or SSH remote key was not OK", code: 60, extra: Some("SSL certificate problem: unable to get local issuer certificate"), but I compile the curl with libressl by source code curl-sys provided was normally to https request. And according to curl's configure script it's iterate to setting the system default ca bundle path as default CA_BUNDLE path. So I have to bring up this PR, although I can set CURL_CA_BUNDLE environment instead at runtime.

example's cargo.toml dependencies (Other section stays cargo init --application's)

[dependencies]
curl = { version = "*', features = [ "static-curl","ssl" ] }

example's main.rs

use curl::easy::Easy;

// Capture output into a local `Vec`.
fn main() {
    let ver = curl::Version::get();
    println!("version is: {}", ver.version());
    println!("ssl version is: {}", ver.ssl_version());
    println!("ca bundle info is: {}", ver.certinfo().unwrap_or("None"));
    println!("ca path info is: {}", ver.certinfo().unwrap_or("None"));

    let mut dst = Vec::new();
    let mut easy = Easy::new();
    easy.url("https://www.rust-lang.org/").unwrap();

    let mut transfer = easy.transfer();
    transfer.write_function(|data| {
        dst.extend_from_slice(data);
        Ok(data.len())
    }).unwrap();
    transfer.perform().unwrap();
}

Cargo run command RUSTFLAGS=“-C target-feature=-crt-static” cargo run,output was

mine-surface3:~/hello# RUSTFLAGS="-C target-feature=-crt-static" cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.11s
     Running `target/debug/hello`
version is: 7.83.0-DEV
ssl version is: LibreSSL/3.4.3
ca bundle info is: None
ca path info is: None
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error { description: "SSL peer certificate or SSH remote key was not OK", code: 60, extra: Some("SSL certificate problem: unable to get local issuer certificate") }', src/main.rs:21:24
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrac 

[sagebind]

Arguably, curl's configure script is wrong, because scanning the host's file system for cert bundle paths tells you nothing about the paths actually in use on the machine you run the program on. You might run the program on the same system sure, but you might also run the same executable on an entirely different one (say something Debian-based with different paths).

As I think about it more, this seems like actually a footgun on curl's part that could lead to "it works on my machine" problems, where SSL works by default only on machines that happen to have the trusted certs bundle at the same path as the machine the program was compiled on.

You can instead set the CURLOPT_CAINFO option at runtime to specify where to find the cert bundles (via Easy::certinfo).

[viscropst]

@sagebind I think you are right. But another thing confused me. When I switching the openssl example works without other configuration. It is openssl worked more than libressl when curl initializing the openssl ? output with verbose on and openssl switched :

mine-surface3:~/hello# RUSTFLAGS="-C target-feature=-crt-static" cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.11s
     Running `target/debug/hello`
version was: 7.83.0-DEV
ssl version: OpenSSL/1.1.1n
ca bundle info was: None
ca path info was None
*   Trying 108.138.36.107:443...
*   Trying 2600:9000:201f:2400:14:513f:eb40:93a1:443...
* Immediate connect fail for 2600:9000:201f:2400:14:513f:eb40:93a1: Network unreachable
*   Trying 2600:9000:201f:2800:14:513f:eb40:93a1:443...
* Immediate connect fail for 2600:9000:201f:2800:14:513f:eb40:93a1: Network unreachable
*   Trying 2600:9000:201f:5000:14:513f:eb40:93a1:443...
* Immediate connect fail for 2600:9000:201f:5000:14:513f:eb40:93a1: Network unreachable
*   Trying 2600:9000:201f:7c00:14:513f:eb40:93a1:443...
* Immediate connect fail for 2600:9000:201f:7c00:14:513f:eb40:93a1: Network unreachable
*   Trying 2600:9000:201f:9800:14:513f:eb40:93a1:443...
* Immediate connect fail for 2600:9000:201f:9800:14:513f:eb40:93a1: Network unreachable
*   Trying 2600:9000:201f:b000:14:513f:eb40:93a1:443...
* Immediate connect fail for 2600:9000:201f:b000:14:513f:eb40:93a1: Network unreachable
*   Trying 2600:9000:201f:e800:14:513f:eb40:93a1:443...
* Immediate connect fail for 2600:9000:201f:e800:14:513f:eb40:93a1: Network unreachable
*   Trying 2600:9000:201f:f600:14:513f:eb40:93a1:443...
* Immediate connect fail for 2600:9000:201f:f600:14:513f:eb40:93a1: Network unreachable
* Connected to www.rust-lang.org (108.138.36.107) port 443 (#0)
* ALPN: offers http/1.1
*  CAfile: /etc/ssl/cert.pem
*  CApath: /etc/ssl/certs
} * TLSv1.3 (OUT), TLS handshake, Client hello (1):
} (512 bytes of data)
{ z* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ (122 bytes of data)
{ { *{ * TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ 
       http/1.1{ (5 bytes of data)
{ * TLSv1.3 (IN), TLS handshake, Certificate (11):
{ (4988 bytes of data)
{ { * TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ (264 bytes of data)
{ 5{ * TLSv1.3 (IN), TLS handshake, Finished (20):
{ (36 bytes of data)
} * TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} } 5} * TLSv1.3 (OUT), TLS handshake, Finished (20):
} (36 bytes of data)
* SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256
* ALPN: server accepted http/1.1
* Server certificate:
*  subject: CN=www.rust-lang.org
*  start date: Sep 12 00:00:00 2021 GMT
*  expire date: Oct 11 23:59:59 2022 GMT
*  subjectAltName: host "www.rust-lang.org" matched cert's "www.rust-lang.org"
*  issuer: C=US; O=Amazon; OU=Server CA 1B; CN=Amazon
*  SSL certificate verify ok.
} I} > GET / HTTP/1.1
Host: www.rust-lang.org
Accept: */*

{ (5 bytes of data)
{ * Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Type: text/html; charset=utf-8
< Content-Length: 19392
< Connection: keep-alive
< Server: Rocket
< X-Xss-Protection: 1; mode=block
< Strict-Transport-Security: max-age=63072000
< X-Content-Type-Options: nosniff
< Referrer-Policy: no-referrer, strict-origin-when-cross-origin
< Content-Security-Policy: default-src 'self'; frame-ancestors 'self'; img-src 'self' avatars.githubusercontent.com; frame-src 'self' player.vimeo.com
< Date: Sun, 01 May 2022 19:45:23 GMT
< Via: 1.1 vegur, 1.1 75964e4626dd702b8dac2690031df25a.cloudfront.net (CloudFront)
< Vary: Accept-Encoding
< X-Cache: Miss from cloudfront
< X-Amz-Cf-Pop: MUC50-P2
< X-Amz-Cf-Id: 7evcox1ILX5OZ_aY7ukZiNPitvw-xfrwi42ukMApmdvbk1wOuIBBMg==
<

[sagebind]

This is getting more into curl internals, but I believe when not specified curl will default to using whatever cert bundle paths the SSL library is configured to use by default. Perhaps on your system, OpenSSL was built to use /etc/ssl/cert.pem but LibreSSL was not built with any default.

It could also be that you have either the SSL_CERT_DIR or SSL_CERT_FILE environment variables set, which OpenSSL looks at by default when the application does not load a particular CA bundle, but I believe LibreSSL does not do this.

[sagebind]

Sorry, forgot to follow up on this, but I do now know the cause of the behavior you are seeing. The reason why it works by default when using OpenSSL is because the curl crate uses openssl-probe during initialization to search common CA certificate paths and populate the SSL_CERT_FILE environment variable if it is not already set. OpenSSL will then use this environment variable automatically to discover and use the CA certificate store.

This breaks down when using LibreSSL, because LibreSSL ignores the SSL_CERT_FILE environment variable entirely and so this probing behavior isn't used. It is then up to the compiler of LibreSSL to know where it should be looking for CA certificates automatically for the system it is built on, or for the application to do this itself. This fails when statically linking LibreSSL because you are the one who is building LibreSSL, and not having given LibreSSL a default CA path at build time, it is up to the caller to specify a CA path at runtime. This behavior is described by the OpenSSL crate (which also will build LibreSSL).

So in either case, whether using LibreSSL or OpenSSL, when statically linking the SSL library, it will have no default CA path configured. openssl-probe will find and set the right path automatically at runtime for OpenSSL, but nothing will happen at runtime for LibreSSL and so LibreSSL is left without any default CA path to use.

Now the search paths that curl checks in its configure script do in fact "save the day" by default when compiling on a system using LibreSSL and running the resulting program on the same system, because curl will set CURLOPT_CAINFO to be CURL_CA_BUNDLE by default. That said, this is still not good behavior, because running the resulting binary on a different system also using LibreSSL but with its CA path in a different location will unexpectedly fail. It might be possible to convince me otherwise, but I am firmly of the opinion that searching CA paths at compile time is (for most applications) the wrong behavior and a footgun when statically linking.

Aside: Setting CA path at compile time can make sense when compiling a dynamic library intended to be installed system-wide. For example, Linux package maintainers compiling libcurl to be installed system-wide on their distro know where their distro stores its CA certificates, and can simply set it as the default. Any application that loads that system-wide library will then "magically" use the correct CA path without needing to do anything extra.

So in conclusion, I am still against this PR as-is. When statically linking curl with LibreSSL, at least as it stands today, you're going to have to set the CA paths yourself in your application. Arguably doing anything super fancy here is kind of outside the scope of this crate.

Now if there was a way to make the found path info produced by openssl-probe work with LibreSSL then I might be open to that. It would be a more modest change, and it would be correct behavior by default. In theory this could be done by secretly setting CURLOPT_CAINFO every time a new easy handle is created. I'm not sure if such a thing would be appropriate though since we try to stick pretty close to vanilla libcurl behavior, I'll have to think about that.

[viscropst]

Thanks for you reminding me. These commit just thinking about build and run the application code with libressl on mine system configuration, but breaks the final binary's protablity on other *nix systems. So I'll find out the other way to setting a default ca bundle path at runtime. If the workload would be done on this PR, I'll re-open it.