Failed to compile on windows with `ssl-vendored` & `cmake-build` combination

[peter-formlogic]

Hey, I can't seem to get this to link on windows with a vendored ssl.

I.e, using:

rdkafka = { version = "0.29.0", features = ["ssl-vendored", "cmake-build"] }

I get the following output:

          librdkafka_sys-54d8d0d1481a5f9e.rlib(rdkafka_ssl.obj) : error LNK2019: unresolved external symbol ENGINE_load_ssl_client_cert referenced in function rd_kafka_ssl_set_certs
          librdkafka_sys-54d8d0d1481a5f9e.rlib(rdkafka_ssl.obj) : error LNK2019: unresolved external symbol ENGINE_free referenced in function rd_kafka_ssl_ctx_init
          librdkafka_sys-54d8d0d1481a5f9e.rlib(rdkafka_ssl.obj) : error LNK2019: unresolved external symbol ENGINE_by_id referenced in function rd_kafka_ssl_ctx_init
          librdkafka_sys-54d8d0d1481a5f9e.rlib(rdkafka_ssl.obj) : error LNK2019: unresolved external symbol ENGINE_ctrl_cmd_string referenced in function rd_kafka_ssl_ctx_init
          librdkafka_sys-54d8d0d1481a5f9e.rlib(rdkafka_ssl.obj) : error LNK2019: unresolved external symbol ENGINE_init referenced in function rd_kafka_ssl_ctx_init

I am also seeing a lot of these errors:

libopenssl_sys-ba70b1b6d496cd9b.rlib(bio_b64.obj) : warning LNK4099: PDB 'ossl_static.pdb' was not found with 'libopenssl_sys-ba70b1b6d496cd9b.rlib(bio_b64.obj)' or at '..\target\debug\deps\ossl_static.pdb'; linking object as if no debug info

But I can confirm that ossl_static.pdb is in the debug\build\openssl-sys-63196b440406ab23\out\openssl-build\install\lib directory alongside the two lib files:

libcrypto.lib  libssl.lib  ossl_static.pdb

The link.exe statement does include that lib directory:

"/LIBPATH:<snipped>\\target\\debug\\build\\openssl-sys-63196b440406ab23\\out\\openssl-build\\install\\lib"

[WaterKnight1998]

I am fighting with the same on Linux

[peter-formlogic]

Ok, I've investigated this more and it appears to be because openssl-src does not include the engine module by default on windows (& musl) targets: https://github.com/alexcrichton/openssl-src-rs/blob/97babda37a440e13a1492e95af25c7fbd33c20fe/src/lib.rs#L193-L201

So when it compiles the static version of openssl, it does so without the appropriate exports, hence why it's complaining.

If you add the following to Cargo.toml it seems to resolve it (note: it needs to be under build-dependencies), but I think we should include this in the rdkafka library when vendoring and so I'll raise a PR:

[build-dependencies]
openssl-src = { version = "111", features = ["force-engine"] }

[WaterKnight1998]

@peter-formlogic

Cargo.toml is like follows but I am still getting an issue

rdkafka = { version = "0.29.0", features = ["cmake-build", "ssl-vendored"], default-features = false }

[build-dependencies]
openssl-src = { version = "111.25.3+1.1.1t", features = [ "force-engine" ] }

Error:

 engines/e_afalg.c:24:10: fatal error: linux/version.h: No such file or directory
     24 | #include <linux/version.h>
        |          ^~~~~~~~~~~~~~~~~
  compilation terminated.
  make[1]: *** [Makefile:5752: engines/e_afalg.o] Error 1
  make[1]: *** Waiting for unfinished jobs....
  make: *** [Makefile:177: build_libs] Error 2
  thread 'main' panicked at '

[peter-formlogic]

Sounds like there are no linux headers installed wherever you're trying to run this.

You could try adding linux-headers-generic as a package on ubuntu and kernel-headers-musl on arch.

Given Cargo.toml:

[package]
name = "rdtest"
version = "0.1.0"
edition = "2021"

[dependencies]
rdkafka = { version = "0.29.0", features = ["cmake-build", "ssl-vendored"]}

[build-dependencies]
openssl-src = { version = "111", features = [ "force-engine" ] }

and main.rs:

use rdkafka::Timestamp;

fn main() {
    let timestamp = Timestamp::now();
    println!("{timestamp:?}");
}

The following docker file compiles & runs just fine:

FROM clux/muslrust

RUN ln -s /bin/g++ /bin/musl-g++

ADD Cargo.toml .
ADD src src

RUN cargo build --release

CMD ["target/x86_64-unknown-linux-musl/release/rdtest"]

[WaterKnight1998]

Morning, after installing kernel-headers-musl on arch now I get the following error: note: rust-lld: error: unable to find library -lresolv

[WaterKnight1998]

Removing "gssapi-vendored", "libz-static"], default-features = false did the trick, thanks so much :)

[WaterKnight1998]

However, in Ubuntu the package you mention doesn't solve the error

[Orca-bit]

in openssl-sys, they add this: https://github.com/sfackler/rust-openssl/blob/745b200425390ab2b0b5aa641a89237b2ff7426e/openssl-sys/Cargo.toml#L28

openssl-src = { version = "300.1.2", optional = true, features = ["legacy"] }

if you add

openssl-src = { version = "300",  features = ["force-engine"] }

these configurations may conflict

so I add this:

[build-dependencies]
openssl-src = { version = "300", features = ["legacy", "force-engine"] }

it works