hotg-ai / rune

Rune provides containers to encapsulate and deploy edgeML pipelines and applications
Apache License 2.0
136 stars 15 forks source link

Make WASM3 build bindgen instead of requiring the CLI program to be installed #259

Open Michael-F-Bryan opened 3 years ago

Michael-F-Bryan commented 3 years ago

This is something both @jacekpie and @kthakore ran into.

Out of the box we are unable to compile wasm3-sys because its build script requires the bindgen CLI tool to be installed. There is a build-bindgen feature flag which tells it to pull bindgen in as a crate instead that we should use.

Michael-F-Bryan commented 3 years ago

It looks like you can't naively apply the following patch because wasm3-sys uses an older version of bindgen (which in turn requires an old version of clang-sys, which clashes with the tflite build script).

diff --git a/crates/wasm3-runtime/Cargo.toml b/crates/wasm3-runtime/Cargo.toml
index c288bc2817..dab6f3a534 100644
--- a/crates/wasm3-runtime/Cargo.toml
+++ b/crates/wasm3-runtime/Cargo.toml
@@ -18,7 +18,7 @@ anyhow = "1.0.40"
 log = "0.4.14"
 hotg-rune-runtime = { path = "../runtime", version = "^0.5.4-dev"}
 hotg-rune-core = { path = "../rune-core", version = "^0.5.4-dev"}
-wasm3 = "0.1.3"
+wasm3 = { version = "0.1.3", features = ["build-bindgen"] }

 [dev-dependencies]
 tempfile = "3.2.0"

The build error:

$ cargo build -p hotg-rune-wasm3-runtime
error: failed to select a version for `clang-sys`.
    ... required by package `bindgen v0.54.0`
    ... which is depended on by `wasm3-sys v0.1.2`
    ... which is depended on by `wasm3 v0.1.3`
    ... which is depended on by `hotg-rune-wasm3-runtime v0.5.4-dev (/home/michael/Documents/hotg-ai/rune/crates/wasm3-runtime)`
versions that meet the requirements `^0.29` are: 0.29.3, 0.29.2, 0.29.1, 0.29.0

the package `clang-sys` links to the native library `clang`, but it conflicts with a previous package which links to `clang` as well:
package `clang-sys v1.2.0`
    ... which is depended on by `bindgen v0.55.1`
    ... which is depended on by `tflite v0.9.6`
    ... which is depended on by `hotg-rune-cli v0.5.4-dev (/home/michael/Documents/hotg-ai/rune/crates/rune-cli)`
Only one package in the dependency graph may specify the same links value. This helps ensure that only one copy of a native library is linked in the final binary. Try to adjust your dependencies so that only one package uses the links ='clang-sys' value. For more information, see https://doc.rust-lang.org/cargo/reference/resolver.html#links.

failed to select a version for `clang-sys` which could resolve this conflict
Michael-F-Bryan commented 3 years ago

Updating to wasm3 v0.2.0 now results in the opposite problem... tflite pulls in bindgen v0.55.1 (clang-sys v0.29.0) while wasm3-sys pulls in bindgen v0.58.1 (clang-sys 1.20), and linking to multiple versions of the same *-sys crate isn't allowed because you'd get duplicate symbol errors when linking in the native library they bind to.

jonas-schievink commented 3 years ago

I've opened https://github.com/boncheolgu/tflite-rs/pull/53 to update tflite's bindgen dependency to a version that shouldn't cause this conflict.

Michael-F-Bryan commented 3 years ago

I just tried making the switch locally and ran into the same linking conflict because the wasm3-sys on crates.io uses bindgen v0.54.0 while hotg-runecoral uses bindgen v0.59.1.

I've created https://github.com/wasm3/wasm3-rs/issues/32 upstream.