stdlib is not included when building a project with cranelift

[alexzanderr]

stdlib is not included when building a project with cranelift

hello

im trying to build my project with cranelift

i've followed the instructions in the readme:

• git clone https://github.com/bjorn3/rustc_codegen_cranelift && cd rustc_codegen_cranelift in my project
• ./y.rs prepare
• ./y.rs build or test.sh
• i've run the ./rustc_codegen_cranelift/dist/cargo-clif run from my project seems to work

but the problem is that when im running cargo-clif run goes through the process of building and the rust standard library is not found.

error i got:

❱  ./cargo-clif run
   Compiling proc-macro2 v1.0.49
   Compiling unicode-ident v1.0.6
   Compiling quote v1.0.23
   Compiling syn v1.0.107
   Compiling serde_derive v1.0.152
   Compiling serde v1.0.152
   Compiling smallvec v1.10.0
   Compiling pkg-config v0.3.26
^C  Building [                           ] 0/280: quote(build.rs), s...

...
...
...
...
...
...

there are a lot of errors

error[E0425]: cannot find function, tuple struct or tuple variant `Err` in this scope
   --> /.cargo/registry/src/github.com-1ecc6299db9ec823/pkg-config-0.3.26/src/lib.rs:824:17
    |
824 |                 Err(Error::Failure {
    |                 ^^^ not found in this scope

error[E0425]: cannot find function, tuple struct or tuple variant `Err` in this scope
   --> /.cargo/registry/src/github.com-1ecc6299db9ec823/pkg-config-0.3.26/src/lib.rs:830:23
    |
830 |         Err(cause) => Err(Error::Command {
    |                       ^^^ not found in this scope

error: could not compile `smallvec` due to 139 previous errors
error: could not compile `pkg-config` due to 153 previous errors

basic stuff from std like Vec, String, Err, Some, etc ... are missing

what am i missing? what is the problem?

i saw that rust-toolchain was present and the script did its stuff with custom toolchain, so i didnt have any influence on the build.

this is the output of the test.sh:

❱  ./test.sh
[BUILD] y.rs
warning: associated function `fetch` is never used
   --> xbuild-example/rustc_codegen_cranelift/build_system/utils.rs:119:19
    |
119 |     pub(crate) fn fetch(&self, cargo: impl AsRef<Path>, dirs: &D...
    |                   ^^^^^
    |
    = note: `#[warn(dead_code)]` on by default

warning: 1 warning emitted

[BUILD] rustc_codegen_cranelift
    Finished release [optimized] target(s) in 0.04s
[BUILD] sysroot None
[BUILD] mini_core
[mini_core                     : codegen mono items] start
[mini_core                     : codegen mono items] end time: 2.555995ms
[BUILD] example
[example                       : codegen mono items] start
[example                       : codegen mono items] end time: 1.547054ms
[JIT] mini_core_hello_world
[mini_core_hello_world         : codegen mono items] start
[mini_core_hello_world         : codegen mono items] end time: 5.280721ms
Rustc codegen cranelift will JIT run the executable, because -Cllvm-args=mode=jit was passed
Hello printf
Hello
World!
free(): invalid pointer

exit code is 1, the output doesnt seem to be something wrong, but about the exit code i dont have a good feeling.

note: prepare and build was successful

extra note: i saw that the other deps were built and serde must have std:

   Compiling proc-macro2 v1.0.49
   Compiling unicode-ident v1.0.6
   Compiling quote v1.0.23
   Compiling syn v1.0.107
   Compiling serde_derive v1.0.152
   Compiling serde v1.0.152
   Compiling smallvec v1.10.0
   Compiling pkg-config v0.3.26
   ....

so i dont know

thanks in advance!

[bjorn3]

this is the output of the test.sh:

This is failure in test.sh is a known issue: https://github.com/bjorn3/rustc_codegen_cranelift/issues/1303 I believe the glibc version determines if this issue shows up or not. Ubuntu 20.04 doesn't have issues, but Ubuntu 22.04 does. Same for Debian 11 working fine and the upcoming Debian 12 not working. It only shows up for this specific jit mode test and not any of the other tests. You aren't using jit mode, so this shouldn't give any issues.

Did you use ./y.rs build or ./test.sh? If you used ./test.sh try ./y.rs build to see if that solves the issue. The above issue probably caused ./test.sh to fail before building the standard library.

[alexzanderr]

i used ./y.rs prepare && ./y.rs build first, in this order

[alexzanderr]

ok now its working ... what ?

i've rerun ./y.rs build and everything was building again (shouldnt because i didnt change anything, except meanwhile i switched to mold linker) and:

• cranelift built with code 0
• test is the same
• now i can build my project with cranelift, so now its working, but i dont know what happened ...

anyway, something is wrong, because i've set my linker to be mold and i dont see all of my cores running when linking (so its not using mold, cuz mold is multi-threaded and uses all cores)

i also have the mold linker defined globally inside ~/.cargo/config.toml, but cranelift doesnt seem to use it

[bjorn3]

ok now its working ... what ?

Weird.

i also have the mold linker defined globally inside ~/.cargo/config.toml, but cranelift doesnt seem to use it

How did you specify the linker? Did you use the rustflags key in ~/.cargo/config.toml? If so I think the RUSTFLAGS env var set by cargo-clif overrides it. If you set the linker using the RUSTFLAGS env var, cargo-clif should append it's own rustflags to it rather than override it. I'm not quite sure how to make cargo-clif append the rustflags when using the rustflags key in ~/.cargo/config.toml too without copying the entire logic cargo uses to determine the value to use for rustflags.

[alexzanderr]

❱ cat ~/.cargo/config.toml

[target.x86_64-unknown-linux-gnu]
linker = "clang"
rustflags = ["-C", "link-arg=-fuse-ld=/usr/bin/mold"]

[alexzanderr]

well for sure cargo-clif ignores my global rustflags because linking is faster without cranelift so cranelift doesnt use mold from settings.

how do i specify to cranelift to use the mold linker?

[alexzanderr]

can i use linker arguments within the build.rs script of my project like this for cranelift to know about external linker?

println!("cargo:rustc-link-arg= ... ");

[alexzanderr]

i think i found one possible working solution: to pass rustflags as env variable to cargo-clif process:

RUSTFLAGS="-C link-arg=-fuse-ld=/usr/bin/mold" \
./rustc_codegen_cranelift/dist/cargo-clif build

this seems to work, how do i know?

• because in the linking part i see all my cores up an running, so mold is running too
• and also i get the fastest clean compilation and 1 line change compilation than any other tests i've made

[bjorn3]

Happy to hear that it works now. Can this issue be closed?

[alexzanderr]

sure i have 1 question before closing, to not open a new issue for that.

can cranelift be used for android, ios, and wasm32 platforms/targert_os'es ?

[bjorn3]

Cranelift doesn't yet support compiling for Webassembly. It also doesn't yet support AArch64 + Mach-O, so ios won't work. I would expect android to work though, but I never tried it.

[alexzanderr]

sure, thanks for everything!