Non-determinsitic output while targetting wasm32 on different OS

[BlackAsLight]

So I create some UserScripts as a hobby for a game and have been looking to write some of them in Rust and compile them down to wasm, but I have noticed that what outputted .wasm binary file is different based off the OS that is building it.

I created a little repo to try and test exactly where it is, using my MacOS builds and GitHub actions for a Ubuntu build. Both running the same version of rustc.

> rustc --version
rustc 1.75.0-nightly (4b85902b4 2023-11-04)

info: syncing channel updates for 'nightly-x86_64-unknown-linux-gnu'
info: latest update on 2023-11-05, rust version 1.75.0-nightly (4b85902b4 2023-11-04)

The program I am testing with is a simple app that prints 'Hello, world!'.

fn main() {
    println!("Hello, world!");
}

The first test I did was using rustc itself to build the .wasm binaries and did manage to get identical outputs across MacOS and Ubuntu

rustc --target=wasm32-unknown-unknown main.rs

The second test I did was switching to using Cargo instead and found that it was now producing different outputs across MacOS and Ubuntu for the same code.

cargo build --release --target=wasm32-unknown-unknown

I don't know why its not being deterministic when I used cargo over rustc as the flags I passed to either on both OS' are identical with the same versions of rustc. You can see the entire repo here. Any help would be appreciated.

[bjorn3]

Can you do cargo clean and then cargo build --release --target=wasm32-unknown-unknown -v to show the rustc invocations? Is there any difference between them? And what exactly are the rustc invocations.

[BlackAsLight]

I made those changes and got this. The only differences that I can see is that the MacOS one has this extra argument --diagnostic-width=383 and that the metadata and extra-filename numbers are different.

MacOS

cargo clean
     Removed 12 files, 1.8MiB total
cargo build --release --target=wasm32-unknown-unknown -v
   Compiling deterministic_rust v0.1.2 (/Users/soul/Projects/deterministic_rust)
     Running `/Users/soul/.rustup/toolchains/nightly-aarch64-apple-darwin/bin/rustc --crate-name deterministic_rust --edition=2021 src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=383 --crate-type bin --emit=dep-info,link -C opt-level=3 -C embed-bitcode=no -C metadata=f1726c506e5d08ac -C extra-filename=-f1726c506e5d08ac --out-dir /Users/soul/Projects/deterministic_rust/target/wasm32-unknown-unknown/release/deps --target wasm32-unknown-unknown -L dependency=/Users/soul/Projects/deterministic_rust/target/wasm32-unknown-unknown/release/deps -L dependency=/Users/soul/Projects/deterministic_rust/target/release/deps`
    Finished release [optimized] target(s) in 0.18s

Ubuntu

cargo clean
     Removed 0 files
cargo build --release --target=wasm32-unknown-unknown -v
   Compiling deterministic_rust v0.1.2 (/home/runner/work/deterministic_rust/deterministic_rust)
     Running `/home/runner/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/rustc --crate-name deterministic_rust --edition=2021 src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C opt-level=3 -C embed-bitcode=no -C metadata=73924012e8d629f5 -C extra-filename=-73924012e8d629f5 --out-dir /home/runner/work/deterministic_rust/deterministic_rust/target/wasm32-unknown-unknown/release/deps --target wasm32-unknown-unknown -L dependency=/home/runner/work/deterministic_rust/deterministic_rust/target/wasm32-unknown-unknown/release/deps -L dependency=/home/runner/work/deterministic_rust/deterministic_rust/target/release/deps`
    Finished release [optimized] target(s) in 0.11s

[bjorn3]

and that the metadata and extra-filename numbers are different.

Different -Cmetadata arguments will result in different compilation outputs. As for why cargo would pass different values: Do you have a global ~/.cargo/config.toml (cargo also reads a file without the .toml extension) If so what is the content?

[BlackAsLight]

Do you have a global ~/.cargo/config.toml (cargo also reads a file without the .toml extension) If so what is the content?

I don't have this file. Is there a way I can overwrite it with something static that won't break anything?

[BlackAsLight]

After much investigation I can't seem to figure out how to manipulate the -C metadata flag in a way that would get me "static results" (Independent of device), so I switched to doing a rustc command instead of going through cargo so I could omit this flag from it.

Was Doing

cargo +nightly build --release --target=wasm32-unknown-unknown -v

Now Doing

mkdir -p target/wasm32-unknown-unknown/release
rustc +nightly --crate-name deterministic_rust --edition=2021 src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=383 --crate-type bin --emit=dep-info,link -C opt-level=3 -C embed-bitcode=no --out-dir target/wasm32-unknown-unknown/release/ --target wasm32-unknown-unknown

[alexcrichton]

The reason that -C metadata is different on different hosts is that Cargo will execute rustc -vV (a "verbose version") which includes the host target. This verbose version is then hashed into the -C metadata value. Because different hosts have different "verbose versions" the -C metadata key will be different.

If you're shooting for deterministic builds across platforms it's probably best to strip the name section from wasm binaries as otherwise the output should be deterministic.

[bjorn3]

-Cmetadata doesn't just affect symbol names, but can also affect symbol definition order afaik and with -Zrandomize-layout struct layouts. Wouldn't it make sense for cargo to strip the host field from rustc -vV before hashing?

[alexcrichton]

Ah excellent point, in that case yeah I think it might be reasonable to strip that out of Cargo's hashing of the rustc version.

[alexcrichton]

I've opened https://github.com/rust-lang/cargo/pull/14107 for this in Cargo

[BlackAsLight]

So with that merge, https://github.com/rust-lang/cargo/pull/14107, it now seems to produce consistent output. repo testing deterministic output