search

rust-lang / cargo

The Rust package manager
https://doc.rust-lang.org/cargo
Apache License 2.0
12.55k stars 2.38k forks source link

cargo builds proc-macros twice based on the profile #13321

Open kchibisov opened 7 months ago

kchibisov commented 7 months ago

Problem

I've recently noticed a difference when building alacritty in debug and release build that I have 216 deps in debug, but 217 in release. I've started investigating and found out that some proc macros are being built twice, but with slight difference (not in features though).

In the log I found the following for --release build (--debug has just one of these lines).

     Running `CARGO=/opt/rust-bin-1.74.1/bin/cargo-bin-1.74.1 CARGO_CRATE_NAME=alacritty_config_derive CARGO_MANIFEST_DIR=/home/kchibisov/src/rust/alacritty-workspace/fork/alacritty_config_derive CARGO_PKG_AUTHORS='Christian Duerr <contact@christianduerr.com>' CARGO_PKG_DESCRIPTION='Failure resistant deserialization derive' CARGO_PKG_HOMEPAGE='https://github.com/alacritty/alacritty' CARGO_PKG_LICENSE='MIT OR Apache-2.0' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=alacritty_config_derive CARGO_PKG_README='' CARGO_PKG_REPOSITORY='' CARGO_PKG_RUST_VERSION=1.70.0 CARGO_PKG_VERSION=0.2.3-dev CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=2 CARGO_PKG_VERSION_PATCH=3 CARGO_PKG_VERSION_PRE=dev CARGO_PRIMARY_PACKAGE=1 LD_LIBRARY_PATH='/home/kchibisov/src/rust/alacritty-workspace/fork/target/release/deps:/opt/rust-bin-1.74.1/lib:/home/kchibisov/.rustup/toolchains/gentoo/lib' /home/kchibisov/.rustup/toolchains/gentoo/bin/rustc --crate-name alacritty_config_derive --edition=2021 alacritty_config_derive/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type proc-macro --emit=dep-info,link -C prefer-dynamic -C embed-bitcode=no -C debug-assertions=off -C metadata=f11f6d33ed698a19 -C extra-filename=-f11f6d33ed698a19 --out-dir /home/kchibisov/src/rust/alacritty-workspace/fork/target/release/deps -L dependency=/home/kchibisov/src/rust/alacritty-workspace/fork/target/release/deps --extern proc_macro2=/home/kchibisov/src/rust/alacritty-workspace/fork/target/release/deps/libproc_macro2-f3902f4e534acb8e.rlib --extern quote=/home/kchibisov/src/rust/alacritty-workspace/fork/target/release/deps/libquote-d1c35e16b2cf01ea.rlib --extern syn=/home/kchibisov/src/rust/alacritty-workspace/fork/target/release/deps/libsyn-b1050101a14e2a06.rlib --extern proc_macro -C link-arg=-fuse-ld=mold`
     Running `CARGO=/opt/rust-bin-1.74.1/bin/cargo-bin-1.74.1 CARGO_CRATE_NAME=alacritty_config_derive CARGO_MANIFEST_DIR=/home/kchibisov/src/rust/alacritty-workspace/fork/alacritty_config_derive CARGO_PKG_AUTHORS='Christian Duerr <contact@christianduerr.com>' CARGO_PKG_DESCRIPTION='Failure resistant deserialization derive' CARGO_PKG_HOMEPAGE='https://github.com/alacritty/alacritty' CARGO_PKG_LICENSE='MIT OR Apache-2.0' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=alacritty_config_derive CARGO_PKG_README='' CARGO_PKG_REPOSITORY='' CARGO_PKG_RUST_VERSION=1.70.0 CARGO_PKG_VERSION=0.2.3-dev CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=2 CARGO_PKG_VERSION_PATCH=3 CARGO_PKG_VERSION_PRE=dev CARGO_PRIMARY_PACKAGE=1 LD_LIBRARY_PATH='/home/kchibisov/src/rust/alacritty-workspace/fork/target/release/deps:/opt/rust-bin-1.74.1/lib:/home/kchibisov/.rustup/toolchains/gentoo/lib' /home/kchibisov/.rustup/toolchains/gentoo/bin/rustc --crate-name alacritty_config_derive --edition=2021 alacritty_config_derive/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type proc-macro --emit=dep-info,link -C prefer-dynamic -C opt-level=3 -C embed-bitcode=no -C debuginfo=1 -C metadata=e13a7f7d7b2067ba -C extra-filename=-e13a7f7d7b2067ba --out-dir /home/kchibisov/src/rust/alacritty-workspace/fork/target/release/deps -L dependency=/home/kchibisov/src/rust/alacritty-workspace/fork/target/release/deps --extern proc_macro2=/home/kchibisov/src/rust/alacritty-workspace/fork/target/release/deps/libproc_macro2-f3902f4e534acb8e.rlib --extern quote=/home/kchibisov/src/rust/alacritty-workspace/fork/target/release/deps/libquote-d1c35e16b2cf01ea.rlib --extern syn=/home/kchibisov/src/rust/alacritty-workspace/fork/target/release/deps/libsyn-b1050101a14e2a06.rlib --extern proc_macro -C link-arg=-fuse-ld=mold`

And the difference in each line is only the following:

embed-bitcode=no -C debug-assertions=off -C metadata=f11f6d33ed698a19 -C extra-filename=-f11f6d33ed698a19
opt-level=3 -C embed-bitcode=no -C debuginfo=1 -C metadata=e13a7f7d7b2067ba -C extra-filename=-e13a7f7d7b2067ba

which is just different optimization level, but I build debug and release at the same time when doing release, so it sounds weird? I've tried removing profile in workspace and it didn't help.

Steps

  1. https://github.com/alacritty/alacritty/
  2. cargo clean; cargo build --jobs 1 -vvv 2> debug.txt >&2
  3. cargo clean; cargo build --release --jobs 1 -vvv 2> release.txt >&2

Then you can sort the files, remove empty lines and some garbage, and diff them to discover the diff in basically one line and after you remove it the end diff is the same with the same amount of occurrences for alacritty_config_derive.

Possible Solution(s)

No response

Notes

I've seen https://github.com/rust-lang/cargo/issues/10645 , but I don't have different features and I'm not sure how to solve that, it seems like some .so artifacts are not used at all.

Version

cargo 1.74.1 (ecb9851af 2023-10-18)
epage commented 7 months ago

proc macros, build scripts, and their dependencies get built for your host computer while everything else gets built for the target. For most people, these are probably the same thing. In dealing with this, cargo tries to unify them as much as possible to remove redundancy but there are likely cases where it can't or isn't yet. This unification is a fairly complicated task.

I wonder if this is related to #11252. I know a previous version of it caused even more duplicate builds but don't remember if that still had some or not.

weihanglo commented 7 months ago

This test might be related:

https://github.com/rust-lang/cargo/blob/a8187e5885a525c4f69e74e81f462d28da46add0/tests/testsuite/features2.rs#L1102-L1106

So this might be the case:

weihanglo commented 7 months ago

Try adding this. It should combine them into one rustc invocation.

[profile.release.package.alacritty_config_derive]
opt-level = 0
debug = 0
kchibisov commented 7 months ago

I'd like to add that I've initially noticed it with sqlx and it's macros feature, where the difference is around ~70 crates for release/debug build(I don't have any profile and it's not a workspace).

Try adding this. It should combine them into one rustc invocation.

Yeah, that fixes it. But I'd assume that the macro itself will run slower.