rust-cross / cargo-zigbuild

Compile Cargo project with zig as linker
MIT License
1.43k stars 51 forks source link

CACHEDIR.TAG not created for some `*-apple-darwin` targets #165

Closed sandhose closed 11 months ago

sandhose commented 11 months ago

Cargo creates CACHEDIR.TAG files in some directories of the target/ folder, so that they are marked as excluded for backups. This is also used by the Swatinem/rust-cache action to know if the directory should be kept for cache or not

Turns out, for some reason with *-apple-darwin targets (at least), cargo-zigbuild doesn't create such a file.

Repo:

Create an empty bin crate:

> cargo new cachedir-repro
     Created binary (application) `cachedir-repro` package
> cd cachedir-repo/

It works as expected with cargo build on the aarch64-apple-darwin target:

> rm -rf target/
> cargo build --target aarch64-apple-darwin
...
> cat target/aarch64-apple-darwin/CACHEDIR.TAG
Signature: 8a477f597d28d172789f06886806bc55
# This file is a cache directory tag created by cargo.
# For information about cache directory tags see https://bford.info/cachedir/

It doesn't with cargo zigbuild on the aarch64-apple-darwin target:

> rm -rf target/
> cargo zigbuild --target aarch64-apple-darwin
...
> cat target/aarch64-apple-darwin/CACHEDIR.TAG
cat: target/aarch64-apple-darwin/CACHEDIR.TAG: No such file or directory

But it works with some targets:

> rm -rf target/
> cargo zigbuild --target aarch64-unknown-linux-gnu
...
> cat target/aarch64-unknown-linux-gnu/CACHEDIR.TAG
Signature: 8a477f597d28d172789f06886806bc55
# This file is a cache directory tag created by cargo.
# For information about cache directory tags see https://bford.info/cachedir/

This is affecting me because it means macOS builds are not properly cached in my CI, with a combination of cargo-zigbuild and Swatinem/rust-cache.

Note that the host target seems irrelevant, I tried both from a Linux host and a macOS host, and it affects AFAIK the aarch64-apple-darwin and the x86_64-apple-darwin targets

messense commented 11 months ago

I'm not sure why it's not created because eventually it's cargo driving the build, cargo-zigbuild only sets up some environment variables to use zig as c/c++ compiler and linker.

Nothing obvious I can do here.

sandhose commented 11 months ago

But the issue exists, can be reliably reproduced, has real consequences (in my case, macOS CI builds not being cached), and looks somehow specific to cargo-zigbuild, since regular equivalent cargo calls work as expected. I can try investigating a bit more on my free time, but I really think the issue should stay open in the meantime

messense commented 11 months ago

Sure, we can open it if you want to investigate.

Do note that I'll close it again if nothing comes up after a month or two.

sandhose commented 11 months ago

I found the culprit:

https://github.com/rust-cross/cargo-zigbuild/blob/0c4827bbf6bcacb0b5eb814be244b506a2961a98/src/zig.rs#L749-L777

Before invoking cargo build, the target/[profile]/deps/ directory is being created, so when cargo is invoked, it skips the target directory creation (done here)

I don't exactly know how one would fix this, but I can confirm commenting the last two lines is enough to fix the issue

messense commented 11 months ago

Thanks, see also https://github.com/rust-lang/cargo/issues/12441.