search

rust-lang / cargo

The Rust package manager
https://doc.rust-lang.org/cargo
Apache License 2.0
12.53k stars 2.37k forks source link

Need a reliable way to get the target dir from the build script #9661

Open kennykerr opened 3 years ago

kennykerr commented 3 years ago

The problem: I need to locate the target dir from the build script.

The solution: I'd love a CARGO_TARGET_DIR environment variable. There's clearly precedent for this as cargo doc depends on such a thing today.

@jyn514 suggested I reach out to the Cargo team for advice.

Currently the Windows crate has been using various hacky solutions that are problematic.

https://github.com/rust-lang/docs.rs/issues/1444#issuecomment-875188411

kennykerr commented 3 months ago

What path would I add?

VorpalBlade commented 2 months ago

I would very much rather not frame these discussions more generally; the concrete details for how a solution is wanting to be used and why can make a big difference.

@epage I would, I'm not interested in generating C++ headers, but I do want to generate man pages and shell completion files from build.rs of my crate, in a predictable location for external packaging tools (e.g. debian, arch linux, etc). OUT_DIR is a poor choice since it has a hash appended to it that change all the time. I don't want to bloat my binary by including the man page or completion generation code into the final binary.

There doesn't seem to be any good solution currently.

epage commented 2 months ago

@VorpalBlade it sounds like you are disagreeing with me but I'm not sure on what. My comment was about us not talking abstractly about this problem but speaking in terms of concrete use cases, like your man page use case.

VorpalBlade commented 2 months ago

@epage Ah then I believe I misunderstood you as wanting to only concentrate on the specific case of generating headers.

This is what I currently do:

This is quite hacky on the PKGBUILD side:

  1. In prepare() I need to clean out the build dir: rm -rf target/release/build/${pkgname}-* in case we are not running on a clean build tree (as might happen when you develop the packaging)
  2. Later on in the package() phase (the bash functions are written in the order they are called for clarity) I locate the build dir using $(ls -d target/release/build/${pkgname}-*/out) (because the bash options are set a bit screwy in PKGBUILDs I needed that ls workaround to expand properly).

This breaks down if there are multiple out dirs as happens when you are developing and changing the package. During development it also leaves cruft in the build tree that is useless:

❯ find target/debug/ -name paketkoll.1 | wc -l
26

That really shouldn't be there in so many copies. In fact I see no reason for cargo retaining all of that. Some sort of garbage collection would be in order for any packages that are part of your workspace (and thus are unlikely to be around in multiple versions at the same time and are also like to change a lot).

epage commented 2 months ago

That really shouldn't be there in so many copies. In fact I see no reason for cargo retaining all of that. Some sort of garbage collection would be in order for any packages that are part of your workspace (and thus are unlikely to be around in multiple versions at the same time and are also like to change a lot).

GC is being tracked at #12633. Our initial focus is on global resources and then target directories as a whole. We'll then evaluate what doing it inside of a target-dir looks like. However, I think its unlikely that we'd leave only one instance present.

VorpalBlade commented 2 months ago

Fair enough, what about the rest of the use case though? As you can see, the current solution is hacky and suboptimal.

epage commented 2 months ago

Yes, if I'm understanding correctly, this is like the C++ use cases where they are wanting to have build.rs generate and stage final artifacts. I did a quick search in our backlog and #13663 looks to be a more specific issue for that and I recommend discussion move over there.

epage commented 2 months ago

I propose to the Cargo team that we close this issue with the encouragement that people move their discussion to more specific, use-case focused issues, creating them where needed.

As mentioned in https://github.com/rust-lang/cargo/issues/9661#issuecomment-1769481293, there are fundamental problems with build.rss accessing CARGO_TARGET_DIR and we need to look for what solutions align with where Cargo is at and where it is going.

Tracked use cases:

Use cases where I could not find an Issue tracking it:

alerque commented 2 months ago

Having watched this issue and a number of the related ones for a while, that sounds to me like a plan that will effectively make sure this is not addressed. When the individual use cases discuss this topic the answer is usually along the lines of "this may affect other things too so we can't change it". Having at least one meta issue open tracking the multiple use cases and how they interact seems like the only way likely to actually reach a conclusion.

epage commented 2 months ago

In looking at the use cases I just summarized, they are so unrelated to each other that I don't see how having a tracking issue for them would help with coordination across them. As for your concern, I can't speak too much to that without examples as that failure case does not ring a bell.

sam0x17 commented 2 months ago

The main issue gives us a way to consolidate power and make it clear a number of different use cases could all benefit from the same improved functionality in build.rs. Dividing these obfuscates the collective community need for an improvement here, and will make it much more difficult to coordinate across different use-cases as a solution is designed