search

rust-lang / cargo

The Rust package manager
https://doc.rust-lang.org/cargo
Apache License 2.0
12.66k stars 2.4k forks source link

Possibility to set the package version dynamically #6583

Open vorner opened 5 years ago

vorner commented 5 years ago

Hello

I'm building a rust application using internal teamcity CI job. I want the build number to be part of the version of the binary (so eg. ./app --version knows from which build it came).

The only way I found so far is to let the build first edit the Cargo.toml (sed -i -e 's/^version = .*/version = "%build.version%"/' Cargo.toml), which seems ugly and fragile.

It would be great if I could somehow override the version from Cargo.toml through the command line ‒ either as a parameter or an environment variable.

Would something like that make sense? Are there plans to support it?

dwijnand commented 5 years ago

I think this could be created as a cargo extension using https://github.com/ordian/toml_edit. You could use https://github.com/killercup/cargo-edit as inspiration.

vorner commented 5 years ago

Actually, I don't want to modify the Cargo.toml at all if possible, just use that different value of version during a single cargo build. So something like cargo build --config package.version="0.1.2".

But yes, you're right, there are certainly less fragile ways to edit Cargo.toml.

dwijnand commented 5 years ago

I think there's a ticket for being able to modify options (I guess I'm both Cargo.toml and .cargo/config) via a command, like git config, but I don't know if it extends to temporary overrides.

mixalturek commented 5 years ago

It's very common use case to specify the version from command line while releasing on a CI server that tracks and auto increments the build number. GNU/make, Maven and Gradle natively support it, manual editing of a build-spec file is not practical and error prone.

make target VERSION=%build.version%
mvn release:prepare -DreleaseVersion=%build.version%
./gradlew build -Pversion=%build.version%
dwijnand commented 5 years ago

mvn release:prepare is based on the maven release plugin. The equivalent is https://github.com/sunng87/cargo-release, and I don't know if it supports specifying the version on the command line.

vorner commented 5 years ago

I don't think it is equivalent for my use case. This is for releasing on crates.io and modifying Cargo.toml.

What I want is somewhat completely the opposite. I want to leave Cargo.toml as it is, not touch git (tags, push) at all ‒ that happens externally. I want to just build the binary, but the binary should be fed with a version different from the one in Cargo.toml, because for this use case, Cargo.toml is not the authoritative source.

dwijnand commented 5 years ago

What does

the binary should be fed with a version different from the one in Cargo.toml

mean? Is it just env!("CARGO_PKG_VERSION") (and variants, like CARGO_PKG_VERSION_MAJOR) usage?

vorner commented 5 years ago

I'm not 100% sure there are no other places the version is passed into the compilation, but I guess so. I haven't tried yet (I might), but I guess cargo will set the env var unconditionally and overwrite it even if I pass it from outside.

dwijnand commented 5 years ago

Maybe we could just change that: only set the environment var if unset.

lukaslueg commented 5 years ago

built may help your use-case to some extent.

vorner commented 5 years ago

Looking at built, it seems to be really nice and useful thing and I'll keep this in mind. But I don't think this will help me here ‒ this doesn't help me push the right version into eg. clap or structopt.

dwijnand commented 5 years ago

Can you not define the version in clap/structopt in terms of built_info::PKG_VERSION?

vorner commented 5 years ago

By default, they take the version from CARGO_PKG_VERSION. I could set that manually to built_info::PKG_VERSION but I don't see how that would help me, because built also takes the value from there.

dwijnand commented 5 years ago

You're right, sorry. I think the solution is to not override the environment variables, then.

Kaiser1989 commented 3 years ago

Is there any progress? Also looking for the feature to externally manipulate the package version. (CI is setting the version, Cargo.toml should not be changed).

An Env-Variable would be nice: CARGO_TOML_PACKAGE_VERSION

This could also be done for every other part of the Cargo.toml:

The place within the code is here: https://github.com/rust-lang/cargo/blob/master/src/cargo/util/toml/mod.rs#L68 There should be an env-override after parsing the toml and before checking for unused.

This should be analogous to https://doc.rust-lang.org/cargo/reference/config.html#environment-variables

I could add the implementation if we find a good name for env variables...

mieubrisse commented 3 years ago

Adding another +1 - we're setting the version upon release, and it'd be great to not need to edit Cargo.toml each time

cgranade commented 3 years ago

By way of another +1 and showing analogous functionality in other toolchains, the .NET SDK allows for overriding version numbers at the command line with commands like dotnet build /p:Version=1.0.1 (https://andrewlock.net/version-vs-versionsuffix-vs-packageversion-what-do-they-all-mean/). I've often found that overriding package versions that way can be useful in using CI pipelines to control how packages are versioned.

Ri0n commented 3 years ago

absolutely same for me. CI and stuff. I was going to use "built"'s GIT_VERSION for that, but we build in docker containers not passing .git inside. So this info is lost.

Humandoodlebug commented 2 years ago

This seems to be working on nightly right now: https://github.com/rust-lang/cargo/issues/6699

e.g. cargo build -Z unstable-options --config 'package.version="1.5.2"'

QAston commented 2 years ago

@Humandoodlebug 

cargo build -Z unstable-options --config 'package.version="1.5.2"'

Will not work because --config overrides the cargo configuration, not the build definition in Cargo.toml

colemickens commented 2 years ago

Editing the Cargo.toml isn't practical in some scenarios anyway - in Nix we don't really have conventional network access during build phases, and so it's not possible rewrite toml and regenerate lock files to workaround this.

I do see that there is a package macro that makes it easy to override the build version, falling back to the Cargo version, but it wouldn't be fun adding this to lots of Rust projects if Cargo could do it: https://docs.rs/git-version/latest/git_version/macro.git_version.html

Snazzie commented 2 years ago

Very surprising that this is still an open issue in feb 2022. 3 years have passed! probably lost/forgotten in the 1.2k issues. Theres no good way to programmatically replace the version in cargo.toml.

Could we @ some of the active maintainers to get some visibility on this issue?

+1 to adding version override in CLI like dotnet.

brandonros commented 2 years ago

Is there a way to get cargo build to blend Cargo.toml version with [[bin]] name output? so that the cargo build --release makes a project-name-version.dll for example?

burdiyan commented 2 years ago

+1 here. I don't understand where the practice of statically specifying a version in a checked in manifest file comes from. I just want to release whatever is already checked in without having to commit again to change any manifests. It's very error prone unless you're using some fancy automated tools. It's so easy to overwrite a version that was already published previously if you release forgetting to change the manifest.

Being able to specify the version in the command line and completely omitting the version from the manifest would be my ideal workflow!

epage commented 2 years ago

I suspect this is a large enough semantic change that this would need a major change proposal or an RFC and need a person to champion the proposal and implementation. It is unlikely for someone else to pick up and do all of that leg work on behalf of someone interested in this work. One challenge though is cargo is currently on a soft feature freeze and the cargo team is wanting to avoid distractions, including mentoring / shepherding, to be able to handle basic maintenance, finish currently committed work ,and to reduce technical debt so we can increase our capacity.

Things off the top of my head that would need to be figured out

burdiyan commented 2 years ago

@epage This is a nice summary to figure out a considerate solution to this problem! Thanks for that!

It's a pity that it's unlikely that this effort would come from the core team, but it's understandable.

On the other hand, maybe an easier and good enough solution could be adding a flag (or env var) to pass the desired version to cargo so it could ignore the one specified in Cargo.toml? I'm absolutely unaware of cargo's internals, but if there's only one place where Cargo.toml is read to extract the version, then it seems like a pretty easy change to make.

In my (somewhat biased) opinion: there should be no static file defining the "current" version of a package at all. This should be determined at release time by looking up the VCS info or elsewhere.

epage commented 2 years ago

On the other hand, maybe an easier and good enough solution could be adding a flag (or env var) to pass the desired version to cargo so it could ignore the one specified in Cargo.toml? I'm absolutely unaware of cargo's internals, but if there's only one place where Cargo.toml is read to extract the version, then it seems like a pretty easy change to make.

That still needs most of the questions I brought up answered.

In my (somewhat biased) opinion: there should be no static file defining the "current" version of a package at all. This should be determined at release time by looking up the VCS info or elsewhere.

Going back to your earlier comment

+1 here. I don't understand where the practice of statically specifying a version in a checked in manifest file comes from

I'd recommend doing a thought exercise to try to understand why so many ecosystems have static version fields. It will both give you more empathy for other ways of doing things but could also help when proposing solutions.

burdiyan commented 2 years ago

@epage

I'd recommend doing a thought exercise to try to understand why so many ecosystems have static version fields. It will both give you more empathy for other ways of doing things but could also help when proposing solutions.

I really did the thought exercise, and what I posted was my considerate conclusion :)

In my career (about 10 years) I've worked closely with several ecosystems: PHP, Python, JavaScript, Go (the longest time among others), and now I'm digging a bit into Rust. Among these, only Rust is so strict about having a static version in a file.

So in my experience the need for a static version file is roughly 50/50 across different ecosystems.

My conclusion is: I don't understand where the practice is coming from, nor do I see any usefulness in it. I never appreciated having it, and in my, admittedly personal and biased opinion, having a static version file checked in to the VCS has zero benefits.

I accidentally discovered this issue today, and to my surprise looks like I'm not the only person wondering about these design decisions, which makes me glad that I'm not the only weirdo complaining about it :)

burdiyan commented 2 years ago

And, don't get me wrong, I'm sure there's reasons why people decided to make static version files a requirement. It's probably because of my ignorance that I don't see it. But, being unable to overwrite it dynamically is what surprises me, and it's what I'm advocating for to be implemented in one way or another.

burdiyan commented 2 years ago

Trying to be useful, I give my opinionated answers to the questions you posted, in case it could be helpful if someone is willing to drive this forward:

  • Should the version field be made completely optional?

Yes. I think it should possible to omit the version in Cargo.toml.

  • What are cargo's semantics without a version field and when no version is specified?

Fail to release. Display a warning about a missing version providing options to either specify it in Cargo.toml or dynamically via flag or env var.

  • What all commands should accept a version?

I'm unaware about the internal of Cargo, but probably any command that is reading Cargo.toml to extract the version in the current implementation.

  • How do we pass in versions, especially when running against multiple packages?

I understand that this is the tricky one here. I see several options with different levels of user-friendliness:

  1. Require static version in case of a multi-package workspace.
  2. Be able to define versions for each package separately similar to --version=package-1=1.0.0,package-2=2.0.0 or anything like that.
  3. Have an explicit integration with different VCS and have conventions about how to name tags (or similar concepts). E.g. <package-name>/<version>.
  4. Only allow releasing one package at a time (probably not practical when there're dependencies between local packages).
  5. Probably many other options and ways to do it...
  • How do we help people when they miss specifying the version for one of the multiple packages?

Similar to the second answer above. Fail to release and provide guidance to fix it.

  • How do we help workspace users with path dependencies?
    • Currently, to publish a crate with a path dependency, it also needs to have a version requirement specified. That version requirement must match - the crate its pointing to. Tools like cargo-release handle updating version requirements of dependents so they don't become stale

That's even trickier. I don't have answer for it from the top of my head. Maybe in this case dynamic version release should be impossible. Or maybe it could depend on some hash identifier of the path dependency instead of semver (which I don't know if is a thing in Cargo at all).

  • Is there any prior art in other ecosystems that we can learn lessons from?

I described my experience with different ecosystems in the previous comments: https://github.com/rust-lang/cargo/issues/6583#issuecomment-1131675725

My favorite one is Go. Let the VCS be the source of truth, have conventions on how to extract versions from VCS, have a proxy in front of the VCS + checksum database for public packages to provide availability, reliability, redundancy, security and other properties. I understand this is way different from many other ecosystems, and probably impractical for Cargo at this point.

epage commented 2 years ago

I really did the thought exercise, and what I posted was my considerate conclusion :)

I'm unsure where the decision came from but from my experience

In JavaScript there're tools to either bump the version automatically,

cargo-edit has a cargo-set-version command. While I've been working to merge other cargo-edit commands into cargo, that one hasn't been on my radar as the need for it is a less clear. If this is of interest, feel free to create an issue here!

PHP doesn't need a static version file either.

Not being familiar enough with the PHP world, I looked this up. Publishing libraries pulls from VCS by default and can fall back to a static version. See https://getcomposer.org/doc/02-libraries.md#library-versioning (which links out to how they determine the version from VCS tags and branches)

In Python there's a version field, but the config is defined in setup.py file, which can run arbitrary code. So one could fetch the version from elsewhere if it's necessary. Again, if I'm not missing anything.

This depends on which python package tool you are using. For Poetry, its static. If I'm reading PEP 621 correctly, the standardized pyproject.toml version field will still support falling back to the __version__ attribute.

(originally looked to see if they more formalized dynamic versions than that for prior art)

Fail to release. Display a warning about a missing version providing options to either specify it in Cargo.toml or dynamically via flag or env var.

Let me see if I understand. cargo publish should fail but every other command should issue a warning? For the warnings, should there be a placeholder version or should any place the version is specified be omitted?

How do we pass in versions, especially when running against multiple packages?

I think how Python has worked around this can provide another idea: have an analogue of a build script for generating metadata. At publish time the version is captured and stored in the crate and the metadata dependencies are dropped.

This gives users the option of VCS, env variable, etc and puts them in control of what VCS is used. This could be specified with a package.version.script = true field.

(was really hoping PEP 621 codified dynamic fields to provide prior art)

How do we help workspace users with path dependencies?

RFC 2906 was looking at removing the version field from path dependencies but it was a breaking change. We decided to drop it rather than come up with a new design but what could be done is to have a dep = { path = "crate/foo", version.workspace = true } for users to explicitly opt-in to it. It can then read the static or dynamic version field, however its specified.

epage commented 2 years ago

In discussing https://github.com/killercup/cargo-edit/issues/782, something I was thinking about is if the cargo version number is the best tool for tracking end-user version numbers and in some cases where people want to inject versions, it'd be better to inject it directly into the application rather than the manifest.

You see this version split in some commercial applications like Windows where they have different version numbers depending on the context. I've also seen this with other proprietary software.

This also makes life a bit easier decoupling marketing from software development, kind of like how some people use code names for projects, not to obscure it, but because they need something to call it while marketing is coming up with a name (we understand, naming is hard)

psFried commented 2 years ago

A little late to the conversation, but I wanted to share a workaround that I've been using for a while now. You can override CARGO_PKG_VERSION in build.rs. Here's an example from the flow project:

fn main() {
    // If we set CARGO_PKG_VERSION this way, then it will override the default value, which is
    // taken from the `version` in Cargo.toml.
    if let Ok(val) = std::env::var("FLOWCTL_RELEASE_VERSION") {
        println!("cargo:rustc-env=CARGO_PKG_VERSION={}", val);
    }
    println!("cargo:rerun-if-env-changed=FLOWCTL_RELEASE_VERSION");
}

In CI, we set the FLOWCTL_RELEASE_VERSION env var based on the github release tag. clap will then use this version without needing to specify it manually. Maybe a little hacky, since Cargo doesn't actually document that you can override variables like this, but it's been working for me for quite some time now.

John15321 commented 1 year ago

I found a cargo plugin called cargo-bump it seems to be doing exactly that in a nice way of a cargo plugin: https://crates.io/crates/cargo-bump

epage commented 1 year ago

@John15321 this issue is about setting the version without changing the source code. I do not believe there is yet an issue for cargo officially having a new subcommand for doing this and would recommend creating a separate issue if that is what you want.

There are also a couple of tools related to cargo bump mentioned in this thread

There is also cargo semver

John15321 commented 1 year ago

@John15321 this issue is about setting the version without changing the source code. I do not believe there is yet an issue for cargo officially having a new subcommand for doing this and would recommend creating a separate issue if that is what you want.

There are also a couple of tools related to cargo bump mentioned in this thread

  • cargo release, particularly cargo release version
  • cargo-edit's cargo set-version

There is also cargo semver

Sorry I did not have that in mind. This issue popped up when I was searching for a solution to changing the version in the toml file using a cli tool, and simply given that I have found one I wanted to share as most likely other people would find this issue as well with the same problem at hand

JosiahParry commented 1 year ago

+1 here. It would be great to have a way to do this. It would be even better if it is not required to publish the crate to bump the crate version.

dead10ck commented 1 year ago

It would be even better if it is not required to publish the crate to bump the crate version.

It is not required to publish a crate to bump its version. All that is required to bump the version is editing the Cargo.toml

kyle-rader-msft commented 1 year ago

Hello, I've found myself here as someone doing a greenfield project at a large company where most of the current tools we build in CI use dotnet's ability to set the version baked into a binary at build time (shown somewhere far above) on the command line. While carefully controlling a semver in the Cargo.toml is ideal - when your team owns too many CLI tools, and needs to ship and consume them automatically, a generated date-based version number becomes very useful. I was quite surprised to find that cargo build --config package.name=1.2.3-1 and similar attempts, don't work for this. As I'm trying to sway folks into writing more tools in Rust, it's a hard up-hill battle at big companies and little things like this result in us having to put other checks in place, reminders to bump versions, and then parse out what it is in CI. In the meantime, I'm hoping the cargo set-version plugin will work, but now we have to install that in the CI pipeline too :(.

juliusl commented 1 year ago

I was curious about this problem so I prototyped how I originally was assuming this would work, which was setting the CARGO_PKG_VERSION_* variables before running cargo commands would override whatever cargo was about to set.

In my prototype I added this before where the CARGO_PKG_VERSION_* variables are set in compilation.rs,

https://github.com/juliusl/cargo/blob/0de91c89e6479016d0ed8719fdc2947044335b36/src/cargo/core/compiler/compilation.rs#L329

Here is what I wrote,

        // Determine if this compilation is for a dependency or for a workspace assembly
        let rpath = self.config.registry_base_path();
        let rpath = rpath.as_path_unlocked();

        // If the manifest path starts with the registry path, then it is a compilation for a dependency
        let version = if !pkg.manifest_path().starts_with(rpath) {
            // Handle pkg_version values
            let pkg_version = pkg.version();
            let major = if let Ok(major) = std::env::var("CARGO_PKG_VERSION_MAJOR") {
                major.parse().unwrap_or(pkg_version.major)
            } else {
                pkg_version.major
            };
            let minor = if let Ok(minor) = std::env::var("CARGO_PKG_VERSION_MINOR") {
                minor.parse().unwrap_or(pkg_version.minor)
            } else {
                pkg_version.minor
            };
            let patch = if let Ok(patch) = std::env::var("CARGO_PKG_VERSION_PATCH") {
                patch.parse().unwrap_or(pkg_version.patch)
            } else {
                pkg_version.patch
            };
            let pre = if let Ok(pre) = std::env::var("CARGO_PKG_VERSION_PRE") {
                pre
            } else {
                pkg_version.pre.to_string()
            };

            semver::Version {
                major,
                minor,
                patch,
                pre: pre.parse().unwrap_or(pkg_version.pre.clone()),
                build: pkg_version.build.clone(),
            }
        } else {
            pkg.version().clone()
        };

As a test I created a small app with this as the main.rs:

fn main() {
    println!("Hello, world!");

    println!("{}", env!("CARGO_PKG_VERSION"));
}

And I initialized with,

cargo init
cargo add tokio

I wanted to add a dependency because I had a suspicion that part of the complexity is that if you handle this by setting an env variable it would apply to all dependencies being compiled. I confirmed that fact and so I handle this issue by checking the manifest path, to figure out if the manifest is from the current working directory or if it was from the cargo registry.

Finally, I tested the above code:

❯ CARGO_PKG_VERSION_PATCH=1004 ../cargo/target/debug/cargo build                                                                                                                                                                                                                                          testver -> master ?
   Compiling pin-project-lite v0.2.13
   Compiling tokio v1.32.0
   Compiling testver v0.1.0 (/home/juliusl/rs/testver)
    Finished dev [unoptimized + debuginfo] target(s) in 1.02s
❯ ./target/debug/testver                                                                                                                                                                                                                                                                                  testver -> master ?
Hello, world!
0.1.1004

I feel like this approach is straightforward enough and is intuitive/aligned with how the rest of cargo works, especially with usage in a CI pipeline.

So, I was wanted to post here and get some feedback and possibly start a PR? Maybe behind a --allow-version-overrides arg action? 👉👈

faern commented 1 year ago

Hello, I've found myself here as someone doing a greenfield project at a large company where most of the current tools we build in CI use dotnet's ability to set the version baked into a binary at build time (shown somewhere far above) on the command line. While carefully controlling a semver in the Cargo.toml is ideal - when your team owns too many CLI tools, and needs to ship and consume them automatically, a generated date-based version number becomes very useful. I was quite surprised to find that cargo build --config package.name=1.2.3-1 and similar attempts, don't work for this. As I'm trying to sway folks into writing more tools in Rust, it's a hard up-hill battle at big companies and little things like this result in us having to put other checks in place, reminders to bump versions, and then parse out what it is in CI. In the meantime, I'm hoping the cargo set-version plugin will work, but now we have to install that in the CI pipeline too :(.

Hi @kyle-rader-msft. We have solved a very similar problem in the following way:

kyle-rader-msft commented 11 months ago

@faern tank you so much sharing that strategy! I need to delve into what's possible with build.rs :) But off the cuff, this sounds much better than my powershell and bash scripts to live update the Cargo.toml during CI... I'll try this out!

epage commented 11 months ago

FYI making the version field optional is effectively approved, see #12786. It comes with the limitation that you can't publish to a registry.

I think @faer's solution is great and was what I was getting at with my earlier comment about separating the role of your product's version from Cargo.toml.

What id like to know if there are people who that solution doesn't work for. Most likely that would be people who have a reason for dynamically setting the version and publish to crates.io.

WhyNotHugo commented 11 months ago

Thanks for tackling this @epage

epage commented 11 months ago

12786 is now merged.

Now that package.version is optional, I feel like this helps us better focus on that field being a tool for registry packages and that it might not be as relevant to products that are not being published to a registry.

In light of that, I propose to the cargo team that we close this issue and instead encourage people to look up the marketing version of their application as they best see fit (e.g. see https://github.com/rust-lang/cargo/issues/6583#issuecomment-1702417194), rather than trying to overload package.version as both the registry version and the marketing version (think Windows build numbers vs marketing version). If there is a reason the registry version needs to be dynamic, let us know!

WhyNotHugo commented 11 months ago

I need the package version to be dynamic (and was previously following #12144).

I use git for version control, so the exact version of my package is stored there, rather than in Cargo.toml. Duplicating the information into Cargo.toml is far from ideal:

Ideally, if the current commit matches a git-tag, then that tag is the current version. If the current commit is 20 commit of the last tag,v0.1.0, then the version would be v0.1.0+20. This is essentially the same idea proposed by op in https://github.com/rust-lang/cargo/issues/12144.

As prior art on this topic, in Python-world I use setuptools-scm, which does exactly this: https://github.com/pypa/setuptools_scm

epage commented 11 months ago

@WhyNotHugo is your version for use in the package registry or is it a marketing version?

If its for marketing reasons, then my question would be "why does it need to be in Cargo.toml at all?".

If for the registry, it has the challenges of

Looking at the above, it feels like we can make something work but that it would have enough caveats and limitations that it wouldn't be up to the expected quality for a cargo feature.

That leaves having some kind of plugin system. At that point, I think of the cost (implementation + maintenance + user support) compared to the benefit. The benefits seems relatively small to me especially when there is the "workaround" of release management tools which we will now be pointing people to as of #12745.

WhyNotHugo commented 11 months ago

There is no "marketing version" or anything alike. Please see https://github.com/rust-lang/cargo/issues/12144 for more in-depth details; what I explained above is covered there in greater detail. I'm just talking about publishing regular libraries and applications written in Rust.

The precedence of setuptools_scm helps but the Python ecosystem doesn't have a native concept of a workspace which further complicates things

Python supports multiple libraries in a single git repository too. For overly complex scenarios, setuptools_scm allow configuring the exact git command used to extract the version. It is perfectly possible to filter tags. E.g.: a package in a multi-package repository might only respect tags prefixed with its name.

Being incompatible with shallow clones when being used as a git dependency which we have active work towards supporting

If somebody is explicitly avoiding version metadata when fetching code, then it is expected that version metadata will be absent. I'm not sure that I understand the issue here.

Looking at the above, it feels like we can make something work but that it would have enough caveats and limitations that it wouldn't be up to the expected quality for a cargo feature.

Let's not let perfect be the enemy of good. The current approach is terrible: someone needs to remember to copy-paste the version into Cargo.toml before tagging a new version. That someone also needs to remember to update the lockfile after updating the version and creating the tag (I've made several bug reports to projects that forget to do this step). Essentially, the version must be specified two and a half times (in concert) for each release.

And builds made from intermediate versions (e.g.: N commits after the last tag) just drag the version from the last tag, rather than having a version that identifies them as distinct.

That leaves having some kind of plugin system. At that point, I think of the cost (implementation + maintenance + user support) compared to the benefit. The benefits seems relatively small to me especially when there is the "workaround" of release management tools which we will now be pointing people to as of https://github.com/rust-lang/cargo/pull/12745.

I do agree that plugins might be an overkill here. Something simpler should definitely be the aim.

I looked briefly into the linked PR, but the suggested tools don't make sense to me. For example, cargo-release takes care of publishing and tagging a release. The usual approach for releasing a new version it to create a git-tag, then push that to CI and CI builds the package and pushes it to a registry or wherever. cargo-release seems to fit perfectly when git isn't you method of version control.

When I tag v1.2.0, I am establishing that the given commit is v1.2.0. The fact that cargo needs to read the value from its own source is an annoyance, because we suddenly need tooling to feed the version from the version control system into cargo. Of course, it's possible to put together a small set of shells scripts to patch this all over the place, but that sounds immensely brittle.

epage commented 11 months ago

There is no "marketing version" or anything alike. Please see #12144 for more in-depth details; what I explained above is covered there in greater detail. I'm just talking about publishing regular libraries and applications written in Rust.

I did look at that and the details I was asking for weren't there, hence why I asked.

The precedence of setuptools_scm helps but the Python ecosystem doesn't have a native concept of a workspace which further complicates things

Python supports multiple libraries in a single git repository too. For overly complex scenarios, setuptools_scm allow configuring the exact git command used to extract the version. It is perfectly possible to filter tags. E.g.: a package in a multi-package repository might only respect tags prefixed with its name.

There is a difference in expectations for a solutions to problems when contrasting python's "here are the tools, do it on your own" and cargo's "here is a first-class, native solution with a smooth workflow". We can learn from Python's experience but the Python experience is not optimized for the problems we'll have (it its janky).

Let's not let perfect be the enemy of good. The current approach is terrible: someone needs to remember to copy-paste the version into Cargo.toml before tagging a new version.

I agree with the sentiment generally but taking it too far you gloss over fundamental problems which I think is happening here.. Without solving path dependencies, the problem hasn't gone away unless you restrict this to just single packages. That is a pretty dramatic restriction. Besides likely leading to more frustration when a feature is accessible sometimes but not always, it makes it unclear if a solution that gets stabilized will scale to workspaces. We are then stuck with supporting that solution for ... forever, increasing the load for the cargo team.

. For example, cargo-release takes care of publishing and tagging a release. The usual approach for releasing a new version it to create a git-tag, then push that to CI and CI builds the package and pushes it to a registry or wherever. cargo-release seems to fit perfectly when git isn't you method of version control.

Let's not over-generalize. That is your workflow but I've not seen that generally applied in the Rust community. Instead, I've seen a lot of people fear release automation to the point that they don't even want to adopt a local-only tool that is dry-run by default and optionally logs everything it does to the screen.

DavidAntliff commented 4 months ago

@epage:

Now that package.version is optional

Maybe I'm missing something, but this, with cargo 1.78.0, doesn't seem to behave how I'd expect after reading this entire conversation:

  1. Remove [package] version = "x.x.x" from Cargo.toml
  2. cargo clean
  3. Build with --config: 
    $ cargo build --config 'package.version="1.2.3"'
Compiling my-program v0.0.0

    I would expect the newly set version number to be applied as the build version.

I'm also using clap with the automatic -V/--version feature, and after building with this, it prints:

$ my-program --version
my-program 0.0.0

I would expect this to reflect the package.version value, as it 100% would if it was set in Cargo.toml instead.

This doesn't seem like "set the package version dynamically" is working? There's a good possibility I've messed something up though.

FWIW, my use case is that I need the version of my program, built in CI, to be the same as other programs built in the same pipeline (monorepo), so rather than modifying Cargo.toml with a sed script, I'm looking to pass the overall version to each program in the pipeline definition. This works fine for Python packages, but I'm having trouble with doing the same for Rust builds. I'm not pushing anything to crates.io BTW, it's all in-house.

Also, I should mention that the version number (used across the monorepo) comes from elsewhere in the pipeline, it's not determinable in the Rust project context - it must be injected.

If a sed script is current best practice, so be it.

EDIT: maybe cargo-edit is the recommended way to do this now?