search

NuGet / Home

Repo for NuGet Client issues
Other
1.5k stars 253 forks source link

Support pre-release packages with floating versions. Version="*-*" and similar. #912

Closed pranavkm closed 4 years ago

pranavkm commented 9 years ago

UPDATE: final solution details

Original Issue: Consider a project.json specification such as

{
   "A: "1.0.*"
}

If the only packages in the feed are prerelease packages such as A.1.0.0-alpha, A.1.0.1-beta, the restore fails stating it was unable to resolve package A.

[anangaur] Adding the requirement as evolved during the discussions:

As Noel, who uses NuGet packages in PackageReference based projects,

Link to spec - https://github.com/NuGet/Home/wiki/Support-pre-release-packages-with-floating-versions Status - Implemented in version 5.6 of NuGet, 16.6 of Visual Studio and 3.1.300 of the .NET Core SDK.

yishaigalatzer commented 9 years ago

We can discuss this for 3.2

yishaigalatzer commented 8 years ago

I looked at how NPM does it, and it looks like it will not allow pre-releases with such a specifier.

We need to work on how we extend our syntax to support ranges (do we go with something similar to npm? Do we also allow 1.*-*? Do we allow for version ranges ala packages.config/nuspec?

pranavkm commented 8 years ago

Do we also allow 1.-

AFAIK it stops when it sees the *. @emgarten would know this better.

yishaigalatzer commented 8 years ago

Yes it does, my comment was a question about what we should do in the future :)

CZEMacLeod commented 8 years ago

Wondering if there is any progress on this as for CI purposes it would really help us to support 7.0.*-* or something like

{
      "version": "7.0.*",
      "prerelease": true
}
yishaigalatzer commented 8 years ago

Not yet, a good indication of progress would be if the item is assigned to someone for work.

CZEMacLeod commented 8 years ago

Unfortunately that's what I thought... Just been open for over a year and bumped again and again... :(

spotlesscoder commented 7 years ago

Please implement this

ghost commented 7 years ago

So this ticket is still in Priority 1 and has been bumped for a couple of years.

We use CI and end up spitting out a lot of packages, the maintenance of keeping up with them is only just worth the value of keeping everything separate. Being able to specificy "Latest prerelease" in our dev branches would really be very useful.

Could we get a vague timeline at least? Maybe pointers on where to start looking at developing a pull request?

emgarten commented 7 years ago

This needs a design to move forward.

The problem is that changing the behavior of * is a breaking change, and adding a new syntax such as 1.0.*-* will break older clients that do not know how to parse it.

Feedback and suggestions on this would be great. I would really like to see this issue fixed.

ghost commented 7 years ago

Alright so let's talk design.

So, is making the syntax 1.0.0-* mean get the prerelease version that is the highest number actually a breaking change? And do we need to support any other variations on that?

CZEMacLeod commented 7 years ago

For our use case, we would like the ability to mix 1.0.* with CI builds - this would result in something like 1.0.*-* or 1.0.*-ci*. This is particularly important for CI builds based on CI builds in our framework, where if we bump a component by 0.0.1 we still want the latest CI build brought into all the other builds - especially for unit testing etc. to make sure that no downstream packages fail. This can be done (sort of) with project references, but our framework is >100 projects and can be a pain to manage that way.

emgarten commented 7 years ago

1.0.0-* as syntax already present in the system? Does it just not function as intended?

1.0.0-* works but does not go above 1.0.0

ghost commented 7 years ago

1.0.0-* works but does not go above 1.0.0

As in, it just resolves the lowest available version? That seems to be what this documentation indicates. We attempted to use this in Visual Studio and were seeing syntax errors, but that might be us using it wrong.

1.0.*-* would be useful.

Okay, fair enough. @emgarten You indicated messing with * syntax would be considered a breaking change, what's the timetable and process that it would take to introduce that? Could we confirm/polish the behavior of a full specific version with floating prerelease in a faster timetable than if we were to fully implement arbitrary version and prerelease?

danielgreen commented 7 years ago

What about using a different symbol than * to indicate a wildcard that includes release and prerelease versions?

E.g. 1.0.%

mkstephenson commented 7 years ago

Is there any update on this? We're in the process of migrating a bunch of projects to NuGet and this is still affecting us, over 2 years after it was first reported.

CZEMacLeod commented 6 years ago

@emgarten Since you say that changing this parsing is a breaking change (I assume you mean at the client side) perhaps we could introduce something similar to the way that nuget packages have a min required client but for projects? (similar to the minClientVersion in nuspec) Then you could have a BeforeRestore target that checked for $(NuGetClientVersion) (or similar) and throws an error if it is too low or does not exist (e.g. for old tools that would not handle the extended wildcards correctly) and ensure that nuget pack and msbuild, vs etc. set the property to the installed client version. Perhaps along the lines of

<Target Name="EnsureNuGetClientVersion" BeforeTargets="Restore" Condition="'$(MinNuGetClientVersion)'!=''">
    <Error Text="You require version $(MinNuGetClientVersion) or higher of the NuGet tools installed to correctly restore this project"
        Condition="'$(NuGetClientVersion)'=='' OR ($([System.Version]::new($(NuGetClientVersion)) &lt; $([System.Version]::new($(MinNuGetClientVersion)))" />
</Target>

Then in your .csproj or Directory.Build.props or whatever simply add something like

<PropertyGroup>
    <MinNuGetClientVersion>4.5.0</MinNuGetClientVersion>
</PropertyGroup>
emgarten commented 6 years ago

@CZEMacLeod that could work, currently the version of NuGet isn't a property in MSBuild, but potentially a task could find the assemblies and check them.

anangaur commented 6 years ago

Why can’t we have something like the following:

<PackageReference Include=“MyPackage” Vesrion=“2.*” IncludePreRelease=“true”/>
CZEMacLeod commented 6 years ago

@anangaur I suggested that (although at the time it was in json rather than msbuild) over a year ago!

forki commented 6 years ago

People may want to restrict versions to specific prerelease channels like rc. See https://fsprojects.github.io/Paket/nuget-dependencies.html#Prereleases for how Paket solves that

emgarten commented 6 years ago

@anangaur we had VersionRange.IncludePreRelease in 3.0.0 which is similar to what you are proposing. We eventually removed it due to the very high number of bugs it caused. Version ranges work best as strings that can be round tripped, having a string + extra metadata that cannot be part of the string is painful.

jainaashish commented 6 years ago

I think we can go ahead with 1.0.*-* which is only supported with newer clients. With older clients, it will still treat it as 1.0.* which should be fine. thoughts?

forki commented 6 years ago

Older clients will probably choke on that.

anangaur commented 6 years ago

@jainaashish

To me 1.0.*-* means any package with pre-release version and starts with 1.0. This will exclude the stable versions of the package. Unless we want to go away from general regex concept.

I thought the problem statement was to be able to define a floating version that includes pre-release versions (by not excluding the stable versions).

anangaur commented 6 years ago

To reiterate the requirements the way I understood:

As Noel, a user who uses NuGet packages,

Looking at the above and to solve the problem in non breaking way for existing clients, additional meta data looks to be the right approach. @emgarten, I would like to understand the issues with this approach (the bugs you mentioned)

emgarten commented 6 years ago

additional meta data looks to be the right approach.

It needs to be possible to express a version range as a string. It makes it easy to display, understand it as a user, and the current code base treats these as strings everywhere.

I agree with @jainaashish, 1.*-* is the clearest representation of this. Excluding stable versions and forcing prerelease would be a very unusual scenario, I think this notation is pretty clear in that it allows prerelease and stable. I think this would be a fairly small fix to enable this.

Support for specific labels would mean that the range would no longer be contiguous, it could be added but I think that would be a different feature.

emgarten commented 6 years ago

Older clients will fail on 1.0.*-* but the alternative is that they parse 1.0.* and then fail to read the include prerelease bool, which will make them silently fail to resolve the package. In my opinion this is worse than failing fast when they can't understand the range.

forki commented 6 years ago

"It needs to be possible to express a version range as a string"

That is an arbitrary requirement.

Am 30.11.2017 23:07 schrieb "Justin Emgarten" notifications@github.com:

Older clients will fail on 1.0.- but the alternative is that they parse 1.0.* and then fail to read the include prerelease bool, which will make them silently fail to resolve the package. In my opinion this is worse than failing fast when they can't understand the range.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NuGet/Home/issues/912#issuecomment-348336325, or mute the thread https://github.com/notifications/unsubscribe-auth/AADgNCrAiAHyKuvevRhg_BNPCPqfWa5iks5s7ycMgaJpZM4FVctT .

anangaur commented 6 years ago

I agree with @jainaashish, 1.- is the clearest representation of this. Excluding stable versions and forcing prerelease would be a very unusual scenario, I think this notation is pretty clear in that it allows prerelease and stable.

@emgarten , @jainaashish Correct me if I am wrong here: Doesn't regex 1.-\ mean a string that starts with 1. followed by any string and then a - and then followed by any string. So it would mean only prerelease versions. I am treating to mean one or more occurrence of any character. (Most regex parsers treat as one or more occurrence of the preceding character which definitely is not something we are following here). Also I am assuming . (dot) is treated as the character dot and not "any character".

emgarten commented 6 years ago

I think it is important to look at these strings as Semantic Versions which have meaning, not as opaque strings. Treating them as regular strings and applying a regex misses a lot of that meaning, and the reasons why we treat stable and prerelease differently, and the intent of the package authors when they versioned things in this way.

We can think of a floating range as two parts:

  1. A range that specifies which versions are allowed
  2. A target within the allowed range to aim for, this is the preferred version that restore will try to get to, or as close to as possible

Stars are used to express the target for part 2. The decision to allow for prerelease or not is made by part 1.

Here is how I would read the ranges: 1.0.* - I want the highest stable patch 1.0.*-* - I want the highest stable patch and I'm ok with allowing non-stable packages to satisfy this constraint

Stars indicate floating to the highest version, and by SemVer rules 1.0.1 is higher than 1.0.1-rc. I can see how by normal wildcard matching it might appear that a - is required. But in my opinion a user would never want to take any prerelease version but not the stable version, and if they did they could use 1.0.1-* which would give 1.0.1-rc.

CZEMacLeod commented 6 years ago

@emgarten My use case would certainly be for something like 1.0.- as above - as you describe it.

1.0.*-* - I want the highest stable patch and I'm ok with allowing non-stable packages to satisfy this constraint

Moreover - I think it would be likely that I might need any stable or non-stable patch package but to constrain it to be above a certain release. e.g. I need any 1.0 release (or pre-release) after 1.0.4-rc1 and be happy (and want to get) 1.0.4-rc2, 1.0.4, 1.0.5-beta1 or 1.0.5 etc. Could this not be covered by something like [1.0.4-rc1,1.1) as it stands? If so isn't 1.0.*-* effectively just [1.0.0-*,1.1) ? E.g. any pre-release/unstable version from 1.0.0 up to (but not including) 1.1?

patricknolan commented 6 years ago

Hi @emgarten has there been any progress or more thought given to this issue?

I need a way for the latest version to always be used regardless of release or pre-release.

So if a package had the following versions available it would use 3.9.3091-rc. At present, if I reference 3.9.* it always uses the latest release version which is 3.9.3088.

It would be very useful if I could reference3.9.*-* in order to get 3.9.3091-rc

3.9.3091-rc 3.9.3090-rc 3.9.3089-rc 3.9.3088-rc 3.9.3088 3.9.3087-rc

forki commented 6 years ago

@patricknolan as a workaround until nuget introduces a way to handle prereleases without * - take a look at https://fsprojects.github.io/Paket/nuget-dependencies.html#Prereleases

emgarten commented 6 years ago

@patricknolan this is set as pri 0 for NuGet, we're hoping to get this added soon.

patricknolan commented 6 years ago

Thanks @emgarten that's great news. I think this issue has been a large wide spread problem and people have just been forced to implement a compromised solution or some painful hacks to get around it.

myermian commented 6 years ago

Reading through all these messages, I'm still a bit confused. I'm hoping that 1.0.*-* would get the prerelease version. In the following, 1.0.2-ci-2 should be selected:

1.0.3 1.0.2 1.0.2-ci-2 1.0.2-ci-1 1.0.1 1.0.1-ci-1

An example of this working out well would involve using a Choose/When... When we are compiling in debug mode, get the highest prerelease version (1.-) which is non-breaking. When we are compiling in release mode, get the highest stable version (1.*) which is non-breaking. It's just one scenario... but the key here is this, we need a way to specify to get the highest version which also happens to be a prerelease package.

anangaur commented 6 years ago

I find the - notation confusing. Instead we should have an explicit flag. In the above example, IMO 1.0.2 is the highest version of the package matching the floating version as prerelease versions are considered lower versions compared to the same versioning without prerelease portion of the versioning.

Hope I didn’t confuse you further?

gwalschl commented 6 years ago

I agree with @patricknolan that this is a very important feature for many of us. I too have the exact same use case as he does and would be extremely grateful if this would be implemented ASAP. We are transitioning from using packages.config to new PackageReference in projects and would like to transition to using NuGet wildcard versioning that meets stated use case. We are holding up this transition until there's a way to overcome this. Thanks!

anangaur commented 6 years ago

@pranavkm I would like to change the description to the following, with your permission. Do let me know if the below covers your usecase and if you are okay with the change?

As Noel, who uses NuGet packages in PackageReference based projects,

pranavkm commented 6 years ago

Sure, our team has abandoned the use of floating versions for our projects, so I'm not entirely vested in this particular feature. Feel free to rephrase it!

gwalschl commented 6 years ago

Any movement towards a solution on this issue?

anangaur commented 6 years ago

This is my proposal for the requirements listed earlier. Open to comments/scrutiny:

[Open]: In the above case whether the prerelease version texts rc, beta be given equal weight-age or alphabetical weight-age? i.e. if 2.8-beta.9 also existed whether NuGet should resolve to 2.8-beta.9 (weightage given to the number after the pre-release text) or 2.8-rc.2 (alphabetically rc > beta as r > b)

anangaur commented 6 years ago

@forki what's the behavior for paket for the open issue above?

forki commented 6 years ago

Rc is newer than beta is newer than alpha. Good thing is alphabetical order works here

gwalschl commented 6 years ago

The suggested package resolution logic looks good. Lots of flexibility the way it's laid out.

rrelyea commented 6 years ago

We care about this. And hope to get to it soon.

anangaur commented 6 years ago

There were some disagreements on whether there should be new attribute to specify the ability to float to pre-release versions. So just thinking aloud here, will the following be an acceptable choice? @jainaashish @nkolev92 @rrelyea

From readability and ease of use perspective, I still like the IncludePrerelease attribute :)

nkolev92 commented 6 years ago

I don't see the reason for introducing a new attribute. That makes things more complicated and adds implementation difficulty.

I don't believe there's an ease of use argument here with the prerelease attribute.

Version ranges already include prerelease packages. 

<PackageReference Include=“MyPackage” Vesrion=“[2.0.0, 3.0.0)"/>

can already resolve to a prerelease package. Adding that new metadata would conflict with that.

Additionally, I don't see the point of introducing a new separator like 

|

when

-

already exists and it's more intuitive since that's how versions are written.

Personally, I am a fan of 

<PackageReference Include=“MyPackage” Vesrion=“2.*-*”/>

Which would basically say, give me highest, even it's prerelease.

I don't see the scenario of, I want rc AND beta versions for example, as that high of a priority, since you should be able to do:

<PackageReference Include=“MyPackage” Vesrion=“2*-rc*”/>

to get latest, stable or rc version.

tldr;

jainaashish commented 6 years ago

why do you need alternate approach?

<PackageReference Include=“MyPackage” Vesrion=“2.*|”/>

I like the additional parameter idea, that seems right which could work across old vs new clients. So we should stick to that.