NuGet pack should not fail for release packages with prerelease development dependency

lostmsu commented 9 years ago

Currently, release package can not have prerelease development dependency. For example, that prevents us from using StyleCop.Analyzers package currently in beta in any release package development,

Internal alias: vimilova

yishaigalatzer commented 9 years ago

That's currently by design. This prevents shipping release packages with pre-release dependencies accidentally.

We can consider adding a flag to override this behavior (and if you want to provide a PR for that we will gladly take it). For now though, I don't see this popping up to the top of our priority list.

Your workaround would be to edit the nuspec after the package was packed, or create a pre-release package with the analyzer.

In general its a pretty bad idea to include an analyzer with your implementation package anyways. You want that in a top level "meta" package that will bring both the implementation and the analyzer, now that package can totally be pre-release.

zidad commented 8 years ago

I'd be interested to see a way to override this. We internally we use nuget, and not being able to create release packages with beta dependencies is holding us back at the moment.

GeertvanHorrik commented 8 years ago

I would need something like this as well (since we use NuGet to ship extensions, we don't care about the prereleases). An option (like -allowprereleasedependencies) to override would be great.

asbjornu commented 8 years ago

I find it very strange that when upgrading a package with -Pre, that flag isn't also cascaded to dependencies. It's very natural for prerelease packages to have prerelease dependencies, so I don't really understand the reasoning here. Although this issue is for packaging, I expect it to be related to the upgrade process, no?

The "by accident" argument is in my opinion only valid if the package is stable. If the package is prerelease, it should by definition be able to have prerelease dependencies without any upgrades to it breaking. As it stands now, you need to do -IgnoreDep alongside -Pre if the package you upgrade to have prerelease dependencies. Excuse my french, but that's just stupid.

NikGovorov commented 8 years ago

I have same issue, it would be perfect to have the flag that would allow to override default behaviour.

peterlai-roboops commented 8 years ago

This is pretty bad design, for example we name our version with the branch it came from, so it makes tracking easier. This is deemed a pre-release version as it has letters in it, and this means we can't package another with an internal package.

FransBouma commented 7 years ago

I also ran into this this morning. I have 1 main package and several depending packages. All depending packages reference the main package, so take a dependency on it. As everything is now in prerelease I can't reference the depending packages as nuget can't find the main package, because I use as dependency version range: (>= 5.1.0 && < 5.2.0), and the main package is in prerelease and has as version 5.1.0-beta-20161027. There's no way to specify this! I really don't want to alter the nuspec files to include some sort of beta version number because that is horrible to automate as these packages are generated by a script.

If A and B are in prerelease, and B depends on A, and I want to add the package B to my project and I specify '-pre', nuget should be able to get A, using the normal versions and append the prerelease suffix, i.e. should really look for 5.1.0.* as it's a prerelease ! I now have no idea how to solve this.

Thanks @asbjornu for pointing out -ignoredep, that at least made me able to install the package.

yishaigalatzer commented 7 years ago

Hey Fran's,

Is your issue that you want pack to just allow the pre-release in the nuspec OR you want resolution to work even though you specified a stable dependency?

Yishai

peterlai-roboops commented 7 years ago

To be honest, it doesn't really matter what the issue is because there's are so many reason one would name their packages which falls foul of the "release" naming standard, that there really should be a flag to allow us to override this behaviour.

And it's a pretty big behaviour, whether to be allowed to package our code or not.

FransBouma commented 7 years ago

@yishaigalatzer

I have A and B, both prerelease, B depends on A. They're generated from scripts which get the version to work with passed to them. Therefore the nuget packages have '-beta-xxxx' as suffix in their filename, but not in their versions. So nuget packages for A and B are:

sd.xxx.A.5.1.0-beta-xxx.dll sd.xxx.B.5.1.0-beta-xxx.dll

but internally the version is set to 5.1.0, and dependencies of B, as it depends on A, is set to (>=5.1.0 && < 5.2.0).

If I now install B in my project, and specify -pre, I'll get the prerelease. I want nuget to then also consider the packages B depends on as prerelease, so it can find them. In my scenario above, installing B prerelease, would then work as it would find sd.xxx.A.5.1.0-beta-xxx as that's a prerelease too and the version matches the version specification of the dependency in B.

Today however, I can't install B at all unless I specify -ignoredep, as nuget searches for the release package 5.1.0 of A, even though B is prerelease, because the dependencies in the nuspec of B refer to >=5.1.0 and for nuget 5.1.0-beta-xxx is not matching that criteria.

yishaigalatzer commented 7 years ago

That's because 5.1.0-betaXXX is indeed lower than 5.1.0. Try specifying 5.1.0-beta as the min dependency.

FransBouma commented 7 years ago

Yes but that's not what I want as that would mean I have to update the nuspec files every time I build a package, so it's not really automate friendly. This is especially unfriendly for packages which have multiple builds per alpha-beta cycle.

yishaigalatzer commented 7 years ago

That's not what I'm suggesting.

1.0.0-beta < 1.0.0-beta1 < 1.0.0

So specify 1.0.0-beta verbatim.

Note that A. We cannot make a change to the resolution here, it's a massive breaking change. B. This is not the same issue the bug is discussing, so if you do want to keep debating this issue, let's take it somewhere else

FransBouma commented 7 years ago

Sorry, but how is what I ran into not the same as this issue? It's a side effect of the prerelease factor not being used for resolvement. If prerelease is seen as prerelease across the resolution of dependencies, it would search for prerelease packages too, which is IMHO the issue discussed here. If not, then this whole system is too convoluted to begin with as I have no idea what on earth it is discussing.

I also disagree with your 'massive breaking change' as it currently simply doesn't work. breaking something that doesn't work today isn't breaking anything.

I'm sorry I'm a bit harsh on this, but I can't believe I have to hand-edit nuget spec files now just for alpha-beta releases and remember to roll back these changes when RTM hits as the automated system of course simply rolls on and doesn't care about this.

There's NO documentation on this either. I searched, I ran into some SO questions, with vague references to old issues but that's it.

yishaigalatzer commented 7 years ago

This bug is about the ability to pack a release package with a pre-release dependency, not about resolution.
I gave you a method that is not requiring editing of dependencies
Changing resolution rules IS a breaking change, and we are not going to consider taking one here.
We can clarify the versioning order in docs, it will happen shortly.
There are numerous ways to automate creating the correct dependencies, and they should change if your dependencies change. In this case it will mean the nuspec in the end package should change between pre-release and release, it could change as you rev your pre-release depenednecies (more on that a later), you could keep it constant by requiring a static 1.0.0-alpha or 1.0.0-beta as a minimum as it will match the range.
Back to revving the min Version - it's a choice you need to make. In my projects I believe in revving it and in same cases making it strict to avoid collisions from refactoring and other breaking changes that happen a lot more often during pre-release. The policy here is something you should determine because you own the development process of these depenednecies.

peterlai-roboops commented 7 years ago

but how a package is determine to be pre-release isn't that good. If it has a word in it then that's it, it's a pre-release.

Your basically determining how every company should name their package via this one feature.

yishaigalatzer commented 7 years ago

Yes we are determining what versions look like, that's what standardization is for, it makes everyone understand the same concept and enables tools to operate on a common ground.

We support semver 1.0.0 and just released initial support for semver 2.0.0 in NuGet 3.5.

Http://www.semver.org

peterlai-roboops commented 7 years ago

yes you are! but i hardly think its for you to determine how we want to use and build internal packages that we have no intention of sharing.

If there are genuine reasons to change it, e.g. some new tools that requires it and if we think it's worth the adaptation then we might change our mind and adapt to it, but it's not for you to decide how our company should name our packages.

FransBouma commented 7 years ago

@yishaigalatzer Fair enough. I understood it is about the same issue as the mechanics are really the same: how to determine which packages match the criteria. Whether the starting package is prerelease and wants to refer to prerelease dependents too or release package wants to refer to prerelease or even prerelease which wants to solely refer to release packages, that's just semantics on the same feature: you have an assembly B which refers to A and B as certain characteristics and A as another set of certain characteristics, and the rules which packages match the criteria set by B's characteristics + dependency specifications so A does or doesn't match is currently too strict. Be it release B referring to prerelease A, prerelease B referring to prerelease A or prerelease B referring to release A (which is automatic), that doesn't matter: they're a set of rules. Currently they only work with hard-coded version numbers, they can't deal with characteristics of the packages: if a package is in 'prerelease' that's something the rules can't deal with. If they did, then the issue at hand, be it release->prerelease, or prerelease->prerelease would not be a problem.

This isn't really 'changing' the resolution rules, it's making them more flexible if they have to be. They're currently hard-coded and can't be changed. If they would have been more flexible, the OP would be able to specify what was required, and people like me would have been able to do that too, with the same change. Additionally, others out there who don't care about this aren't affected.

I must say I hate the 'solution' to alter the nugetspec files with a hardcoded 'beta' reference in the dependency, especially because it's something in a file I have to take care of before RTM so change the file manually before the automated system will spit out a set of assets (we're not talking 1 package here, but many), and because they're all versioned together, so B refers to A, and they're versioned together. To me a pretty default concept, but it's not possible to use this, other than hard-coded version references. As it's not documented either, it tells me the nuget team hasn't thought about any of this as something that people would do.

asbjornu commented 7 years ago

I understood it is about the same issue as the mechanics are really the same: how to determine which packages match the criteria. Whether the starting package is prerelease and wants to refer to prerelease dependents too or release package wants to refer to prerelease or even prerelease which wants to solely refer to release packages, that's just semantics on the same feature

I completely agree with all of the above. 👍

I must say I hate the 'solution' to alter the nugetspec files with a hardcoded 'beta' reference in the dependency, especially because it's something in a file I have to take care of before RTM so change the file manually before the automated system will spit out a set of assets (we're not talking 1 package here, but many), and because they're all versioned together, so B refers to A, and they're versioned together.

Yep. We have many solutions with projects being packaged up as .nupkg where each project in the solution depends on another, all up to a .Core project at the top. When these projects are published as pre-release packages, they are basically impossible to install without adding -Pre -IgnoreDep when upgrading them.

To fix this according to @yishaigalatzer's description, it will require a lot of monkeypatching on each and every package to fix the dependency chain and remove any version number after the pre-release label.

yishaigalatzer commented 7 years ago

No one said editing the nuspec is ideal, I'm saying the nuspec inside the package has the right representation. Definitely a pain point that needs a discussion, on a separate issue.

The breaking change is very real, the pre-release version will install instead of the rtm one if it stays on the feed, when installing with minimal depenednecies mode (default).

FransBouma commented 7 years ago

The breaking change is very real, the pre-release version will install instead of the rtm one if it stays on the feed, when installing with minimal depenednecies mode (default).

No, as what I suggested (and which solves the point in the OP as well) is a set of rule parameters which configure what Nuget should do in several situations. The current behavior can stay as-is. With additional parameters nuget can behave a little different, i.e. the rules get additional input / different input parameters and can with that decide whether a package P does or doesn't fall into the set of packages matching the criteria. This doesn't break anything as the current behavior doesn't change. It only offers additional behaviors based on input parameters for the rules (which are currently strict and not configurable to solve the points addressed in this thread).

yishaigalatzer commented 7 years ago

I beg you, start a new thread and jot out your suggestions in concrete terms

FransBouma commented 7 years ago

Why another thread for a solution that covers this thread too, as the problem of this thread is actually caused by the same cause as the problem I described, namely: rules which are too strict for a given situation?

asbjornu commented 7 years ago

@yishaigalatzer: The problem we're discussing does not seem to be isolated to what this issue describes in its title, so can't the title of this issue just be edited instead? The problem is the dependency resolving algorithm, which is behaving the same way for both nuget pack and nuget update / install.

FransBouma commented 7 years ago

EDIT The hotfix argument I made isn't really valid, as one can specify 'highest path' filter option with nuget and get a matching dependent package. For beta and custom versioning this isn't going to fly of course.

yishaigalatzer commented 7 years ago

Frans, I agree hand editing is awful. Brain storming ideas, will post when I have a good one

Sent from Ninehttp://www.9folders.com/

From: Frans Bouma notifications@github.com Sent: Nov 2, 2016 3:54 AM To: NuGet/Home Cc: Yishai Galatzer; Mention Subject: Re: [NuGet/Home] NuGet pack should not fail for release packages with prerelease development dependency (#1365)

@yishaigalatzerhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fyishaigalatzer&data=02%7C01%7Cyigalatz%40microsoft.com%7Caffa5d26598e4491e46308d4030ea40c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636136808798392840&sdata=LFm072YoBemHlcycP6jVgHXz4uCvqwxI7HoPu160lEU%3D&reserved=0 handediting nuspec isn't an option actually. Every time we release a hotfix it's the same problem: I have to manually alter 12 nuspec files by hand, build using automated system, deploy using automated system, then go back to the 12 nuspec files and handedit them again back to normal so the final minor version will again be without 'hotfix'. The only option to automate this is to generate the nuspec files using a tool as there's no config option to pass in a value.

Flexibility is what we're after, so we can tailor the process to our needs, be it release referencing prerelease, prerelease referencing prerelease, prerelease referencing release.

You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FNuGet%2FHome%2Fissues%2F1365%23issuecomment-257832601&data=02%7C01%7Cyigalatz%40microsoft.com%7Caffa5d26598e4491e46308d4030ea40c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636136808798392840&sdata=zAabnkH%2BFymmu7fMbU7kO%2FVpQJCRhQ9zkXCpGZJIkPw%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABLmt2JN_sLTvzdz51VN4Vwc02H37HPjks5q6GvsgaJpZM4F6rTu&data=02%7C01%7Cyigalatz%40microsoft.com%7Caffa5d26598e4491e46308d4030ea40c%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636136808798392840&sdata=cVYojSEc5y5en3Ohb1kNwy%2BQeCmChnyyo4hqFduwZHI%3D&reserved=0.

peterlai-roboops commented 7 years ago

just give us the option to override the behaviour.

We should be able to decide what's best for us, if what we decide doesn't work with some other tool then that's our own fault and we will correct that, but you shouldn't be forcing it on us.

I just don't get why there's push back on this, the users are saying we 100% need this but for some reason you know best. Generally, what happens is someone else release a product that does do what the user want and we all move to that. e.g. maven, years we been telling them it's too rigid, and now gradle and sbt is taking over.

haf commented 7 years ago

@yishaigalatzer Can we override it yet? This is real pain for me when I have a single target in Logary that depends on a pre-release package but the suite of targets as such isn't a pre-release. (Also, the idea that you can't transitively depend on pre-release packages and be non-prerelease is wrong)

enricosada commented 7 years ago

Seems i got lots of good company there (hi @haf).

I have the same problem, and is really annoying. It's just about dotnet pack, not resolution on restore.

Make it a warning if you really want, but not an error, you cannot know why i am using a prerelease deps. Or add an argument to override or make it a warning (noisy but still).

mythz commented 7 years ago

Just got hit with this issue, I'm trying to build a .NET Core MySql package with nuget pack where the only .NET Core package of MySql.Data available is a pre-release package.

Things I tried to get it to build a NuGet package with a -pre release package dependency:

<dependency id="MySql.Data" version="[7.0.6-ir31, )" />
<dependency id="MySql.Data" version="7.0.6-ir31" />
<dependency id="MySql.Data" version="[7.0.6-ir31]" />
<dependency id="MySql.Data" version="[7.0.6-ir31]" developmentDependency="true" />

What worked:

None of them. We had to change our CI build itself to run a special-cased custom script for just this one NuGet package which looks like:

nuget pack ServiceStack.OrmLite.MySql.Core.nuspec -version 1.0.%version%-pre

hopefully this helps someone else as this workaround was not obvious

IMO they all should work, I'm clearly asking for the pre-release dependency. If I happen to accidentally publish a package with a pre-release dependency that would clearly be my fault for making a hard-coded reference to a pre-release package. In which case the fix would be the same as every other bad NuGet package, i.e. delist it and publish a new version. I personally never publish pre-release packages and thanks to running into unintuitive non-standard behavior like this, I never will.

enricosada commented 7 years ago

@yishaigalatzer @rrelyea any update on this? Please is possible to add a workaround to unblock us? Like:

an env var NUGET_SUPPRESS_ERR_1234 (temporary fix but easy without impact on anyone and no api/cli changes) until a real solution is choosen
downgrade the error as warning.

I'll implement anyone of that and send PR if is approved. Or just tell what you want.

This issue is really annoying and common becuase lots of netstandard lib are prerelease atm but is ok for my lib, because as author I know that.

For f# lib is a big issue too becuase

Was not like that in preview2
The mandatory microsoft fsharp.core package is prerelease probably until VS2017 RTM

yishaigalatzer commented 7 years ago

We should be able to take a pr for this. I agree we should prioritize it otherwise

enricosada commented 7 years ago

@yishaigalatzer i'll send a PR tomorrow. What do you prefer? Downgrade as warning or env var or something else? Ihmo preferred is downgrade as warning

yishaigalatzer commented 7 years ago

Yes but with a way to clear the warning or warning as error

Sent from Ninehttp://www.9folders.com/

From: Enrico Sada notifications@github.com Sent: Dec 11, 2016 1:20 PM To: NuGet/Home Cc: Yishai Galatzer; Mention Subject: Re: [NuGet/Home] NuGet pack should not fail for release packages with prerelease development dependency (#1365)

@yishaigalatzerhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fyishaigalatzer&data=02%7C01%7Cyigalatz%40microsoft.com%7Ca80b03a5c1704e19797208d421b7a94e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636170520093848777&sdata=QOwBQcDE4VzO%2B8qM4wu3LaWfqt3571QMTvaCzSgQfDQ%3D&reserved=0 i'll send a PR tomorrow. What do you prefer? Downgrade as warning or env var or something else? Ihmo preferred is downgrade as warning

- You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FNuGet%2FHome%2Fissues%2F1365%23issuecomment-266276336&data=02%7C01%7Cyigalatz%40microsoft.com%7Ca80b03a5c1704e19797208d421b7a94e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636170520093848777&sdata=GYl2Nw9elEx2WDlRqpu2Iq1%2Bx%2Fxv7lBbK09rYgH8MqE%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FABLmt69CQA3YNM1pjyXwlJBJDkKTUY8Lks5rG9xmgaJpZM4F6rTu&data=02%7C01%7Cyigalatz%40microsoft.com%7Ca80b03a5c1704e19797208d421b7a94e%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C636170520093848777&sdata=SmbBrcpLBK%2B%2F2l4UHtIC%2BOQjE4u%2BuM2E5p290MK5Nhs%3D&reserved=0.

lostmsu commented 7 years ago

I think it should be a warning.

peterlai-roboops commented 7 years ago

OMG, was that all that was needed to advance this? we do the work and you approve? Its fantastic that this is being advanced but I kinda wish this was said months ago.

enricosada commented 7 years ago

didnt forget about it, i'll send a PR max this weekend

Joe4evr commented 7 years ago

@enricosada So where's that PR now?

rohit21agrawal commented 7 years ago

This has already been fixed in the latest nightly builds. The error has been downgraded to a warning

https://github.com/NuGet/NuGet.Client/pull/1073

BrainCrumbz commented 7 years ago

Is that nuget.exe available through some channel to be downloaded, especially for using it from a CI environment? Thanks

haf commented 7 years ago

Wow, amazing turn-around time there!

pantonis commented 7 years ago

I have the same problem on my CI. I am using nuget 3.5.0. It is a pain. What version of nuget includes this fix?

shytikov commented 6 years ago

Ok, wait, someone uses this functionality on production and actually expects things to fail in case if pre-release dependencies exists.

I believe that behavior should be configurable. Now what is done is just breaking behavior with no workaround. How we supposed verify that package is allowed to be called stable?