Open timotei opened 8 years ago
@emgarten
"Normalization is by design. Versions..."
Is this your comment on @StingyJack's assessment?
@emgarten I am not talking about some custom script going crazy. I am talking about plain Visual Studio Nuget install and restore. Let me elaborate the situation I faced yesterday:
I used the VS2017 Nuget UI to update a package from 9.4.326.0 to 9.5.429.0. Within the packages .nuspec nothing changed except the version string.
The new package got downloaded into packages\name.9.5.429.0\
. Everything built fine both in my machine and on our build server.
A college checked out my code and opened it with VS2015. On package restore, the package got downloaded into packages\name.9.5.429\
and the reference was broken.
He issued a -Reinstall
from his package management console and the package was reinstalled into the same directory but the reference in the .csproj
was fixed. That broke the build on VS2017 and thereby on the build server.
And here comes the strange part:
packages\name.9.4.326.0\
. We double checked the changes in the .nuspec
-file, but there are none except the version string.
We didn't find a method yet, to install 9.5.429.0 in a way, so that we can use it in both VS 2015 and 2017.
Two other colleges face the same issue at the moment with System.Windows.Interactivity 4.5.0(.0).
So I am not talking about normalizing or not, I am talking about having the a consistent behavior with the last two (fully updated) versions of Visual Studio.
And I think it is fair to assume that it is a bug that the .0 gets skipped on one package version, but not on another.
Most users in this thread are having problems due to powershell get unnecessarily extracting and re-packing packages just to copy them from one feed to another. This is an issue with powershell get. Re-packing a package will always lead to some changes.
@emgarten - @KevinMarquette is an undeniable powershell powerhouse, but is the only one here who mentioned powershell. I know that reading can be hard or tedious sometimes, but there are 12+ other people in this thread. Most of us are reacting to having someone else's versioning scheme forced upon us without notice, and without giving any consideration to its effects or any workaround or recourse.
SemVer is a good idea (for nuget.org), but that doesn't make it appropriate for everyone. Its not something to be wielded like Maslow's Hammer, or followed like a cargo cult.
We removed using Semver as well here at work. Too much complication for too little gain.
Sent from my iPhone
On Mar 24, 2018, at 7:17 AM, Andrew Stanton notifications@github.com<mailto:notifications@github.com> wrote:
Most users in this thread are having problems due to powershell get unnecessarily extracting and re-packing packages just to copy them from one feed to another. This is an issue with powershell get. Re-packing a package will always lead to some changes.
@emgartenhttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Femgarten&data=02%7C01%7C%7Cb8a911ae29894874ff0e08d591899309%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636574942414851351&sdata=Ng8a0OJelHd5%2FZVMov43jhHRzqVtpi5Qw%2B4OUxwlOXo%3D&reserved=0 - @KevinMarquettehttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FKevinMarquette&data=02%7C01%7C%7Cb8a911ae29894874ff0e08d591899309%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636574942414851351&sdata=qqOccXN%2Fr95gJXr3pbL4U8kblAmYedhtHJP2W9RWRT0%3D&reserved=0 is an undeniable powershell powerhouse, but is the only one here who mentioned powershell. I know that reading can be hard or tedious sometimeshttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FNuGet%2FHome%2Fissues%2F6534%23issuecomment-363422986&data=02%7C01%7C%7Cb8a911ae29894874ff0e08d591899309%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636574942414851351&sdata=o5FPeLdmn9bN8et1UHe6MWB%2F3bcgMwgWW85Fycy7Nck%3D&reserved=0, but there are 12+ other people in this thread. Most of us are reacting to having someone else's versioning scheme forced upon us without notice, and without giving any consideration to its effects or any workaround or recourse.
SemVer is a good idea (for nuget.orghttp://nuget.org), but that doesn't make it appropriate for everyone. Its not something to be wielded like Maslow's Hammer, or followed like a cargo cult.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FNuGet%2FHome%2Fissues%2F3050%23issuecomment-375888685&data=02%7C01%7C%7Cb8a911ae29894874ff0e08d591899309%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636574942414851351&sdata=vHFj4h1gUSo9t7rcvF6LKDskBbNHOA6KODVulPhIFlI%3D&reserved=0, or mute the threadhttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FACyJCujXhrZhTJk9ZvbDrp4ldOYknNdOks5thkdegaJpZM4I-_5n&data=02%7C01%7C%7Cb8a911ae29894874ff0e08d591899309%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636574942414851351&sdata=Ft3%2BdrIYNRhJW9ZNkS5CneBpN0lfjKm21hfoko8Lmk4%3D&reserved=0.
Speaking for myself and for the System.Data.SQLite project, which has approximately 8 million downloads on NuGet.org, we will never use any newer version (of NuGet) that forces all versions to be "normalized", as forcing SemVer upon our users would be too much of a breaking change.
Versions must be treated as Semantic Versions, not as strings.
So NuGet only supports semver? Four digit version numbers like Explicit Versioning are not supported?
@emgarten
Would you comment on @StingyJack's assessment of your position:
"This reads as 'We broke it, but aren't willing to fix it. One of you can.' Is that really what is being said?"
Four digit version numbers like Explicit Versioning are not supported?
@fabich NuGet supports four digits such as 1.2.3.4, that hasn't changed. This issue covers what happens when pack is used on an assembly version such as 1.0
or 1.0.0.0
and how it is normalized since they are the same version according to semver rules. Note also that having 1.0.0.0
in a nupkg is still fine, this issue is purely around how pack transforms the package if you create it using the latest tools.
Would you comment on @StingyJack's assessment of your position:
@InteXX I answered your question earlier here: https://github.com/NuGet/Home/issues/3050#issuecomment-374833832
And again, a pull request for this would be great! 🍰
Details on what is needed for a switch here: https://github.com/NuGet/Home/issues/3050#issuecomment-343800795
@emgarten
We're on the heels of two years with this very serious issue.
Your silence (and inaction) on the problem are deafening.
@emgarten
"@InteXX I answered your question earlier here: #3050 (comment)"
And your response was unclear that it was intended for me. I asked for clarification and received no reply.
"Versions must be treated as Semantic Versions, not as strings"
Two questions:
@InteXX would you share your exact scenario where you are having problems?
@emgarten
"would you share your exact scenario where you having problems"
Certainly. I'd love to.
Would you answer my two questions first.
@emgarten
"I asked for clarification and received no reply."
In addition, in that answer you didn't comment on @StingyJack's assessment of your position. Your response may have been an answer, yes, but not to the question I asked.
@emgarten - there are a few specific problems here.
By your posture and position, we have come to know that the Nuget team has chosen to dismiss the (slightly odd but venerable) 30+ year old expected four part version numbering system. This could be and in some cases is a breaking change, and in other cases its unexpected and unwanted due to the work it has the potential to create.
I don't think I should need to state the obvious here, but Its breaking because tools, process, and artifacts expect 4 parts. What happens when you "1.2.3".Split('.')[3]
? Or when going through the Big Pharma vendor audit or validation wringers and having to write up a deviation because one of the components that go into the software is actually version 2.3.4
instead of the expected 2.3.4.0
?
If you (and the rest of MSFT) want to use semver for your stuff, and there are no regulatory compliance, legal, or other procedural issues (both process changes and document template changes) with doing so, and it helps you make better stuff, by all means go for it. However, "good for you" does not mean "good for me". I'm the guy who has an unexpected broken build because tooling cant find the expected outputs, or crashes outright with an IndexOutOfRange, and then has to deal with the hotseat and extra paperwork during system acceptance testing and validation.
@emgarten
"And again, a pull request for this would be great!"
You broke it, you can fix it. Have you considered how much you've cost us with this breaking—and arguably unnecessary—change of yours?
If I sound like I'm getting a bit frustrated here, it's because I am.
It's frustrating to attempt polite, reasonable discourse, only to encounter silence and obfuscation.
@emgarten "Versions must be treated as Semantic Versions, not as strings"
I'll also ask Why? I manage a couple hundred packages- anything from .net console apps to internal websites to SSRS report bundles and even a dozen Access front ends. SemVer is pretty much pointless- there's little cross package dependency, so I'd end up with a bunch of 1.0.XXX packages.
I use a Git -> TeamCity -> ProGet -> Octopus Deploy pattern for everything (Including Access!) What I find much more useful is a yyyy.MM.dd.BuildCount format, i.e. 2018.06.07.44 While this is covered under the "Legacy" support, it's still stripping the leading zeros on the month/day.
As @mistachkin said- I authored a version value of "2018.06.07.44", and that's what I expect to come out, not "2018.6.7.44", and I am certainly not interested in the "SemVer" way of 1.0.0+2018.06.07.44
@emgarten
Would you comment on @StingyJack's assessment of your position:
"This reads as 'We broke it, but aren't willing to fix it. One of you can.' Is that really what is being said?"
@emgarten
"would you share your exact scenario where you having problems"
Certainly. I'd love to.
Would you answer my two questions first.
@StormRider01 is this for style purposes, or are your tools having trouble consuming 2018.6.7.44
over 2018.06.07.44
?
I can see how you could consider the 0 padding to be data, but the same thing applies to XML for the nuspec. If you format the nuspec file one way, and after pack whitespace has been removed due to formatting, would you consider that a bug?
The nuspec is XML and should be used with tools that understand the XML format. It isn't safe to assume to treat the nuspec contents as a string. In the same way, the contents of the version element are of SemVer format, and it has more meaning than just a string.
@emgarten
The pervasive question that seems to be giving you significant trouble is Why?
Why must you shove SemVer down our throats, in the face of us begging you not to? What problem does it solve that's worse than the problem it's creating?
Is this someone's mere personal preference that you're forcing upon us? What issues did the design team face when considering this decision? Did they consider the trouble they'd be causing us?
Your silence is very frustrating, Justin.
@emgarten it's certainly a style purpose, but it also creates a disconnect- TeamCity's build has 2018.06.07.44
, but Octopus sees it as 2018.6.7.44
. I don't do any automation against Octopus, so I don't know if that would cause a tooling problem.
As far as the nuspec being XML, well, here's the .xsd
<xs:element name="version" maxOccurs="1" minOccurs="1" type="xs:string" />
So yes, it's a string. Explicitly.
If it was an absolute requirement for version to be SemVer in XML, I would expect this:
<version major="1" minor="2" patch="3" pre-release="Alpha" meta="commit3a445b3c2f" />
With the .xsd:
<xs:complexType>
<xs:simpleContent>
<xs:extension base="xs:string">
<xs:attribute type="xs:int" name="major"/>
<xs:attribute type="xs:int" name="minor"/>
<xs:attribute type="xs:int" name="patch"/>
<xs:attribute type="xs:string" name="pre-release"/>
<xs:attribute type="xs:string" name="meta"/>
</xs:extension>
</xs:simpleContent>
</xs:complexType>
</xs:element>
@StormRider01 thanks for the example, I can see wanting to keep these versions in sync if TeamCity is producing the version string here.
This thread has been going on for 2 years this month and I would like to see this draw to a close. All questions have already been answered here ad nauseam in my opinion.
Summary of this thread: Version normalization does not work in some scenarios where external tooling is expecting a different format, or where different stylistic choices are desired.
Current state: Normalization does work for the majority of .NET project scenarios, and it has drastically reduced the number of problems seen from conflicting versions such as 1.0 vs 1.0.0.0. It has helped make things deterministic, and improved performance since the nupkg path on disk can be calculated.
Going forward: To solve both of these problems a switch should be added to pack that keeps the version as-is. -VerbatimVersion
. This would be a minor change to the pack command.
I'm happy to see the passion for NuGet package creation in this thread, and I would like to see it go towards a pull request for fixing this. I would be happy to review this pull request, but I am no longer going to watch or participate on this issue.
If no one is willing to submit a pull request for -VerbatimVersion
then I think this issue should be closed.
@emgarten
Please review the issue history. Much of the delay has been due to your own silence and obfuscation. We've posted problem report after problem report, with little to no response.
When there does come a response, it's in the form of telling us how we're doing it wrong. There's been no consideration offered toward the problems that this breaking change is causing; there's only been refusal to address the issue.
And you still haven't told us why you broke it in the first place!
So you've delayed and obfuscated for two years, and now you want to close the issue and ignore the problem you caused. Put our shoe on your foot for a little bit—would you enjoy being treated like this?
Why did you do this to us? What problem did the breaking change solve that's worse than the problems it's creating?
To break this for no apparent reason and then ask us to fix it is disingenuous at best, downright rude at worst.
Do I sound frustrated? Yes. You've frustrated me.
@emgarten
All questions have already been answered here ad nauseam in my opinion
They certainly have not. Anyone reading this thread can recognize this.
Here is a pull request. Please review.
There is now a custom build of NuGet that addresses the issue under discussion, signed with my EV Code Signing certificate, available here. It is a "Debug" build because I was unable to figure out the necessary steps to make the "Release" build work. The PDB file is included.
I was referred to this issue by PowerShell/PowerShellGallery#55 because it seems that even Microsoft's own PowerShellGallery contributions are affected by unwanted normalisation in the .nuspec
file as opposed to the .psd1
files.
Prominent examples that I've come across include:
SqlServerDsc.nuspec
says 12.3.0
, SqlServerDsc.psd1
says 12.3.0.0
NetworkingDsc.nuspec
says 7.0.0
, NetworkingDsc.psd1
says 7.0.0.0
xActiveDirectory.nuspec
says 2.24.0
, xActiveDirectory.psd1
says 2.24.0.0
... but there are countless PowerShell/DSC packages on PowerShellGallery with the exact same issue. This is getting out of hand.
I've hit this problem with two separate tools now – Artifactory and ProGet. Apparently they use the .nuspec
file to get the package version – to me this doesn't seem unreasonable, but I am happy to be corrected.
As a result, PowerShell will pull packages into C:\Program Files\WindowsPowerShell\Modules
naming the folder after the .nuspec
version, and then decides you can't import the module because the .psd1
has a different version number!
If you want semantic versioning to be enforced for packages, then fine, NuGet should hard-fail when a package version number does not comply with semantic versioning.
What it should absolutely not do is take the version from somewhere else, mangle it and hide the issue.
Incidentally, -VerbatimVersion
in NuGet/NuGet.Client#2374 does not really fix the problem either. It is yet another attempt to hide the issue and not really solve it, especially because I am wondering just how many developers are out there not even realising that NuGet is doing this to their packages before they publish them.
Clearly not even Microsoft's own people realise this either, or else there would not be so many packages on PowerShellGallery that are broken in this way.
We'd prefer to keep everybody normalizing.
@rrelyea - We'd prefer to have the version numbers be used as they were specified as the added behavior is...
Removing this coercion to your team's definition of semver (this was a breaking change that was introduced and did come with a major version incrementation of nuget - irony?) would stop the breakage that is happening with packages and would abate the loss of time wasted troubleshooting these breaks.
The nuget team at MSFT wanted this normalization to make dealing with packages on nuget.org easier, but as far as I have read here and elsewhere did not fix a breakage somewhere.
This should be reverted for the simple reason that QOL improvements cannot come at the cost of breakage to users' software or pipeline.
@StingyJack
I see the good work you've contributed in your PR remains unaccepted. Obtuse describes the response pretty well, especially from @emgarten (who seems to have left the conversation). The warden from Shawshank Redemption comes to mind.
I'd almost consider a PR fixing the original break and making normalization opt-in, but I doubt it'd gain much traction.
@InteXX - Thanks, but I actually wont submit a PR for this because I think there is a community negative behavior that needs to be addressed, and the pain of having to undo this change should be enough to correct that behavior.
Per his profile, @emgarten works on Azure IOT stuff now. For this issue, I dont know if he was griefing (again), spouting a party line, or both, but I let him off the hook when I saw that he moved on and have tagged @rrelyea (NuGet PM) instead.
From what I understand, the normalization was done to make dealing with packages @ nuget.org easier to manage. To me this means that any specific versioning rules need to be enforced for packages that go to nuget.org, but only for that. Any and all private repos should not be beholden to that rule, and if anything there should be a -NugetOrgPublishRules
switch that performs the version coercion and any other nuget.org specific checks at pack time.
To me this means that any specific versioning rules need to be enforced for packages that go to nuget.org, but only for that
Right – and this train of thought is what also led me to raise PowerShell/PowerShellGallery#55 so that version numbers can be validated on the point of upload to PSGallery, rather than in the actual packaging process itself. I have no idea why NuGet mangling version numbers by itself is acceptable, especially when the process is invisible to the user and has resulted in invalid metadata being published to countless packages on PSGallery.
Any and all private repos should not be beholden to that rule, and if anything there should be a -NugetOrgPublishRules switch that performs the version coercion and any other nuget.org specific checks at pack time.
Exactly. Fix the default behaviour and make this semver-specific behaviour optional, rather than the other way around.
/cc: @rrelyea
The Octopus team also had to work around this issue: https://github.com/OctopusDeploy/Issues/issues/2809
Facing this issue when I replaced the 4th part of the version string of https://www.nuget.org/packages/WizardWrx.AnyCSV/ with a zero, to eliminate multiple versions of a given build in the repository, I devised a workaround in the form of a three-line custom task, of which only the first is strictly required.
Since the correction can be accomplished by a single C# statement, the task is implemented inline as a fragment.
` <UsingTask TaskName="NuGetPackageNameFixup" TaskFactory="CodeTaskFactory" AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.Core.dll">
<Task>
<Code Type="Fragment" Language="cs">
<![CDATA[
ActualNugetPackageVersion = RawNugetPackageVersion.EndsWith ( ".0" ) ? RawNugetPackageVersion.Substring(0,RawNugetPackageVersion.Length-2) : RawNugetPackageVersion;
Log.LogMessage ( "Custom Task NuGetPackageNameFixup: RawNugetPackageVersion = "+ RawNugetPackageVersion );
Log.LogMessage ( "Custom Task NuGetPackageNameFixup: ActualNugetPackageVersion = "+ ActualNugetPackageVersion );
]]>
</Code>
</Task>
` The implementation is straightforward.
` <Target Name="Publish" AfterTargets="Package" Condition=" '$(Configuration)' == 'Release' ">
<Exec WorkingDirectory="$(PackageDir)"
Command="NuGet.exe push $(AssemblyName).$(NuGetPackageVersion).nupkg" />
`
The task has one input parameter and one output parameter, and the two Log.LogMessage
statements are optional.
Since I had never done such a thing before, I spent several hours puzzling over why the task appeard to run, yet the push command had an empty string where I expected to see the output. The trick is that the Output
tag inside the NuGetPackageNameFixup
task block is required to convert the output into a MSBuild environment variable.
My next step is to move the custom task and the target that consumes it into my custom C# targets file, so that my other package builders can consume it.
@txwizard - that seems like a lot of work and complexity just to get the specified version number respected.
@rrelyea - is this two year old issue with 85 comments and 137 thumbs ups siding with the version normalization being unwanted and less than 20 thumbs ups siding with the POV defending the version normalization really going to continue to linger as a sore spot, or will the corrective work* be scheduled soon?
* The correction being to revert the normalization from nuget core and enforce it for nuget.org and not require us to add an additional flag or parameter to continue what should be a compatible behavior. It does not matter how obscene or offensive anyone on the nuget team thinks non semver version numbers might be, Microsoft has had 4 part version numbers for like 40 years now. We are all using 4 part version numbers for everything. Using a 3 part number for this one part of our development ecosystem sometimes creates more work for us, and having a usually 4 part number show up as a 3 part number can become a ridiculous problem in some regulated and [validated ](https://en.wikipedia.org/wiki/Validation(drug_manufacture)#The_validationprocess)industries.
@StingyJack,, thanks for the backup.
@txwizard - that seems like a lot of work and complexity just to get the specified version number respected.
It seems that the issue has suddenly been elevated to Priority 1, as I believe it should be.
Not only was devising that task nontrivial, but making it part of the build script is even more work, which remains to be done. I would love to be able to set that thankless task aside, and find some better-tasting fish to fry.
@txwizard
It seems that the issue has suddenly been elevated to Priority 1
Please clarify: do you mean within the NuGet team? If so, where are you seeing the indication?
People in the Windows PowerShell DSC space are also suffering from that issue quite a lot. With the current way how NuGet forcefully normalized the versions we are unable to upload DSC resources to internal NuGet repositories. NuGet normalized the version in the package but not in the PSD1 file where PowerShell keeps track of versions. This mismatch causes the module to be unavailable when downloading it from in internal NuGet feed.
Actually I like the idea of SemVer but the force with how it is imposed makes live really hard. I can follow the arguments of people that version number should not be changed when packaging a piece of software. At least I would like to have a fallback to keep things as they have been.
I am involved in a number of IaC / DSC projects and this issue has cost me and the customers way too much time. From a customer perspective this change has made the whole way of providing software with NuGet unpredictable and overly complicated.
@anangaur - @rrelyea probably muted this issue long back and may need an IRL tag
@anangaur as I pointed out twitter 3 months ago, fixing this was effectively pocket vetoed.
@emgarten spent far more time arguing against fixing this on this issue as well as on a PR: https://github.com/NuGet/NuGet.Client/pull/2374 wearing it down to the point of @mistachkin giving up and closing the PR.
@rrelyea didn't publicly comment on the PR when @jainaashish asked for comment last year, so clearly something's going on, and it's not out in the open.
Microsoft's response to this problem they've created and maintained has been a clear lesson on how NOT to behave (toward one's customers).
Just asked on the powershell issue: "Is there somebody from powershell gallery that could engage with the nuget team to discuss the variety of options to fix this?"
@rrelyea - "put it back the way it was" should cover the necessary work for all of those affected instead of just for one group of your choosing. Then you can add your nuget.org specific restrictions
@rrelyea
@StingyJack is correct.
@InteXX , I may have misinterpreted the labels shown in the upper right corner of the issue page, where I see a Priority 1 label. In any case, has anyone in this thread reviewed https://github.com/NuGet/Home/issues/8159?
@rrelyea Is the forced SemVer behavior going to be rolled back or not?
Also tagging @aortiz-msft since this item has had a Triage discussion
I see this issue has dragged on for nearly 4 years, without an end in sight. There must be some powerful forces wanting to ram SemVer down everyone's throats.
Hello,
It seems latest NuGet pack ignores the version set in the nuspec (or via the
-Version
override flag) and the dependencies' versions, by normalising them, even if nobody asked for it :(It seems this was "fixed" for this https://github.com/NuGet/Home/issues/2039, but to me this seems a super counter-intuitive change, that breaks (at least for us) the daily workflows.
For the meantime, we had to do a workaround to "fix" the versions in the generated nupkg. I'm hoping that this can be fixed either by removing this weird logic, or allowing the nuget pack command to generate nuget package which contains the same versions as in the nuspec.
I'm using
NuGet Version: 3.4.4.1321
.With this nuspec:
executing
nuget pack mylib.nuspec -Version 1.9.0.0-SNAPSHOT
generates this .nuspec:As we can see:
1.9.0-SNAPSHOT
instead of1.9.0.0-SNAPSHOT
(even if we specifically overrode it via the command line) - same happens with a non-SNAPSHOT version3.16.0
instead of3.16.0.0
mylib.1.9.0.0-SNAPSHOT.nupkg