Expand `--version` option to allow range

dtgm commented 8 years ago

Summary

Currently, issuing upgrades of packages restricted to a major version requires a unique id. This is typically done by appending the version to the id such as with community dotnet packages, e.g. dotnet3, dotnet4, etc.

Proposal

Use nuspec versioning (as already used in nuspec files to determine valid dependency version) to select version with choco. This could be for install/upgrade/pin etc.

Current

--version=VALUE
Version - A specific version to install.

Proposed

--version=2.0, --version=[2.0]
Version - Select exact version.

--version=[1.0,2.0)
Version - Select most recent version from a range >=1.0 && <2.0

--version=(,2.0)
Version - Select most recent version from a range >0.0 && <2.0.

For example, if you wanted to restrict install of a package to version 9.0 Just like when declaring a dependency in *.nuspec:

<dependencies>
  <dependency id="example" version="[9.0,10.0)" />
</dependencies>

one time use: choco install example --version=[9.0,10.0) stored use (pin): choco pin add -n=example --version [9.0,10.0)

Adding #798 would be a very nice complement to this.

Adding to template? On one hand, it clarifies the situation, on the other, it may encourage people to a bunch of packages that should be deprecated when the feature gets implemented. I mean right now we have teamviewer, teamviewer7, teamviewer8, teamviewer9, teamviewer10.... I would see a lot of packages needing to the same. Also, it can be hard to declare a dependency sometimes as sometimes packages get created with also minor versions, dotnet4.6 and dotnet4.6.1. So a package declaring dotnet4.6 as a dependency will not get upgraded since 4.6.1 is a different package (saw that when I was looking at paint.net). But I can easily see this affecting businesses just as well that want a easily maintainable upgrade channel for supporting different versions among their different environments.

How does nuget handle interpreting the dependency xml element? Can we leverage what they are doing there? We would probably want to separate out interpreting the version and call on that as required to retrieve the latest version within the provided parameters(?)

ferventcoder commented 8 years ago

Bigger discussion - .NET 4.6 and 4.6.1 may be side by side installs.

dtgm commented 8 years ago

^^^ Widens scope for sure if needing to handle multiple versions of same id, which dotnet would. That should probably get logged as a new feature enhancement that would build upon this one.

Requiring upgrade to specific major version is relatively easy
Requiring upgrade to specific versions AND allowing multiple versions may be more difficult depending on how -m is implemented

-m, --sxs, --sidebyside, --side-by-side, --allowmultiple, --allow-multiple, --allowmultipleversions, --allow-multiple-v
rsions
    AllowMultipleVersions - Should multiple versions of a package be
      installed? Defaults to false.

-m doesn't work for dependency packages, right?

For example, a dependent on c version ==1.0 b dependent on c version ==2.0

I think the behavior is if b is installed, a will still install, but c will remain as 2.0

There would have to be a way to follow the dependencies strict with -m recursive option, which would work for dotnet which installs the versions to different registry keys. But would be bad if not used correctly by packages as allowing side-by-side packages to install is meaningless if the package just reinstalls to the same registry key.

ferventcoder commented 8 years ago

Our recommendation is that for when the natively installed app can have two versions of it installed, that is two different packages. So .NET Framework 3.5 versus .NET Framework 4.5. Those are different installs, thus different packages.

dtgm commented 8 years ago

I agree and it should continue like that for now, but also ... stop and dream a little, man!

Imagine a world where all the dotnetX maintainers worked together.... Imagine a world where packages depended on dotnet and specified version required....

Most programs just need at least a certain version of installed, say dotnet >=4.0 So if dotnet4.6 is installed, well, gonna need to install dotnet 4.0 too because it is a separate package, and that builds a needlessly bloated system.

But many programs also do require a specific version so it's not a solution to merge them all until there is some mechanism to say, "Hey, dependency X, I know we already have a version of you installed, but we need you too, bruh."

Requiring side-by-side install of latest release in 4.5.x.x

<dependencies>
  <dependency id="dotnet" version="[4.5,4.6)" force="yes" />
</dependencies>

So by adding an attribute force (or whatever) to dependency element, that dependent package would be forced to be installed side-by-side (-m), rather than prompting an upgrade per usual....BUT that would still introduce bloat, only more slowly, if even one package misused this property since it would be way more difficult to determine no longer relevant dependent packages. But that is a job for a dependency cleaner that I am just going to very conveniently assume that is already well developed and will magically handle all of that.

jberezanski commented 8 years ago

I've been thinking about converging the .NET packages for quite a while now (as hinted here). There are exactly two packages needed, one for each .NET "family": 2.x/3.x (2.0, 3.0, 3.5, 3.5.1) and 4.x (4.0, 4.0.1, 4.0.2, 4.0.3, 4.5, 4.5.1, 4.5.2, 4.6, 4.6.1, 4.6.2). Both families are independent of each other and install side-by-side. Within each family, each new release completely overwrites/supersedes the older ones, while remaining backward-compatible.

Back on topic: I fully support this feature suggestion. It would help greatly in cases where subsequent versions of a commercial package have different licensing, for example a license is valid only for a specific major version (e.g. Resharper - there is currently no way to tell choco "I have a license for 9.x, so please install the latest release before 10.0").

ferventcoder commented 8 years ago

@jberezanski I agree the convergence is helpful. I also think this is a great feature suggestion. 👍

dtgm commented 8 years ago

@jberezanski @ferventcoder Forgot I did this meta-analysis of package families with distinct installers awhile back. Specifically, looking at the possibility of combining them and how might be best to specify "editions" to install, or at least having a consistent naming in use https://github.com/dtgm/chocolatey-packages/issues/172

svmastersamurai commented 8 years ago

👍 To this idea. This would be handy for those of us that use chocolatey in a configuration management setting where we want to enforce a bare minimum baseline for a chocolatey package version but allow users to upgrade at their leisure.

In particular, Chef's chocolatey_package provider gets real upset if I specify a version and the package versions don't match since even if you are more up-to-date than what I say the program exits with a non-zero exit code and then the Chef resource throws an exception and causes the converge to fail.

Having this built into chocolatey itself would mean Chef wouldn't really have to touch the code for their provider at all since we could just update the version string we use in our cookbooks and let chocolatey handle the check for us.

Either that or throw an expected non-zero exit code with an additional flag on the command line that we can trap... but this feature just seems way cooler ;)

ferventcoder commented 7 years ago

Related to #1278 as well as #688

felixfbecker commented 7 years ago

Personally I feel like semver ranges as used by npm, Composer etc. are more popular and more intuitive

ferventcoder commented 7 years ago

@svmastersamurai min version with Chef (or other config mgmt) is typically achieved by having the min version in your sources and then using latest or whatever Chef uses. But then upgrade paths over time when tbe min changes (might be achieved in the same way), but you only want the upgrade if it is below the min. I agree that is a tougher nut to crack. I bet the Chef resource could support a min_version property. You might ask Chef to see what they think.

pklapperich commented 5 years ago

Maybe I'm dense, but I don't see the point of having a range. It seems sufficient to support =, '<', and <= like other package managers. If you choose a range [2.0,3.0) that's really the same as <3.0. If you choose a range [2.0,3.0] that's really the same as <=3.0.

In what circumstances would you want to define a range and NOT have the package manager choose the newest package that satisfies that range? Newest satisfying will never need to look at the bottom end of the range, so < and <= are sufficiently expressive.

ferventcoder commented 5 years ago

I don't personally know all of the reasons and constraints folks might have, so I'm not really into limiting them because I don't necessarily understand their needs.

If you choose a range [2.0,3.0] that's really the same as <=3.0.

@pklapperich I mean, it's close, but 1.0.0 would also satisfy <=3.0 where it would not satisfy the constraints of [2.0,3.0]. This behavior:

Have 1.0.0 installed. Specify upgrade with <=3.0 - constraint is met, nothing to do
Have 1.0.0 installed. Specify upgrade with [2.0,3.0] - constraint is not met, upgrade is performed to the highest available
similar with <3.0 and [2.0,3.0).

In what circumstances would you want to define a range and NOT have the package manager choose the newest package that satisfies that range? Newest satisfying will never need to look at the bottom end of the range, so < and <= are sufficiently expressive.

I think if the upgrade is performed, it would absolutely choose the latest package that satisfies the range, but choosing whether to action or not is really where the range comes in. It's possible that they are sufficiently expressive, but I can't say I understand everyone's situation.

pklapperich commented 5 years ago

Have 1.0.0 installed. Specify upgrade with <=3.0 - constraint is met, nothing to do

Wait, what? According to the proposal, this should upgrade if a version newer than 1.0.0 is available. That's how all other package managers I've used work (apt, yum, npm, etc).

This statement makes me concerned that setting a version range won't even solve the common use case I see (ie, avoid a bleeding edge version, but still get bug fixes). For example, I pin Node to [8.0,9.0). I have 8.12.0 installed. Node pushes a security update 8.12.1. I ask choco to upgrade and choco says "derpy derp, nothing to do. Constraints are met." On my non-windows systems I either have a package for Node 8.x.x or I have node pinned to <9.0 and I get updates whenever I run the upgrade/update command.

I can understand a need for ranges in package dependencies (ex a package depends on [2.0,3.0)) but I still don't see a reason for ranges on either pinning or the command-line. I definitely want 'upgrade' to fetch the newest within the constraints.

The proposal above is

--version=[1.0,2.0)
Version - Select most recent version from a range >=1.0 && <2.0

--version=(,2.0)
Version - Select most recent version from a range >0.0 && <2.0.

For both of these cases, (select the most recent in range) I would expect it to pick the same package unless the latest package were less than 1.0 in which case the top would give an error and the bottom would install something. The bottom end of the range seems of little use.

bcurran3 commented 5 years ago

@pklapperich

Not to be a doggy downer here, but...

Maybe I'm dense, but I don't see the point of having a range. It seems sufficient to support =, '<', and <= like other package managers. If you choose a range [2.0,3.0) that's really the same as <3.0. If you choose a range [2.0,3.0] that's really the same as <=3.0.

Chocolatey is built on top of NuGet v2. With that stated, the range (ranging?) is already baked in. ; i.e https://docs.microsoft.com/en-us/nuget/reference/package-versioning so I think this "argument" might be with the wrong people. :)

The original opened issue is about choco.exe processing an install range via command line seems to be an enhancement to actually use more of that possible/available range (ranging?) system.

Of course, I could be misunderstanding something, but that's where @ferventcoder steps in.

EDIT: Re-reading your last post, could your request/"argument" be summarized as when a package is already installed and an upgrade request is made, if the package is less than the highest available within the range, Chocolatey should update the package to the highest version within the range if it isn't already? That seems like desirable behavior to me and might need a new issue opened to make it happen.

pklapperich commented 5 years ago

@bcurran3

Chocolatey is built on top of NuGet v2. With that stated, the range (ranging?) is already baked in. ; i.e https://docs.microsoft.com/en-us/nuget/reference/package-versioning so I think this "argument" might be with the wrong people. :)

NuGet allows the ranges for package dependencies, which I said make perfect sense. There are reasons why a package maintainer might need to specify a range, and chocolatey should not prevent that. As far as I know, NuGet's cli --version option only allows a specific version, just as chocolatey does now.

But this feature request also includes the command line options for choco, where I don't really think range-start makes sense, only range-end.

From the command line, I'm always telling choco to operate on the named packages. That means choco doesn't have to decide whether or not it should perform an operation; it should or I wouldn't have listed the package name in my request... IE per the proposal in the issue description:

choco install --version (,8.13) packagename
choco install --version (6,8.13) packagename

choco upgrade --version (,8.13) packagename
choco upgrade --version (6,8.13) packagename

all 4 of these commands result in the same outcome: the system now has the latest version of packagename that's less than 8.13 (either it was installed or it was upgraded). The command result was the same with/without the 6. Specifying the 6 was pointless.

So why not just just use --version <8.13 and --version =<8.13? It's less typing and has the same result.

Package dependencies SHOULD be expressive and support ranges. But from the command line, ranges don't provide any benefit that I can see; it's just extra characters to type. I haven't yet thought of a scenario where a cli operation includes --version [y,x] and the bottom end of the range should have any effect.

That seems like desirable behavior to me and might need a new issue opened to make it happen.

That's already in the original proposal of this issue, so it seems like it belongs here. But maybe this issue should be split into 2 issues?

kshanafelt commented 5 years ago

I can foresee a reason where the bottom end of a range could have an effect

choco install --version (6.0,8.13) packagename

Means I require the latest version of packagename greater than 6.0 but less than 8.13.
So if the latest version my sources can come up with is version 5.8, Chocolatey should not install it.

Just like if I were to run this today:

choco install --version 6.0.0 packagename

and if the latest version of packagename my sources can come up with is version 5.8, Chocolatey fails to install it, as it should.

Also I see no reason why

choco install --version <=8.13 packagename

couldn't by an alias for

choco install --version [,8.13] packagename

This could be a valid use case if someone has multiple consumers with access to different Chocolatey sources (or different permissions on those sources).

For example, I could have an Alpha-Testers, Beta-Testers, and Pre-Release-Testers source, and if I let my consumers know that my changes will require them to have packagename version [4.0,5.12]

Then maybe the Alpha-Testers grab 5.12, the Beta-Testers get 4.0, and the Pre-Release-Testers get a message from Chocolatey that their sources cannot meet this version requirement.

And every one of them was able to do this running the same command.

choco install --version [4.0,5.12] packagename

Chocolatey not installing an unneeded version of a package is a much better user-scenario then installing a version that won't work for their use case.

mvorisek commented 5 years ago

The choco can lookup all versions like choco list php --by-id-only -a (or with `-e switch once https://github.com/chocolatey/choco/issues/1843 is solved).

Why are currenly no versioning constraints supported?

I understand that is will need some modifications of the code and the data structures. For simplicity I would recommend starting with support only for the choco install command, i.e. accept version like [7.3, 7.4), choco the latest matching version and simply install it the found/fixed version. This will allow very effective package management with tools like Ansible.

God-damnit-all commented 5 years ago

This would be very useful for Python. A leap from 3.7 to 3.8 will break a lot of things, and older versions of Python still receive updates. I'd really like to pin it to --version=3.7.* for instance.

Jackenmen commented 5 years ago

@ImportTaste a good temporal solution for this is creating your own meta package:

Create a folder python37-pin

Inside the folder create a file python37-pin.nuspec with contents:

<?xml version="1.0"?>
<package xmlns="http://schemas.microsoft.com/packaging/2011/08/nuspec.xsd">
<metadata>
<id>python37-pin</id>
<version>3.7</version>
<title>Python 3.7</title>
<owners>jack1142</owners>
<authors>Python Software Foundation</authors>
<requireLicenseAcceptance>false</requireLicenseAcceptance>
<description><![CDATA[Python 3.x is a programming language that lets you work more quickly and integrate your systems more effectively. You can learn to use Python 3.x and see almost immediate gains in productivity and lower maintenance costs.]]></description>
<dependencies>
  <dependency id="python3" version="[3.7,3.8)" />
</dependencies>
</metadata>
<files>
</files>
</package>

Open terminal prompt in python37-pin folder and use command choco pack to make a python37-pin meta package.
Install the package with: choco install python37-pin --source "'.;https://chocolatey.org/api/v2/'"

God-damnit-all commented 5 years ago

@jack1142 It's tied to both the python and python3 packages. Would I just add <dependency id="python" version="[3.7,3.8)" /> to the dependencies section?

Also, should I unpin python and python3 after installing this?

Jackenmen commented 5 years ago

@ImportTaste nope, just python3, python is metapackage that uses python3 in the end. You may want to uninstall python meta package though.

God-damnit-all commented 5 years ago

@jack1142 Can't, the python metapackage is a dependency to something else.

I'm not sure this is working unfortunately. I followed your instructions to the letter and then I ran choco upgrade all --noop, these bits were in the output:

You have python v3.7.5 installed. Version 3.8.0 is available based on your source(s).

python37-pin was not found with the source(s) listed.
 If you specified a particular version and are receiving this message, it is possible that the package name exists but the version does not.
 Version: ""; Source(s): "https://chocolatey.org/api/v2/"

You have python3 v3.7.5 installed. Version 3.8.0 is available based on your source(s).

Chocolatey can upgrade 2/30 packages.
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).

Can upgrade:
 - python3 v3.8.0
 - python v3.8.0

Warnings:
 - python37-pin - python37-pin was not found with the source(s) listed.

Jackenmen commented 5 years ago

@ImportTaste they worked just fine cause python didn't get upgraded though it is weird that chocolatey mentions python-pin there in the warnings instead of python37-pin (did you by any chance have different id in nuspec and different name of nuspec file?). Anyway since you said that metapackage is dependency to something else, you should also add python to depedencies, so python37-pin.nuspec should look like this:

<?xml version="1.0"?>
<package xmlns="http://schemas.microsoft.com/packaging/2011/08/nuspec.xsd">
  <metadata>
    <id>python37-pin</id>
    <version>3.7</version>
    <title>Python 3.7</title>
    <owners>jack1142</owners>
    <authors>Python Software Foundation</authors>
    <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <description><![CDATA[Python 3.x is a programming language that lets you work more quickly and integrate your systems more effectively. You can learn to use Python 3.x and see almost immediate gains in productivity and lower maintenance costs.]]></description>
    <dependencies>
      <dependency id="python" version="[3.7,3.8)" />
      <dependency id="python3" version="[3.7,3.8)" />
    </dependencies>
  </metadata>
  <files>
  </files>
</package>

Edit: remember to repack the package with choco pack and then you can uninstall python37-pin and install it again

God-damnit-all commented 5 years ago

it is weird that chocolatey mentions python-pin there in the warnings instead of python37-pin (did you by any chance have different id in nuspec and different name of nuspec file?).

Damnit, I tried to correct that before you saw it. I actually typed out that last line manually because I missed it when I was transporting the text from my laptop to my desktop. I was mainly concerned with this warning it threw:

python37-pin was not found with the source(s) listed.
 If you specified a particular version and are receiving this message, it is possible that the package name exists but the version does not.
 Version: ""; Source(s): "https://chocolatey.org/api/v2/"

Why does this happen?

Jackenmen commented 5 years ago

@ImportTaste I'm sorry but actually you should forget my whole advice cause I just figured that chocolatey fails to update python 3.7.4 to 3.7.5 when having that "pin package" installed cause it wants to use python 3.8 which it obviously can't because of dependency but instead of trying to resolve it, it just doesn't upgrade python at all. The error you mentioned could be solved by choco source add -n=python37-pin -s "C:\path\to\python37-pin\folder" but since the package doesn't actually solve the issue properly you're not gonna need it.

aaronsteers commented 3 years ago

I was directed here to this feature request by some comments regarding python version freezing - and I see that many of the comments here are actually related to python as well. I've logged a new ticket in https://github.com/chocolatey-community/chocolatey-coreteampackages/issues/1602 to discuss a reevaluation of granularity for publishing python packages.

This new ticket proposes that we think of "minor" Python version releases as if they were each major releases - adopting a strategy of publishing a new public meta-package for each in the chocolatey package directory starting with python38. This wouldn't cancel out any capabilities of the existing python and python3 packages, but it would simplify version pinning within new major (read: "non-backwards compatible") python release.

Would greatly appreciate this group's ideas and combined brain power there in that thread:

RE: https://github.com/chocolatey-community/chocolatey-coreteampackages/issues/1602

joel-schaal commented 1 year ago

Is there any news around that issue? I tried it on my side and it still does not work indeed:

$ choco upgrade -y notepadplusplus --version="(,8.2)"
Chocolatey v2.1.0
Upgrading the following packages:
notepadplusplus
By upgrading, you accept licenses for the packages.

Chocolatey upgraded 0/0 packages.
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).
'(,8.2)' is not a valid version string.
Parameter name: value

In the log file, I was presented with that stack trace:

2023-09-06 13:52:36,350 20748 [ERROR] - More Details: System.ArgumentException: '(,8.2)' is not a valid version string.
Parameter name: value
   at NuGet.Versioning.NuGetVersion.Parse(String value)
   at chocolatey.infrastructure.app.services.NugetService.Upgrade(ChocolateyConfiguration config, Action`2 continueAction, Boolean performAction, Action`2 beforeUpgradeAction)
   at chocolatey.infrastructure.app.services.NugetService.Upgrade(ChocolateyConfiguration config, Action`2 continueAction, Action`2 beforeUpgradeAction)
   at chocolatey.infrastructure.app.services.ChocolateyPackageService.Upgrade(ChocolateyConfiguration config)
   at chocolatey.infrastructure.app.runners.GenericRunner.Run(ChocolateyConfiguration config, Container container, Boolean isConsole, Action`1 parseArgs)
   at chocolatey.infrastructure.app.runners.ConsoleApplication.Run(String[] args, ChocolateyConfiguration config, Container container)
   at chocolatey.console.Program.Main(String[] args)

I looked a bit in NugetService.Upgrade and I believe NuGetVersion version should either be converted to a VersionRange, or the appropriate NuGetVersion should be looked up (through a find with the appropriate call to the nuget backend), to minimize code change.

I just scratched the surface (I'm quite new to the codebase, and not that familiar with C#). Can someone more versed investigate more thoroughly?

pauby commented 1 year ago

This hasn't been assigned to a milestone and isn't something we're working on.

Nuspec versioning doesn't work with the --version option (as you discovered).

chocolatey / choco

Expand `--version` option to allow range #800

Summary

Proposal

Related