Publish-Module can't overwrite - nuget can

jongisli commented 6 years ago

Expected Behavior

Overwriting modules should be possible - without bumping version number.

Current Behavior

I'm developing PowerShell modules, and I have two nuget repositories, one for snapshots (modules under development) and another one for releases (production ready modules). I'm using Artifactory as an artifact management system, and I've set up both repositories as nuget feeds in there.

First time I publish a module with PowerShellGet, it works. If I run the Publish-Module command again right after, it fails because it won't allow me to overwrite (the user psmodulewriter has permission to overwrite):

PS C:\> Publish-Module -Path 'C:\Workspace\code\UtilsModule' -Repository 'PSSnapshots' -NuGetApiKey 'psmodulewriter:password' -Credential 'psmodulewriter' -Force
PS C:\> Publish-Module -Path 'C:\Workspace\code\UtilsModule' -Repository 'PSSnapshots' -NuGetApiKey 'psmodulewriter:password' -Credential 'psmodulewriter' -Force
Publish-Module : The module 'UtilsModule' with version '0.0.1' cannot be published as the current version '0.0.1' is already available in the repository 'https://xxxx.jfrog.io/xxxxartifactory/api/nuget/xxxxPowerShellRepo-Snapshots/'.
At line:1 char:1
+ Publish-Module -Path 'C:\Workspace\code\UtilsModule' -Repository 'P ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [Publish-Module], InvalidOperationException
    + FullyQualifiedErrorId : ModuleVersionIsAlreadyAvailableInTheGallery,Publish-Module

Possible Solution

A possible solution would be to use the -Force parameter in line 1257 on the master branch in PowerShellGet/PSModule.psm1 like so:

if(-not $Force -and ($currentPSGetItemInfo.Version -eq $moduleInfo.Version))

Context

I'm using a CI/CD process and would like to test the modules I'm developing through the pipeline often, without bumping the version number every time.

This is possible using nuget. If I download the .nupkg from Artifactory and push it with nuget, it overwrites the file without any problem:

PS C:\> nuget push C:\temp\UtilsModule.0.0.1.nupkg -Source PSSnapshots
Pushing UtilsModule.0.0.1.nupkg to 'https://xxxx.jfrog.io/xxxx/api/nuget/xxxxPowerShellRepo-Snapshots'...
  PUT https://xxxx.jfrog.io/xxxx/api/nuget/xxxxPowerShellRepo-Snapshots/
  Created https://xxxx.jfrog.io/xxxx/api/nuget/xxxxPowerShellRepo-Snapshots/ 751ms
Your package was pushed.
PS C:\> nuget push C:\temp\UtilsModule.0.0.1.nupkg -Source PSSnapshots
Pushing UtilsModule.0.0.1.nupkg to 'https://xxxx.jfrog.io/xxxx/api/nuget/xxxxPowerShellRepo-Snapshots'...
  PUT https://xxxx.jfrog.io/xxxx/api/nuget/xxxxPowerShellRepo-Snapshots/
  Created https://xxxx.jfrog.io/xxxx/api/nuget/xxxxPowerShellRepo-Snapshots/ 675ms
Your package was pushed.

Your Environment


PS C:\> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      5.1.14409.1012
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.14409.1012
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1

PS C:\> Get-Module

ModuleType Version    Name                                ExportedCommands
---------- -------    ----                                ----------------
Script     1.0        GroupSet                            {BuildResourceCommonParameters, BuildResourceString, GroupSet}
Manifest   3.1.0.0    Microsoft.PowerShell.Management     {Add-Computer, Add-Content, Checkpoint-Computer, Clear-Content...}
Manifest   3.0.0.0    Microsoft.PowerShell.Security       {ConvertFrom-SecureString, ConvertTo-SecureString, Get-Acl, Get-AuthenticodeSignature...}
Manifest   3.1.0.0    Microsoft.PowerShell.Utility        {Add-Member, Add-Type, Clear-Variable, Compare-Object...}
Manifest   3.0.0.0    Microsoft.WSMan.Management          {Connect-WSMan, Disable-WSManCredSSP, Disconnect-WSMan, Enable-WSManCredSSP...}
Script     1.1.6.0    PackageManagement                   {Find-Package, Find-PackageProvider, Get-Package, Get-PackageProvider...}
Script     1.5.0.0    PowerShellGet                       {Find-Command, Find-DscResource, Find-Module, Find-RoleCapability...}
Script     1.0        ProcessSet                          {BuildResourceCommonParameters, BuildResourceString, ProcessSet}
Manifest   1.1        PSDesiredStateConfiguration         {Invoke-DscResource, Publish-DscConfiguration, Set-DscLocalConfigurationManager, Start-DscConfiguration...}
Script     1.0        ServiceSet                          {BuildResourceCommonParameters, BuildResourceString, ServiceSet}
Script     1.0        WindowsFeatureSet                   {BuildResourceCommonParameters, BuildResourceString, WindowsFeatureSet}
Script     1.0        WindowsOptionalFeatureSet           {BuildResourceCommonParameters, BuildResourceString, WindowsOptionalFeatureSet}

PS C:\> Get-PackageProvider

Name                     Version          DynamicOptions
----                     -------          --------------
msi                      3.0.0.0          AdditionalArguments
msu                      3.0.0.0
NuGet                    2.8.5.210        Destination, ExcludeVersion, Scope, SkipDependencies, Headers, FilterOnTag, Contains, AllowPrereleaseVersions, ConfigFile, SkipValidate
PowerShellGet            1.5.0.0          PackageManagementProvider, Type, Scope, AllowClobber, SkipPublisherCheck, InstallUpdate, NoPathUpdate, Filter, Tag, Includes, DscResource, RoleCapability, Command, AcceptLicense, PublishLocation, ScriptSourceLocation, ScriptPublishLocation
Programs                 3.0.0.0          IncludeWindowsInstaller, IncludeSystemComponent

petr-stupka commented 6 years ago

+1

Sholdaway commented 6 years ago

+1 from me too

erlendwtf commented 6 years ago

+1

ghost commented 6 years ago

If the one use case that is listed here is the only one, this is not a good feature as it will be abused. If there are other use cases, we would very much like to hear about them.

The use case of concern is described as: "I'm using a CI/CD process and would like to test the modules..." You should not publish to the PowerShell Gallery something you are testing, you should only publish items that are in at least a preview release state. This is a free public repo, and if everyone starts routing their release pipeline tests to the PSGallery we bear the cost of that testing, and the users of the PSGallery have to guess what has not been tested. Testing your CI/CD pipeline should be done to an internal private repo, then route to PSgallery for release.

jongisli commented 6 years ago

I’m not publishing at all to the public Gallery. If this can be abused in the Gallery, my opinion is that the Gallery needs hardening.

We are indeed using private repos as you suggest. We have two NuGet repositories. One for snapshots and another for releases - we want to be able to overwrite the snapshots.

This is very standard workflow in software development. And it’s strange to hardcode a security feature like this is the package manager instead of letting the users control this in the artifact management system (e.g. Nexus/Artifactory).

Jon

On Fri, 6 Jul 2018 at 20.33, J. Keith Bankston [MSFT] < notifications@github.com> wrote:

If the one use case that is listed here is the only one, this is not a good feature as it will be abused. If there are other use cases, we would very much like to hear about them.

The use case of concern is described as: "I'm using a CI/CD process and would like to test the modules..." You should not publish to the PowerShell Gallery something you are testing, you should only publish items that are in at least a preview release state. This is a free public repo, and if everyone starts routing their release pipeline tests to the PSGallery we bear the cost of that testing, and the users of the PSGallery have to guess what has not been tested. Testing your CI/CD pipeline should be done to an internal private repo, then route to PSgallery for release.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/PowerShell/PowerShellGet/issues/200#issuecomment-403113920, or mute the thread https://github.com/notifications/unsubscribe-auth/AANqMV_3Q1cInhscNsq5uVzK8lFYsasUks5uD61sgaJpZM4QrmzV .

ghost commented 6 years ago

@jongisli That's why I asked about the use case. There are other situations where we support a capability in general but not for PSGallery. We have hardened the PSGallery by not allowing the capability. However, if the ask is to support this for repos other than the PSGallery, that's something we can consider.

jongisli commented 6 years ago

“However, if the ask is to support this for repos other than the PSGallery, that's something we can consider.”

That’s exactly the request :-)

Jon

On Fri, 6 Jul 2018 at 21.00, J. Keith Bankston [MSFT] < notifications@github.com> wrote:

@jongisli https://github.com/jongisli That's why I asked about the use case. There are other situations where we support a capability in general but not for PSGallery. We have hardened the PSGallery by not allowing the capability. However, if the ask is to support this for repos other than the PSGallery, that's something we can consider.

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/PowerShell/PowerShellGet/issues/200#issuecomment-403119995, or mute the thread https://github.com/notifications/unsubscribe-auth/AANqMaZN0J80ErjC0swES6OfbZPZKIzHks5uD7O2gaJpZM4QrmzV .

petr-stupka commented 6 years ago

Yes, repos other than PSGallery is the case. I found a workaround. I’m checking with Find-Module if the Module and the specific version exist and if yes then using nuget.exe to delete the existing Module from repo then reuploading the tested version again. I can cooy-paste the code if someone is interested.

jongisli commented 6 years ago

We have workarounds. I made the issue on github because I felt it should be implemented this way in Publish-Module, not requiring any workarounds.

Jon

On Fri, 6 Jul 2018 at 21.31, schwaizi notifications@github.com wrote:

Yes, repos other than PSGallery is the case. I found a workaround. I’m checking with Find-Module if the Module and the specific version exist and if yes then using nuget.exe to delete the existing Module from repo then reuploading the tested version again. I can cooy-paste the code if someone is interested.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/PowerShell/PowerShellGet/issues/200#issuecomment-403126673, or mute the thread https://github.com/notifications/unsubscribe-auth/AANqMTVee0jFDLEG4sTKAx5hG6Z20PNzks5uD7sfgaJpZM4QrmzV .