search

PowerShell / PSResourceGet

PSResourceGet is the package manager for PowerShell
https://www.powershellgallery.com/packages/Microsoft.PowerShell.PSResourceGet
MIT License
490 stars 94 forks source link

Unable to install modules without `NormalizedVersion` property from Artifactory (post-rebase `InstallName()` references?) #1586

Closed sean-r-williams closed 7 months ago

sean-r-williams commented 8 months ago

Prerequisites

Steps to reproduce

  1. Create an Artifactory NuGet repo and publish a PowerShell module to it
  2. [Optional] Create a remote repository in Artifactory that proxies PSGallery, then create a virtual repository that aggregates the remote+local repos
  3. Install said local package with Install-PSResource using either the local or virtual repo
  4. [Optional] Install module JiraPS from the virtual repo

Expected behavior

PS> Install-PSResource Deployment-toolkit -Repository Artifactory
PS> # (no error)
PS> Install-PSResource JiraPS -Repository Artifactory
PS> # (no error)

Actual behavior

PS> Install-PSResource Deployment-toolkit -Repository Artifactory
Install-PSResource: 'Response status code does not indicate success: 405.' Request sent: 'https://artifactory.f.q.d.n/artifactory/api/nuget/v2/psgallery-nuget-virtual/package/Deployment-toolkit'
Install-PSResource: Package(s) 'Deployment-toolkit' could not be installed from repository 'Artifactory'.
PS> Install-PSResource JiraPS -Repository Artifactory
PS> # (no error)

Error details

Exception             :
    Type    : Microsoft.PowerShell.PSResourceGet.UtilClasses.ResourceNotFoundException
    Message : Package(s) 'Deployment-toolkit' could not be installed from repository 'Artifactory'.
    HResult : -2146233088
TargetObject          : Microsoft.PowerShell.PSResourceGet.Cmdlets.InstallPSResource
CategoryInfo          : InvalidData: (Microsoft.PowerShel…s.InstallPSResource:InstallPSResource) [Install-PSResource], ResourceNotFoundException
FullyQualifiedErrorId : InstallPackageFailure,Microsoft.PowerShell.PSResourceGet.Cmdlets.InstallPSResource
InvocationInfo        :
    MyCommand        : Install-PSResource
    ScriptLineNumber : 1
    OffsetInLine     : 1
    HistoryId        : 14
    Line             : Install-PSResource Deployment-toolkit -Repository Artifactory -Reinstall
    Statement        : Install-PSResource Deployment-toolkit -Repository Artifactory -Reinstall
    PositionMessage  : At line:1 char:1
                       + Install-PSResource Deployment-toolkit -Repository Artifactory -Re …
                       + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    InvocationName   : Install-PSResource
    CommandOrigin    : Internal
ScriptStackTrace      : at <ScriptBlock>, <No file>: line 1
PipelineIterationInfo :

Environment data

ModuleType Version    PreRelease Name                                ExportedCommands
---------- -------    ---------- ----                                ----------------
Binary     1.0.2.0               Microsoft.PowerShell.PSResourceGet  {Find-PSResource, Get-InstalledPSResource, Get-PSResourceRepository, Get-PSScriptFileInfo…}

Name                           Value
----                           -----
PSVersion                      7.4.1
PSEdition                      Core
GitCommitId                    7.4.1
OS                             Microsoft Windows 10.0.19045
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Visuals

The really confusing part is:

It's a little hard to follow the commit graph, but the v1.0.2 tag appears to have been created off-branch through a rebase without this PR's merge commit. It's not clear what other PRs to master were missed as part of this operation.

sean-r-williams commented 8 months ago

For comparison, here are the FindPackagesById() calls for:

The NormalizedVersion field is not set by Artifactory, so it makes sense why it's missing in the former case.

The codebase under the v1.0.2 tag is pretty hard to follow given the rebase, but it seems like we're hitting the following logic-path:

At a minimum, it seems like #1499, #1506, and possibly other PRs were rebased out of v1.0.2. @alerickson, can you provide more context on the rationale for rebasing these changes out of the release?

sean-r-williams commented 8 months ago

@alerickson @SydneyhSmith How can we ensure v1.0.3 includes these missing PRs/commits?

The release process seems to have changed significantly between v1.0.1 and v1.0.2 - I don't see any rebasing occurring beforehand. My team is once-again blocked because we can't download the packages we've published to Artifactory.

I had previously contributed with the expectation that the head of master would become v1.0.2 - rebasing out the aforementioned PRs changed the behavior of PSResourceGet in this area, and we're left without a functional package-management workflow as a result. I'm trying to keep my team engaged/on the golden-path here, but this pretty significantly eroded their interest (and trust).

alerickson commented 8 months ago

@sean-r-williams, we don't rebase for our releases, v1.0.2 is a servicing release, so we cherry-picked the PRs to be included in the release. For that release in particular we cherry-picked PRs that were not related to ACR work, which is why you don't see the PR you referenced. Since PSResourceGet ships in PowerShell now, we follow the PowerShell model of only cherry-picking bug or security fixes for servicing releases. If you want to see the commits that were included in the release you can view it here: https://github.com/PowerShell/PSResourceGet/compare/v1.0.1...v1.0.2. I think we either didn't cherry-pick far enough back or PR #1506 accidentally got skipped, in any case the PR was definitely missed.

We're not on a specific release cadence so it's easy to do another patch release to include something that's missing. That said we'll be having a minor release in the next week or two that will include everything that's in master.

sean-r-williams commented 8 months ago

@alerickson apologies for the confusion - I wasn't being clear in git terminology.

With PSResourceGet v1.0.1, there was a release branch created in GitHub so it was at least somewhat clear (from the commit history) what commits were "in" that release. There was cherry-picking in that release (again, apologies for the mix-up on my part), but all of the commits to master around that time period were included in the cherry-pick operation so there weren't any surprises.

For 1.0.2, no branch was published - a release wasn't posted to GitHub until well after it hit the Gallery (#1553). The cherry-pick distance was also much larger, and PRs like the ones mentioned earlier got missed.

There's nothing wrong with a cherry-pick release model, but having fixes fall through the cracks like this is really painful - especially when there's only been two patch releases since v1.0.0 shipped last year.

it's easy to do another patch release to include something that's missing.

Artifactory with NuGet v2 feeds is still wholly unusable, which blocks us from being able to use PSResourceGet. (Artifactory does not translate between NuGet v2 and v3 and we're aggregating PSGallery alongside our packages, so we can't move to NuGet v3 internally until Gallery supports v3.) What would it take to ship a v1.0.3 patch in the meantime that includes the removal of InstallName() from #1506 and specifically version null-coalescing from #1499?

The work is already "done" in this case and, based on what you've described, should have shipped in v1.0.2. Those two PRs are, AFAICT, the minimum needed to unblock us while your team continues work on ACR for v1.1.

That said we'll be having a minor release in the next week or two that will include everything that's in master.

Can your team commit to a v1.0.3 patch release within that timeframe in the event v1.1 is delayed? My concern in waiting for v1.1 is that there's larger intersectional feature work in that release and time-frames are naturally more likely to shift depending on that work.

alerickson commented 8 months ago

Working on the release of 1.0.3 now. Just FYI both of the PRs you referred to touch ACR code which is why they weren't included in the cherry-picks and shipped in 1.0.2.

alerickson commented 7 months ago

@sean-r-williams have the release out now: https://www.powershellgallery.com/packages/Microsoft.PowerShell.PSResourceGet/1.0.3 thanks for bringing this to our attention. Again, next release will include everything in master, but if there's further issues with this release (potentially other PRs that should have been included, reach out to us). Thanks for your patience with this-- it's a new system we have in place. And appreciate the thorough information to help with debugging.

sean-r-williams commented 7 months ago

@alerickson Thanks a bunch for getting this out the door. I've just confirmed that this issue is indeed resolved with v1.0.3.

I appreciate you and your team's willingness to make this right. I'll be sure to let you know if I spot any other problems with 1.0.3.

Cheers!