Fetch a module with the version form x.x.x.x fails (not using semantic versioning)

[johlju]

If trying to get a module version that is not using semantic versioning Get-ModuleFastPlan fails with an exception.

iwr bit.ly/modulefast | iex

@(
    @{ModuleName = 'PSDscResources'; RequiredVersion = '2.12.0.0' }
) | Get-ModuleFastPlan

fails with error:

OperationStopped:                                                                                                       
Line |
 346 |  …             throw [InvalidOperationException]"$currentModuleSpec`: a  …
     |                ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | PSDscResources@2.12.1-0000000000+SYSTEMVERSION.HASREVISION: a matching module was not found in the https://pwsh.gallery/index.json repository that satisfies the version constraints. If this happens during dependency lookup, it is a bug in ModuleFast.

[LaurentDardenne]

Powershell mixes version formats or adapts them:

[Version]'2.12.0.0'

Major  Minor  Build  Revision
-----  -----  -----  --------
2      12     0      0

[System.Management.Automation.SemanticVersion]'2.12.0.0'
InvalidArgument: Cannot convert value "2.12.0.0" to type "System.Management.Automation.SemanticVersion". Error: "The input string '' was not in a correct format."

[System.Management.Automation.SemanticVersion]'2.12.0.0-beta'
InvalidArgument: Cannot convert value "2.12.0.0-beta" to type "System.Management.Automation.SemanticVersion". Error: "The input string '' was not in a correct format."

[System.Management.Automation.SemanticVersion]'2.12.0-beta'

Major  Minor  Patch  PreReleaseLabel BuildLabel
-----  -----  -----  --------------- ----------
2      12     0      beta

:-/

[JustinGrote]

I investigated further and found this is because I am using NormalizedVersion from the gallery for the purposes of version comparison but I should use Version for purposes of defining the installation folder.

You'll see here as an example: https://www.powershellgallery.com/api/v2/FindPackagesById()?id=%27SS.PowerShell%27&semVerLevel=2.0.0&$orderby=IsLatestVersion%20desc,IsAbsoluteLatestVersion%20desc,Created%20desc&$select=GUID,Version,NormalizedVersion,Dependencies,IsLatestVersion,IsAbsoluteLatestVersion

SS.PowerShell 1.0.0.0 reports 1.0.0.0 for Version but 1.0.0 for NormalizedVersion

[JustinGrote]

The problem was on the pwsh.gallery side, it should be fixed with https://github.com/JustinGrote/PWSHGallery/commit/3f60e74c8d69336bd207d88bfcb768f62980df77

@johlju can you verify? works for me, does not require a modulefast update.

[johlju]

Yes that seems to work, but now others are failing. 🤔

This works:

PS > $plan = @(@{ModuleName = 'PSDscResources'; RequiredVersion = '2.12.0.0' } ) | Get-ModuleFastPlan 
PS> Install-ModuleFast -ModulesToInstall $plan -Destination '/Users/johlju/source/SqlServerDsc/output/RequiredModules' -NoProfileUpdate -NoPSModulePathUpdate -Update -Verbose -Debug -Confirm:$false
DEBUG: /Users/johlju/source/SqlServerDsc/output/RequiredModules does not have a PSDscResources folder. Skipping...      
DEBUG: /Users/johlju/.local/share/powershell/Modules does not have a PSDscResources folder. Skipping...                 
DEBUG: /usr/local/share/powershell/Modules does not have a PSDscResources folder. Skipping...                           
DEBUG: /usr/local/microsoft/powershell/7/Modules does not have a PSDscResources folder. Skipping...                     
DEBUG: PSDscResources: fetch info from https://pwsh.gallery/psdscresources/index.json                                   
DEBUG: PSDscResources@2.12.1-0000000000+SYSTEMVERSION.HASREVISION: Processing Response                                  
DEBUG: PSDscResources@2.12.1-0000000000+SYSTEMVERSION.HASREVISION: Received Response with 2 pages                       
DEBUG: PSDscResources@2.12.1-0000000000+SYSTEMVERSION.HASREVISION: Found satisfying version 2.12.0.0 in the inlined index.
VERBOSE: PSDscResources@2.12.1-0000000000+SYSTEMVERSION.HASREVISION: Adding to install plan                             
DEBUG: Remaining Tasks: 0                                                                                               
VERBOSE: Performing the operation "Install 1 Modules" on target "/Users/johlju/source/SqlServerDsc/output/RequiredModules".
VERBOSE: PSDscResources@2.12.1-0000000000+SYSTEMVERSION.HASREVISION: Starting Download for https://www.powershellgallery.com/api/v2/package/PSDscResources/2.12.0
VERBOSE: PSDscResources@2.12.1-0000000000+SYSTEMVERSION.HASREVISION: Starting Extract Job to /Users/johlju/source/SqlServerDsc/output/RequiredModules/PSDscResources/2.12.0.0
VERBOSE: PSDscResources@2.12.1-0000000000+SYSTEMVERSION.HASREVISION: Successfuly installed to /Users/johlju/source/SqlServerDsc/output/RequiredModules/PSDscResources/2.12.0.0
VERBOSE: ✅ All required modules installed! Exiting.

This no longer works (tested with two different modules using semantic version):

PS> $plan = @(@{ModuleName = 'PSPKI'; RequiredVersion = '3.7.2' }) | Get-ModuleFastPlan 
Exception:                                                                                                              
Line |
 257 |  …                   $response = $completedTask.GetAwaiter().GetResult()
     |                      ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
     | PSPKI@3.7.2: Failed to fetch module PSPKI@3.7.2 from https://pwsh.gallery/index.json. Error: Exception calling "GetResult" with "0" argument(s): "Response status code does not indicate success: 500 (Internal Server Error)."

[JustinGrote]

Guess I dropped a regression in there! I'll check.

[JustinGrote]

That package has a weird mix of versions which screwed up the index builder:

Fixed in https://github.com/JustinGrote/PWSHGallery/commit/daa3a53b7f0488cb9ae88616c77ec5a39a12ea68, try it again.

[johlju]

Now I get another error when running this:

PS > $plan = @(@{ModuleName = 'PSPKI'; RequiredVersion = '3.7.2' }) | Get-ModuleFastPlan 
Where-Object:                                                                                                           
Line |
 376 |                      | Where-Object {
     |                        ~~~~~~~~~~~~~~
     | Method invocation failed because [System.Double] does not contain a method named 'contains'.

Also this regressed, what worked before now no longer returns a plan at all (no error message), it now returns $null: My bad, this was the expected output because I forgot to remove the already saved module from the path. 🙂

PS > $plan = @(@{ModuleName = 'PSDscResources'; RequiredVersion = '2.12.0.0' }) | Get-ModuleFastPlan
PS > $null -eq $plan
True 

[JustinGrote]

That's probably specific to modulefastplan, install-modulefast worked fine, let me take a look.

[JustinGrote]

This is because any two-digit version is getting incorrectly translated to a number

I'll fix that.

[JustinGrote]

@johlju should be good now: https://github.com/JustinGrote/PWSHGallery/commit/9d41585bb95138ab2a02a34e7161fcb092d4105a

I made a script to get every version that exists on pwshgallery and will test against all of those as well:

$versions = [System.Collections.Generic.HashSet[string]]::new()
$sort = [System.Collections.Generic.List[string]]::new()

#Roughly the number of packages per the powershellgallery.com website
for($i=0; $i -lt 300000; $i=$i+100) {
    $sort.Add($i)
}

$sort | ForEach-Object -ThrottleLimit 30 -Parallel {
    $skip = $_
    Write-Host "Page $_"
    $uri = "https://www.powershellgallery.com/api/v2/Packages()?`$skip=$skip"

    [xml]$data = (Invoke-WebRequest $uri).Content
    $data.feed.entry.properties.version
}
| Foreach-Object {
    $versions.Add($_) | Out-Null
}

[johlju]

It works now! Closing this. Thank you!