search

PowerShell / PowerShellGetv2

PowerShellGet is the Package Manager for PowerShell
https://www.PowerShellGallery.com
MIT License
429 stars 138 forks source link

Register-PSRepository -Default fails silently on undocumented URL ping. #350

Open BatmanAMA opened 5 years ago

BatmanAMA commented 5 years ago

Description of the issue

Register-PSRepository pings an undocumented endpoint(https://go.microsoft.com/fwlink/?LinkID=397631&clcid=0x409). When this ping-endpoint fails for any reason, it doesn't throw an error.

I also would recommend that this function ping what the endpoint resolves to (https://www.powershellgallery.com/api/v2/items/psscript) instead so that firewall rules allowing https://www.powershellgallery.com/ work without needing to know this other URL. If that can't/won't be done, the additional URL should be added to the documentation for register-psrepository.

Steps to reproduce

Configure a third party firewall (we have Palo Alto, but I don't think that would matter) to drop internet based traffic without a specific matching rule. Add in a rule to allow https traffic to www.powershellgallery.com

Register-PSRepository -Default

Expected behavior

Either:

  1. An Error
  2. Get-PSRepository should now appear as below: 
    Name                      InstallationPolicy   SourceLocation
----                      ------------------   --------------
PSGallery                 Untrusted            https://www.powershellgallery.com/api/v2

Actual behavior

No output to Error, Warning or Information channels from Register-PSRepository and afterwards, the output of Get-PSRepository has not changed (if it was empty, it remains so. If you had other repos, only they are shown.)

I set $DebugPreference to continue and I see it's trying to ping a different url (https://go.microsoft.com/fwlink/?LinkID=397631&clcid=0x409) entirely. This fails due to the default deny of the firewall and the command returns $Null instead of throwing an error.

debug output

PS C:\Users\me> $DebugPreference = 'Continue'
PS C:\Users\me> Register-PSRepository -Default -Verbose
DEBUG: 00:00:00.0000013 Calling New() : MethodName = 'GetDynamicOptions'
DEBUG: 00:00:00.0000637 Verbose: True
DEBUG: 00:00:00.0001013 Name: PSGallery
DEBUG: 00:00:00.0063406 INVOKING PowerShell Fn Get-DynamicOptions with args Provider that has length 1
DEBUG: 00:00:00.0105389 In PowerShellGet Provider - 'Get-DynamicOptions'.
DEBUG: 00:00:00.0157741 Done calling powershell «Get-DynamicOptions» «PSModule»
DEBUG: 00:00:00.0259331 Calling New() : MethodName = 'GetDynamicOptions'
DEBUG: 00:00:00.0259855 Verbose: True
DEBUG: 00:00:00.0260235 Name: PSGallery
DEBUG: 00:00:00.0305833 INVOKING PowerShell Fn Get-DynamicOptions with args Source that has length 1
DEBUG: 00:00:00.0348635 In PowerShellGet Provider - 'Get-DynamicOptions'.
DEBUG: 00:00:00.0420243 Done calling powershell «Get-DynamicOptions» «PSModule»
DEBUG: 00:00:00.1133341 Calling New() : MethodName = 'ResolvePackageSources'
DEBUG: 00:00:00.1163129 MessageResolver: Microsoft.PowerShell.PackageManagement.Cmdlets.GetMessageString
DEBUG: 00:00:00.1194694 ProviderName: PowerShellGet
DEBUG: 00:00:00.1222116 Verbose: True
DEBUG: 00:00:00.1272916 Name: PSGallery
DEBUG: 00:00:00.1377080 INVOKING PowerShell Fn Resolve-PackageSource with args  that has length 0
DEBUG: 00:00:00.1463272 In PowerShellGet Provider - 'Resolve-PackageSource'.
DEBUG: 00:00:00.1505658 PowerShell Script 'PSModule' Function 'Resolve-PackageSource' returns null.
DEBUG: 00:00:00.1649575 Done calling powershell «Resolve-PackageSource» «PSModule»
VERBOSE: Performing the operation "Register Module Repository." on target "Module Repository 'PSGallery' () in provider
 'PowerShellGet'.".
DEBUG: 00:00:00.1858111 Calling New() : MethodName = 'AddPackageSource'
DEBUG: 00:00:00.1886048 MessageResolver: Microsoft.PowerShell.PackageManagement.Cmdlets.GetMessageString
DEBUG: 00:00:00.1933776 ProviderName: PowerShellGet
DEBUG: 00:00:00.2003163 Verbose: True
DEBUG: 00:00:00.2035498 Name: PSGallery
DEBUG: 00:00:00.2126883 INVOKING PowerShell Fn Add-PackageSource with args PSGallery, , False that has length 3
DEBUG: 00:00:00.2269958 In PowerShellGet Provider - 'Add-PackageSource'.
DEBUG: 00:00:00.2297457 OPTION: MessageResolver => Microsoft.PowerShell.PackageManagement.Cmdlets.GetMessageString
DEBUG: 00:00:00.2328421 OPTION: ProviderName => PowerShellGet
DEBUG: 00:00:00.2377289 OPTION: Verbose => True
DEBUG: 00:00:00.2409456 OPTION: Name => PSGallery
DEBUG: 00:00:21.2710616 Ping-Endpoint: location=https://go.microsoft.com/fwlink/?LinkID=397631&clcid=0x409,
statuscode=, resolvedLocation=
DEBUG: 00:00:42.2946099 Ping-Endpoint: location=https://go.microsoft.com/fwlink/?LinkID=622995&clcid=0x409,
statuscode=, resolvedLocation=
DEBUG: 00:00:42.3039244 PowerShell Script 'PSModule' Function 'Add-PackageSource' returns null.
DEBUG: 00:00:42.3135763 Done calling powershell «Add-PackageSource» «PSModule»
PS C:\Users\me> Get-PSRepository
DEBUG: 00:00:00.0000013 Calling New() : MethodName = 'GetDynamicOptions'
DEBUG: 00:00:00.0056987 INVOKING PowerShell Fn Get-DynamicOptions with args Provider that has length 1
DEBUG: 00:00:00.0100287 In PowerShellGet Provider - 'Get-DynamicOptions'.
DEBUG: 00:00:00.0149603 Done calling powershell «Get-DynamicOptions» «PSModule»
DEBUG: 00:00:00.0453016 Calling New() : MethodName = 'GetDynamicOptions'
DEBUG: 00:00:00.0508574 INVOKING PowerShell Fn Get-DynamicOptions with args Source that has length 1
DEBUG: 00:00:00.0550146 In PowerShellGet Provider - 'Get-DynamicOptions'.
DEBUG: 00:00:00.0620076 Done calling powershell «Get-DynamicOptions» «PSModule»
DEBUG: 00:00:00.1145727 Calling New() : MethodName = 'ResolvePackageSources'
DEBUG: 00:00:00.1173800 MessageResolver: Microsoft.PowerShell.PackageManagement.Cmdlets.GetMessageString
DEBUG: 00:00:00.1201385 ProviderName: PowerShellGet
DEBUG: 00:00:00.1350703 INVOKING PowerShell Fn Resolve-PackageSource with args  that has length 0
DEBUG: 00:00:00.1433851 In PowerShellGet Provider - 'Resolve-PackageSource'.
DEBUG: 00:00:00.1469777 PowerShell Script 'PSModule' Function 'Resolve-PackageSource' returns null.
DEBUG: 00:00:00.1591316 Done calling powershell «Resolve-PackageSource» «PSModule»
WARNING: Unable to find module repositories.

Environment data

> $PSVersionTable
Name                           Value
----                           -----
PSVersion                      5.1.14393.2515
PSEdition                      Desktop
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}
BuildVersion                   10.0.14393.2515
CLRVersion                     4.0.30319.42000
WSManStackVersion              3.0
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
> Get-Module -ListAvailable PowerShellGet,PackageManagement

    Directory: C:\Program Files\WindowsPowerShell\Modules

ModuleType Version    Name                                ExportedCommands
---------- -------    ----                                ----------------
Script     1.1.7.2    PackageManagement                   {Find-Package, Get-Package, Get-PackageProvider, Get-Packa...
Script     1.1.7.0    PackageManagement                   {Find-Package, Get-Package, Get-PackageProvider, Get-Packa...
Binary     1.0.0.1    PackageManagement                   {Find-Package, Get-Package, Get-PackageProvider, Get-Packa...
Script     2.0.1      PowerShellGet                       {Find-Command, Find-DSCResource, Find-Module, Find-RoleCap...
Script     1.6.6      PowerShellGet                       {Find-Command, Find-DSCResource, Find-Module, Find-RoleCap...
Script     1.5.0.0    PowerShellGet                       {Install-Module, Find-Module, Save-Module, Update-Module...}
Script     1.0.0.1    PowerShellGet                       {Install-Module, Find-Module, Save-Module, Update-Module...}
> Get-PackageProvider -ListAvailable
Name                     Version          DynamicOptions
----                     -------          --------------
msi                      3.0.0.0          AdditionalArguments
msu                      3.0.0.0
nuget                    2.8.5.208
NuGet                    2.8.5.210        Destination, ExcludeVersion, Scope, SkipDependencies, Headers, FilterOnTag...
PowerShellGet            2.0.1.0          PackageManagementProvider, Type, Scope, AllowClobber, SkipPublisherCheck, ...
PowerShellGet            1.6.6.0
PowerShellGet            1.5.0.0
PowerShellGet            1.0.0.1
Programs                 3.0.0.0          IncludeWindowsInstaller, IncludeSystemComponent
edyoung commented 5 years ago

Should be resolved by above PR and included in next release

edyoung commented 5 years ago

Turns out this is more complicated than I thought. Have backed out the main change until the wrinkles are sorted out, but current code (to be included in next release) removes the fwlinks and directly goes to www.powershellgallery.com

muzzar78 commented 5 years ago

I have found another related issue when trying to 'Register-PSRepository -Default -proxy http://proxy:8080 -proxycredential $cred' behind a web proxy. The proxy credentials aren't getting passed to Resolve-Location and when Resolve-Location executes 'Ping-Endpoint' the status code returned is 407 (Proxy authentication required). The registration then fails silently.

PSCore 6.1.1 PowerShellGet 2.0.4 PackageManagement 1.2.4

Attached is the debug output from the failed attempt failed_debug.txt

I did a quick fix by changing https://github.com/PowerShell/PowerShellGet/blob/9e3b40f2661a59190b132356aba1be632ff0fe7b/src/PowerShellGet/private/functions/Resolve-Location.ps1#L65 to 

if((($statusCode -eq 200) -or ($statusCode -eq 401) -or ($statusCode -eq 407)) -and $resolvedLocation)

and then PSGallery was registered correctly. Attached is the debug output with the quick fix. successful_debug.txt

edyoung commented 5 years ago

Since we no longer ping go.microsoft.com, I think we can resolve this issue, even though we still actually contact the repository URL (which is what I was trying to remove).

edyoung commented 5 years ago

But if we can't reach powershellgallery.com, we still fail silently which is a problem.

avishnyakov commented 5 years ago

Same-same here, exactly as @BatmanAMA described. 

docker run --rm -it mcr.microsoft.com/powershell:ubuntu-16.04

Register-PSRepository -Default
Get-PSRepository

Best guess is that remote endpoints are blocked, due to either proxy settings or additional software that replaces SSL certs, and therefore we seem to have same-same issues as in this ticket.

Silent fail, no visibility even with the -Debug / -Verbose flags. Looking into more details.

Benny1007 commented 5 years ago

Stepping back a bit, why does it even ping (web request) an end point using an overly elaborate Ping-Endpoint function? I should be able to register a PowerShell repository if that repository is unavailable. Downstream components/functions should then error out that the repository is unavailable and report the source location for someone to investigate why. I don't see why this cmdlet has to be so convoluted.