Register-PSRepository raises errors about "invalid Web Uri", despite underlying httpclient recognizing the Uri and [Uri] parsing

jzabroski commented 6 years ago

I have a very simple problem I want to solve: Due to reasons beyond my control, the company I work for uses a ".local" domain name internally, for everything. My repository url is "http://packages.mycompany.local/nuget/Mycompany-Development/api/v2" - which according to Register-PSRepository is not a "Web Uri" (whatever the heck that means).

It appears this is a bogus error message, largely due to the sophisticated and overly complicated sanity checks Register-PSRepository tries to perform. I've extracted all the required sub-functions to replicate the behavior, modulo the final error trap that surfaces the wrong error.

function Get-LocationString
{
    [CmdletBinding(PositionalBinding=$false)]
    Param
    (
        [Parameter()]
        [Uri]
        $LocationUri
    )

    $LocationString = $null

    if($LocationUri)
    {
        if($LocationUri.Scheme -eq 'file')
        {
            $LocationString = $LocationUri.OriginalString
        }
        elseif($LocationUri.AbsoluteUri)
        {
            $LocationString = $LocationUri.AbsoluteUri
        }
        else
        {
            $LocationString = $LocationUri.ToString()
        }
    }

    return $LocationString
}

function Resolve-Location
{
    [CmdletBinding()]
    [OutputType([string])]
    Param
    (
        [Parameter(Mandatory=$true)]
        [string]
        $Location,

        [Parameter(Mandatory=$true)]
        [string]
        $LocationParameterName,

        [Parameter()]
        $Credential,

        [Parameter()]
        $Proxy,

        [Parameter()]
        $ProxyCredential,

        [Parameter()]
        [System.Management.Automation.PSCmdlet]
        $CallerPSCmdlet
    )

    # Ping and resolve the specified location
    if(-not (Test-WebUri -uri $Location))
    {
        if(Microsoft.PowerShell.Management\Test-Path -Path $Location)
        {
            return $Location
        }
        elseif($CallerPSCmdlet)
        {
            $message = $LocalizedData.PathNotFound -f ($Location)
            ThrowError -ExceptionName "System.ArgumentException" `
                       -ExceptionMessage $message `
                       -ErrorId "PathNotFound" `
                       -CallerPSCmdlet $CallerPSCmdlet `
                       -ErrorCategory InvalidArgument `
                       -ExceptionObject $Location
        }
    }
    else
    {
        $pingResult = Ping-Endpoint -Endpoint $Location -Credential $Credential -Proxy $Proxy -ProxyCredential $ProxyCredential
        $statusCode = $null
        $exception = $null
        $resolvedLocation = $null
        if($pingResult -and $pingResult.ContainsKey($Script:ResponseUri))
        {
            $resolvedLocation = $pingResult[$Script:ResponseUri]
        }

        if($pingResult -and $pingResult.ContainsKey($Script:StatusCode))
        {
            $statusCode = $pingResult[$Script:StatusCode]
        }

        Write-Debug -Message "Ping-Endpoint: location=$Location, statuscode=$statusCode, resolvedLocation=$resolvedLocation"

        if((($statusCode -eq 200) -or ($statusCode -eq 401)) -and $resolvedLocation)
        {
            return $resolvedLocation
        }
        elseif($CallerPSCmdlet)
        {
            $message = $LocalizedData.InvalidWebUri -f ($Location, $LocationParameterName)
            ThrowError -ExceptionName "System.ArgumentException" `
                       -ExceptionMessage $message `
                       -ErrorId "InvalidWebUri" `
                       -CallerPSCmdlet $CallerPSCmdlet `
                       -ErrorCategory InvalidArgument `
                       -ExceptionObject $Location
        }
    }
}

function Test-WebUri
{
    [CmdletBinding()]
    [OutputType([bool])]
    Param
    (
        [Parameter(Mandatory=$true)]
        [ValidateNotNullOrEmpty()]
        [Uri]
        $uri
    )

    return ($uri.AbsoluteURI -ne $null) -and ($uri.Scheme -match '[http|https]')
}

function Ping-Endpoint
{
    [CmdletBinding()]
    param
    (
        [Parameter()]
        [ValidateNotNullOrEmpty()]
        [string]
        $Endpoint,

        [Parameter()]
        $Credential,

        [Parameter()]
        $Proxy,

        [Parameter()]
        $ProxyCredential,

        [Parameter()]
        [switch]
        $AllowAutoRedirect = $true
    )

    $results = @{}

    $WebProxy = $null
    if($Proxy -and ('Microsoft.PowerShell.Commands.PowerShellGet.InternalWebProxy' -as [Type]))
    {
        $ProxyNetworkCredential = $null
        if($ProxyCredential)
        {
            $ProxyNetworkCredential = $ProxyCredential.GetNetworkCredential()
        }

        $WebProxy = New-Object Microsoft.PowerShell.Commands.PowerShellGet.InternalWebProxy -ArgumentList $Proxy,$ProxyNetworkCredential
    }

    if(HttpClientApisAvailable)
    {
        $response = $null
        try
        {
            $handler = New-Object System.Net.Http.HttpClientHandler

            if($Credential)
            {
                $handler.Credentials = $Credential.GetNetworkCredential()
            }
            else
            {
                $handler.UseDefaultCredentials = $true
            }

            if($WebProxy)
            {
                $handler.Proxy = $WebProxy
            }

            $httpClient = New-Object System.Net.Http.HttpClient -ArgumentList $handler
            $response = $httpclient.GetAsync($endpoint)
        }
        catch
        {
        }

        if ($response -ne $null -and $response.result -ne $null)
        {
            Write-Host $response.Result
            $results.Add($Script:ResponseUri,$response.Result.RequestMessage.RequestUri.AbsoluteUri.ToString())
            $results.Add($Script:StatusCode,$response.result.StatusCode.value__)
        }
    }
    else
    {
        $iss = [System.Management.Automation.Runspaces.InitialSessionState]::Create()
        $iss.types.clear()
        $iss.formats.clear()
        $iss.LanguageMode = "FullLanguage"

        $WebRequestcmd =  @'
            param($Credential, $WebProxy)
            try
            {{
                $request = [System.Net.WebRequest]::Create("{0}")
                $request.Method = 'GET'
                $request.Timeout = 30000
                if($Credential)
                {{
                    $request.Credentials = $Credential.GetNetworkCredential()
                }}
                else
                {{
                    $request.Credentials = [System.Net.CredentialCache]::DefaultNetworkCredentials
                }}
                $request.AllowAutoRedirect = ${1}
                if($WebProxy)
                {{
                    $request.Proxy = $WebProxy
                }}
                $response = [System.Net.HttpWebResponse]$request.GetResponse()
                if($response.StatusCode.value__ -eq 302)
                {{
                    $response.Headers["Location"].ToString()
                }}
                else
                {{
                    $response
                }}
                $response.Close()
            }}
            catch [System.Net.WebException]
            {{
                "Error:System.Net.WebException"
            }}
'@ -f $EndPoint, $AllowAutoRedirect

        $ps = [powershell]::Create($iss).AddScript($WebRequestcmd)

        if($WebProxy)
        {
            $null = $ps.AddParameter('WebProxy', $WebProxy)
        }

        if($Credential)
        {
            $null = $ps.AddParameter('Credential', $Credential)
        }

        $response = $ps.Invoke()
        $ps.dispose()
        if ($response -ne "Error:System.Net.WebException")
        {
            if($AllowAutoRedirect)
            {
                $results.Add($Script:ResponseUri,$response.ResponseUri.ToString())
                $results.Add($Script:StatusCode,$response.StatusCode.value__)
            }
            else
            {
                $results.Add($Script:ResponseUri,[String]$response)
            }
        }
    }
    return $results
}

function HttpClientApisAvailable
{
    $HttpClientApisAvailable = $false
    try
    {
        [System.Net.Http.HttpClient]
        $HttpClientApisAvailable = $true
    }
    catch
    {
    }
    return $HttpClientApisAvailable
}

$SourceLocation = [Uri] "http://packages.mycompany.local/nuget/Mycompany-Development/api/v2" 

# Ping and resolve the specified location
            $SourceLocation = Resolve-Location -Location (Get-LocationString -LocationUri $SourceLocation) `
                                               -LocationParameterName 'SourceLocation' `
                                               -Credential $Credential `
                                               -Proxy $Proxy `
                                               -ProxyCredential $ProxyCredential `
                                               -CallerPSCmdlet $PSCmdlet

I'm not 100% sure, but I think the issue is that Resolve-Location treats any failures as "Web Uri" failures, even though the error appears to be due to $Script:$ResponseUri not existing/being a null key. (I'm not a PowerShell expert and while I generally understand the concept of scope-modifiers, I generally program in C# where you can't dynamically change scopes. To a C# programmer, this looks like bad code or non-sensical code.)

This appears to be an issue since PowerShellGet was called OneGet: https://github.com/OneGet/TestBuild/issues/2

In that issue, it appears that Ping-Endpoint was to blame. Oddly, this was 3 years ago and still not fixed as OneGet got renamed PowerShellGet.

bmanikm commented 6 years ago

@jzabroski I agree, InvalidWebUri error message is confusing/misleading. This error can happen if the specified source location requires credential and/or proxy parameters. We need to update the error message to include this information.

Please let us know if you can ping your specified source location using Invoke-WebRequest cmdlet.

# If required, specify the Credential and/or Proxy related parameters
Invoke-WebRequest -Uri "http://packages.mycompany.local/nuget/Mycompany-Development/api/v2" -Verbose

Possible resolution:

Update the error message to include the authentication and proxy related information.
Convert the error message to warning message and proceed to register the repository without failing.

jzabroski commented 6 years ago

@bmanikm Thanks for your response.

PS C:\WINDOWS\system32> Invoke-WebRequest -Uri "http://packages.mycompany.local/nuget/Mycompany-Development/api/v2" -Verbose -Credential $(Get-Credential)

cmdlet Get-Credential at command pipeline position 1
Supply values for the following parameters:
Credential
VERBOSE: GET http://packages.mycompany.local/nuget/Mycompany-Development/api/v2 with 0-byte payload
Invoke-WebRequest : OData method is not implemented.
At line:1 char:1
+ Invoke-WebRequest -Uri "http://packages.mycompany.local/nuget/Mycompany- ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

I think I wasted all day on this, and hope to recoup the loss by contributing to make this better. Here are all the ways I see this could fail:

PackageManagementProvider dependency is huge. Is this for backward compatibility? Please at least add comments so people like me investigating understand what it's trying to accomplish.
As mentioned in https://github.com/PowerShell/PowerShellGet/issues/180#issuecomment-417442603 , I have been able to simplify my problem report and discovered a failure to load type [System.Net.Http.HttpClient]. This is bizarre, because, check this out (each block of powershell is a fresh, clean terminal session):

PS C:\WINDOWS\system32> [system.net.http.httpclient]
Unable to find type [system.net.http.httpclient].
At line:1 char:1
+ [system.net.http.httpclient]
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (system.net.http.httpclient:TypeName) [], RuntimeException
    + FullyQualifiedErrorId : TypeNotFound

PS C:\WINDOWS\system32> Add-Type -AssemblyName System.Net.Http
PS C:\WINDOWS\system32> [System.Net.Http.HttpClient]

IsPublic IsSerial Name                                     BaseType
-------- -------- ----                                     --------
True     False    HttpClient                               System.Net.Http.HttpMessageInvoker

PS C:\WINDOWS\system32> Register-PSRepository -Name Mycompany-Development -SourceLocation "http://packages.mycompany.local/nuget/Mycompany-Development/api/v2" -PublishLocation http://packages.mycompany.local/nuget/Mycompany-Development/api/v2/package
Register-PSRepository : The specified Uri 'http://packages.mycompany.local/nuget/Mycompany-Development/api/v2' for parameter 'SourceLocation' is an invalid Web Uri. Please ensure that it meets the Web Uri requirements.
At line:1 char:1
+ Register-PSRepository -Name Mycompany-Development -SourceLocation "http:/ ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (http://packages...elopment/api/v2:String) [Register-PSRepository], ArgumentException
    + FullyQualifiedErrorId : InvalidWebUri,Register-PSRepository

PS C:\WINDOWS\system32> $Error
Register-PSRepository : The specified Uri 'http://packages.mycompany.local/nuget/Mycompany-Development/api/v2' for parameter 'SourceLocation' is an invalid Web Uri. Please ensure that it meets the Web Uri requirements.
At line:1 char:1
+ Register-PSRepository -Name Mycompany-Development -SourceLocation "http:/ ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (http://packages...elopment/api/v2:String) [Register-PSRepository], ArgumentException
    + FullyQualifiedErrorId : InvalidWebUri,Register-PSRepository

Get-Variable : Cannot find a variable with the name 'PackageManagementProvider'.
At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\1.6.7\PSModule.psm1:11275 char:16
+ ...          if(Get-Variable -Name PackageManagementProvider -ErrorAction ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (PackageManagementProvider:String) [Get-Variable], ItemNotFoundException
    + FullyQualifiedErrorId : VariableNotFound,Microsoft.PowerShell.Commands.GetVariableCommand

finally, the error sequence I mentioned in PowerShell/PowerShellGetv2#180 :

PS C:\WINDOWS\system32> Register-PSRepository -Name Mycompany-Development -SourceLocation "http://packages.mycompany.local/nuget/Mycompany-Development/api/v2" -PublishLocation http://packages.mycompany.local/nuget/Mycompany-Development/api/v2/package
Register-PSRepository : The specified Uri 'http://packages.mycompany.local/nuget/Mycompany-Development/api/v2' for parameter 'SourceLocation' is an invalid Web Uri. Please ensure that it meets the Web Uri requirements.
At line:1 char:1
+ Register-PSRepository -Name Mycompany-Development -SourceLocation "http:/ ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (http://packages...elopment/api/v2:String) [Register-PSRepository], ArgumentException
    + FullyQualifiedErrorId : InvalidWebUri,Register-PSRepository

PS C:\WINDOWS\system32> $Error
Register-PSRepository : The specified Uri 'http://packages.mycompany.local/nuget/Mycompany-Development/api/v2' for parameter 'SourceLocation' is an invalid Web Uri. Please ensure that it meets the Web Uri requirements.
At line:1 char:1
+ Register-PSRepository -Name Mycompany-Development -SourceLocation "http:/ ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (http://packages...elopment/api/v2:String) [Register-PSRepository], ArgumentException
    + FullyQualifiedErrorId : InvalidWebUri,Register-PSRepository

Unable to find type [System.Net.Http.HttpClient].
At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\1.6.7\PSModule.psm1:3020 char:9
+         [System.Net.Http.HttpClient]
+         ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.Http.HttpClient:TypeName) [], RuntimeException
    + FullyQualifiedErrorId : TypeNotFound

Get-Variable : Cannot find a variable with the name 'PackageManagementProvider'.
At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\1.6.7\PSModule.psm1:11275 char:16
+ ...          if(Get-Variable -Name PackageManagementProvider -ErrorAction ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : ObjectNotFound: (PackageManagementProvider:String) [Get-Variable], ItemNotFoundException
    + FullyQualifiedErrorId : VariableNotFound,Microsoft.PowerShell.Commands.GetVariableCommand

Possible resolution:

Update the error message to include the authentication and proxy related information.

Convert the error message to warning message and proceed to register the repository without failing.

I think the error message should probably link to an open-ended issue on PowerShellGet to help people get help. In this day and age, trying to provide the perfect error message a priori is too rigid. The other major problem is, well, PowerShell sucks at error messages and the default Output representation of an ErrorRecord object is TERRIBLE. I want to run back to C# every time I touch PowerShell.

jzabroski commented 6 years ago

Here is another possible solution to the problem: I was on ProGet 5.0.12 , and the latest available version is 5.1.8. I saw this in the release notes for 5.1.3:

PG-1304 - FIX: NuGet connectors append a slash to the OData function call, which may be incompatible with some third-party NuGet servers.

I'm upgrading now.

Edit: That did not fix the problem.

PS C:\WINDOWS\system32> Invoke-WebRequest -Uri "http://packages.mycompany.local/nuget/Mycompany-Development/api/v2" -Verbose -Credential $(Get-Credential)

cmdlet Get-Credential at command pipeline position 1
Supply values for the following parameters:
Credential
VERBOSE: GET http://packages.mycompany.local/nuget/Mycompany-Development/api/v2 with 0-byte payload
Invoke-WebRequest : OData method is not implemented.
At line:1 char:1
+ Invoke-WebRequest -Uri "http://packages.mycompany.local/nuget/Mycompany- ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

jzabroski commented 6 years ago

OK, after upgrading, I noticed my feed URL changed to omit "/api/v2". I got further this time, but still no luck. In this scenario, it appears to silently fail - it uploads a blank folder to the feed (confirmed by exploring to "\packages\d$\ProGet\Packages.nugetv2\F3\Mycompany.PowerShell.SqlClone"):

PS C:\WINDOWS\system32> Register-PSRepository -Name Mycompany-Development -SourceLocation "http://packages.mycompany.local/nuget/Mycompany-Development/" -PublishLocation http://packages.mycompany.local/nuget/Mycompany-Development/api/v2/package
PS C:\WINDOWS\system32> Get-PSRepository

Name                      InstallationPolicy   SourceLocation
----                      ------------------   --------------
PSGallery                 Untrusted            https://www.powershellgallery.com/api/v2/
Mycompany-Development         Untrusted            http://packages.mycompany.local/nuget/Mycompany-Development/

PS C:\WINDOWS\system32> Publish-Module -Path C:\source\Infrastructure\SqlClone\Powershell\Mycompany.PowerShell.SqlClone\ -Repository Mycompany-Development -NuGetApiKey $secretkey -Verbose
VERBOSE: Repository details, Name = 'Mycompany-Development', Location = 'http://packages.mycompany.local/nuget/Mycompany-Development/'; IsTrusted = 'False'; IsRegistered = 'True'.
VERBOSE: Repository details, Name = 'Mycompany-Development', Location = 'http://packages.mycompany.local/nuget/Mycompany-Development/'; IsTrusted = 'False'; IsRegistered = 'True'.
VERBOSE: Publish Location:'http://packages.mycompany.local/nuget/Mycompany-Development/api/v2/package'.
VERBOSE: Module 'Mycompany.PowerShell.SqlClone' was found in 'C:\source\Infrastructure\SqlClone\Powershell\Mycompany.PowerShell.SqlClone'.
VERBOSE: Repository details, Name = 'Mycompany-Development', Location = 'http://packages.mycompany.local/nuget/Mycompany-Development/'; IsTrusted = 'False'; IsRegistered = 'True'.
VERBOSE: Using the provider 'PowerShellGet' for searching packages.
VERBOSE: Using the specified source names : 'Mycompany-Development'.
VERBOSE: Getting the provider object for the PackageManagement Provider 'NuGet'.
VERBOSE: The specified Location is 'http://packages.mycompany.local/nuget/Mycompany-Development/' and PackageManagementProvider is 'NuGet'.
VERBOSE: Searching repository 'http://packages.mycompany.local/nuget/Mycompany-Development/FindPackagesById()?id='Mycompany.PowerShell.SqlClone'' for ''.
VERBOSE: Total package yield:'0' for the specified package 'Mycompany.PowerShell.SqlClone'.
VERBOSE: Performing the operation "Publish-Module" on target "Version '1.0' of module 'Mycompany.PowerShell.SqlClone'".
VERBOSE: Successfully published module 'Mycompany.PowerShell.SqlClone' to the module publish location 'http://packages.mycompany.local/nuget/Mycompany-Development/api/v2/package'. Please allow few minutes for 'Mycompany.PowerShell.SqlClone' to show up in the search results.

Edit: I deleted the folder and tried again. Now the folder is not even being created.

bmanikm commented 6 years ago

@jzabroski Did you try publishing a nupkg to your feed using nuget.exe? Do you know if your ProGet feed requires any specific version of NuGet.exe?

jzabroski commented 6 years ago

The immediate problem here is to fix the error handling in Register-PSRepository. In my scenario, the issue appears to be rather subtle. I appended /api/v2 to the feed url. I'm not really clear on when the error I gave is actually ever useful. Are there any tests that describe the user story for the error message I got? Because when I put in a truly bogus Uri I get a totally different error.

On top of that, it's pretty clear PowerShellGet is not loading its dependencies correctly. System.Net.Http isnt loaded. That's 4 bugs I found in 5 hours. I seriously recommend a peer review of the code, checking for error handling and other best practices, because it's missing everywhere I looked

edyoung commented 6 years ago

Above PR makes the message more accurate and converts to a warning. I'm not able to repro the 'not loading dependencies properly' issue.

edyoung commented 5 years ago

Unfortunately this turned out to have more side-effects than I initially realized, so won't be included in next PowerShellGet release.

michha commented 5 years ago

I have a similar problem. I try to check if the PSGallery is present on a system.

Write-Verbose "Checking if Default PS Repository exists."
Get-PSRepository -Name 'PSGallery' -ErrorVariable ev1 -ErrorAction SilentlyContinue
If ($null -eq $ev1 -or $ev1.count -eq 0) {
    Write-Host "Default PS Repository is present."
}
else {
    Write-Host "Default PS Repository is absent."
    :
    :

Although PSGallery is present on the system, $ev1 is still set and therefor my code tries to install it . If I output the content of $ev1 it shows:

2019-01-09T08:43:17.7188680Z PSGallery Untrusted https://www.powershellgallery.com/api/v2
2019-01-09T08:43:17.7196006Z Unable to find type [System.Net.Http.HttpClient]. Unable to find type [System.Net.Http.HttpClient].

PowerShellGet version is 1.0.0.1 on Windows Server 2016 (14393.2551)

jzabroski commented 5 years ago

@edyoung Can we at least see the pull request you were planning to put together?

edyoung commented 5 years ago

PowerShell/PowerShellGetv2#352 . The main problem is that there is logic which tries to determine whether the repo supports /api/v2 or api/v3 and/or has a scripts endpoint. Without querying the endpoint this isn't possible to do.

DmitryLobanov commented 5 years ago

I get the same error. On both PowerShell 5.1 and PowerShell Core (6.1) when trying to Register-PSRepository for our local ProGet repo. It's funny, though, that Register-PSRepository fails if specified the following URL: https://nuget.our.domain/feeds/feed-name/, but succeeds, if specified the following URL: https://nuget.our.domain/api/v2/feed-name/. Although, entering 2nd URL in Web browser automatically switches us to the 1st URL.

That's kind of confusing.

jzabroski commented 5 years ago

@edyoung Shouldn't registering the repository be responsible for telling the PSGet the endpoint to query? If not in the URL then in metadata. Otherwise I'll just go ahead and install Office 2017 on my Windows 95

edyoung commented 5 years ago

@jzabroski I'm afraid I don't quite understand what you mean.

Register-psrepository tries (possibly too hard) to do "the right thing" - so you can say register-psrepository https://foo.com and it will add /API/v2 if that path exists on the server, for example. It will also follow redirects which may be the reason for @dmitrylobanov 's issue.

jzabroski commented 5 years ago

That's exactly what I mean. It is trying too hard. I believe Visual Studio fails on equivalent lax URLs.

On Tue, Mar 12, 2019 at 11:53 PM Edwin Young notifications@github.com wrote:

@jzabroski https://github.com/jzabroski I'm afraid I don't quite understand what you mean.

Register-psrepository tries (possibly too hard) to do "the right thing" - so you can say register-psrepository https://foo.com and it will add /API/v2 if that path exists on the server, for example. It will also follow redirects which may be the reason for @DmitryLobanov https://github.com/DmitryLobanov 's issue.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/PowerShell/PowerShellGet/issues/317#issuecomment-472269691, or mute the thread https://github.com/notifications/unsubscribe-auth/AAbT_eIgVrEfEQD98U9yuvKuzNGybGUCks5vWHY1gaJpZM4WUEQK .

adrian-andersson commented 5 years ago

I have the same problems, and a very strange work-around.

Scenario: Internal nuget repository, credentials required

Module Version Information: PackageManagement 1.3.1 & 1.4.2 PowershellGet 2.0.4 & 2.1.5

PowerShell Versions: 5.1 on Windows 10 & 6.2.1 on Ubuntu 18.04

Issue: Running register-psRepository results in error invalid uri

Workaround:

Windows: Revert the NUGET provider back to 2.8.5.208

Can be done by installing the packageProvider for Nuget, the latest version found with Find-PackageProvider -Name nuget as of this post is 2.8.5.208

Register-packageProvider nuget

You may also need to remove/rename the Microsoft.PackageManagement.NuGetProvider.dll dll from the PackageManagement module as well

Linux: register-packageprovider nuget error's out under Linux, since nuget version 2.8.5.208 does not work under Linux - however it DOES work to register the repository, just not pull/update packages

You can force it to install like this:

$nugetPath = (get-packageprovider -Name nuget).providerpath
$nugetPathBkp =  "$nugetPath.3"
Rename-Item $nugetPath -NewName $nugetPathBkp
Invoke-RestMethod -Uri "https://onegetcdn.azureedge.net/providers/Microsoft.PackageManagement.NuGetProvider-2.8.5.208.dll" -OutFile $nugetPath

#Do the register-psRepository part HERE

#Then replace the 2
Rename-Item $nugetPath -NewName "$nugetPath.2"
Copy-Item $nugetPathBkp $nugetPath

It's possible that doing the same thing on Windows would work (Using Nuget 2.8.5.208 for register, then reverting it back to the proper 3.x) but I have not yet tried this

dmpe commented 5 years ago

still facing same issue. even with powershell 7 core preview

godefroi commented 5 years ago

This is an issue for us using Artifactory, is there any chance we could get it addressed?

jzabroski commented 5 years ago

@godefroi Can you elaborate what error you get with Artifactory? I think its hard for the team to suss out if this is the same error or not.

godefroi commented 5 years ago

@jzabroski this error exactly. Artifactory doesn't register any calls at all; as far as I can tell , PowerShellGet is simply angry that the provided URI doesn't look like a URI that it expects.

my.name@MYHOSTNAME ~
$ register-psrepository -name PSModules -sourcelocation 'https://artifactory.mycompany.com/api/nuget/psmodules-local' -installationpolicy trusted
register-psrepository : The specified Uri 'https://artifactory.mycompany.com/api/nuget/psmodules-local' for parameter 'SourceLocation' is an invalid Web Uri. Please ensure that it meets the Web Uri requirements.
At line:1 char:1
+ register-psrepository -name PSModules -sourcelocation 'https: ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo          : InvalidArgument: (https://artifactory\u2026get/psmodules-local:String) [Register-PSRepository], ArgumentException
+ FullyQualifiedErrorId : InvalidWebUri,Register-PSRepository

bencebano commented 4 years ago

I have the same issues with Artifactory with some spice. We do not use https internally, and auth only needed for publishing with a nugetapikey. We have only one location, so no separate publish/source for modules and scripts. http://artifactory-pro.mycompany.com/artifactory/api/nuget/MyRepo

We have several domains, artifactory resides on the INT one. The repository works as intended from our %MYCOMPANY% domain, and our DEV domain, where there are no whitelisting policies on network level. Below the Invoke-WebRequest block: ` Invoke-WebRequest -Uri 'http://artifactory-pro.mycompany.com/artifactory/api/nuget/AppMgmtRepo/' -Verbose -Credential $(Get-Credential)

cmdlet Get-Credential at command pipeline position 1 Supply values for the following parameters: Credential VERBOSE: GET http://artifactory-pro.mycompany.com/artifactory/api/nuget/MyRepo/ with 0-byte payload VERBOSE: received -1-byte response of content type application/xml;charset=utf-8

StatusCode : 200 StatusDescription : OK Content : <?xml version="1.0" encoding="utf-8"?>

                <service xmlns="http://ww...

RawContent : HTTP/1.1 200 OK X-Artifactory-Id: $$$$$$$$$$$$$$:$$$$$$$$:$$$$$$$$:-8000 DataServiceVersion: 1.0 Transfer-Encoding: chunked Content-Type: application/xml;charset=utf-8 Date: Wed, 13 Nov 2019... Forms : {} Headers : {[X-Artifactory-Id, $$$$$$$$$$$$$$:$$$$$$$$:$$$$$$$$:-8000], [DataServiceVersion, 1.0], [Transfer-Encoding, chunked], [Content-Type, application/xml;charset=utf-8]...} Images : {} InputFields : {} Links : {} ParsedHtml : mshtml.HTMLDocumentClass RawContentLength : 570 `

Tough we do have a PROD and UAT domain, where you need to allow it through the firewall. We have let both http and https comm through, I can confirm that telnet works on port 80 and 443 from the correspondant servers, but getting this nasty invalid Web Uri error.

I have also tried the workaround mentioned at stackoverflow, it successfully registers the repo with SourceLocation set as $env:TEMP, but when I try to use Set-PsRepository cmdlet to fix it fails with same Web Uri error.

Futhermore I have tried to manually edit the "C:\Users\%USER%\AppData\Local\Microsoft\Windows\PowerShell\PowerShellGet\PSRepositories.xml" file with correct path, after restarting WinRM it picked up the endpoint without any error, altough when I try to list the modules it throws the below error: Find-Module -Repository MyRepo -Verbose VERBOSE: Repository details, Name = 'MyRepo', Location = 'http://artifactory-pro.mycompany.com/artifactory/api/nuget/MyRepo/'; IsTrusted = 'True'; IsRegistered = 'True'. VERBOSE: Using the provider 'PowerShellGet' for searching packages. VERBOSE: Using the specified source names : 'MyRepo'. VERBOSE: Getting the provider object for the PackageManagement Provider 'NuGet'. VERBOSE: The specified Location is 'http://artifactory-pro.mycompany.com/artifactory/api/nuget/MyRepo/' and PackageManagementProvider is 'NuGet'. VERBOSE: Total package yield:'0' for the specified package ''. VERBOSE: An error occurred while sending the request. VERBOSE: Retry downloading 'http://artifactory-pro.mycompany.com/artifactory/api/nuget/MyRepo/' for '2' more times VERBOSE: An error occurred while sending the request. VERBOSE: Retry downloading 'http://artifactory-pro.mycompany.com/artifactory/api/nuget/MyRepo/' for '1' more times VERBOSE: An error occurred while sending the request. VERBOSE: Retry downloading 'http://artifactory-pro.mycompany.com/artifactory/api/nuget/MyRepo/' for '0' more times WARNING: Unable to resolve package source 'http://artifactory-pro.mycompany.com/artifactory/api/nuget/MyRepo/'. VERBOSE: Total package yield:'0' for the specified package ''.

I've tried to resolve the Artifactory as backend server FQDN and IP as well, all resulted with the same errors.

These errors appears on Server 2016 Hosts with PowerShell 5.1, but only on our domains where we need to allow these connections through firewall. Everywhere else I do have upgraded WMF to 5.1, but the underlying servers spans from Server 2008 to Server 2016, and all works as it should.

All help much appreciated.

SylvainMartel commented 3 years ago

Anyone found some kind of workaround for this for the current powershellget version(2.2.5)? powershellget 3.0 has been in eternal beta(and we're not even sure we'll be able to upgrade to it, we haven't looked at the prerequisites) so it's not a solution for the moment and we simply can't register our internal repo with any new servers we build.

woter1832 commented 3 years ago

@SydneyhSmith, Please reopen this issue, or provide a fix or workaround to 2.2.x branch. As @kinwolfqc says, PSGet is still in beta, after 18 months!

I may be misunderstanding, but the error The specified Uri 'https://pkgs.dev.azure.com/<ORG>/<PROJECT>/_packaging/<FEED>/nuget/v2' for parameter 'SourceLocation' is an invalid Web Uri. Please ensure that it meets the Web Uri requirements. only happens when -Proxy parameter is used, however, after attempting to use PowerShellGet V3.0.11, I get the error: ##[error]Register-PSResourceRepository : Proxy and ProxyCredential are not yet implemented. Please rerun cmdlet with other. So, how have you "fixed" this bug in 3.0.11, if you haven't even implemented proxy support?

jamie-tillman commented 1 year ago

I have a Windows 11 system, with Powershell 5.1 installed, and the OS is fully updated as of current date. This error is occurring on it when connecting to an Azure DevOps hosted repo. How do I even begin to troubleshoot the issue?

PowerShell / PSResourceGet

Register-PSRepository raises errors about "invalid Web Uri", despite underlying httpclient recognizing the Uri and [Uri] parsing #23