PowerShell / PowerShellGetv2

PowerShellGet is the Package Manager for PowerShell
https://www.PowerShellGallery.com
MIT License
430 stars 138 forks source link

PowerShellGet 2.1.4: Credential Providers | Authentication is STILL BROKEN #619

Open Jaykul opened 5 years ago

Jaykul commented 5 years ago

I don't know why PowerShell/PowerShellGetv2#133 was closed when this is still so COMPLETELY BROKEN

We cannot work with repositories that require credentials unless we ... ... pass credentials ... with every, ... single ... call.

That's not how any of this is supposed to work.

For example, following the docs for the Azure Artifacts CredProvider, and the blog post from earlier this month we should be able to install the credential provider:

iwr https://raw.githubusercontent.com/microsoft/artifacts-credprovider/master/helpers/installcredprovider.ps1 -out ~\Downloads\installcredprovider.ps1
~\Downloads\installcredprovider.ps1 -AddNetfx

Set something like this:

$ENV:VSS_NUGET_EXTERNAL_FEED_ENDPOINTS = @'
{
   "endpointCredentials": [{
      "endpoint":"https://poshcode.pkgs.visualstudio.com/_packaging/PowerShell/nuget/v2",
      "username":"Jaykul",
      "password":"thisshouldbeareaallylongpattokenstringyougetfromtheweb"
   }]}
'@

Register the feed:

Register-PSRepository -Name Test -Source https://poshcode.pkgs.visualstudio.com/_packaging/PowerShell/nuget/v2

And get a list of modules:

Find-Module * -Repository Test

But what actually happens is that with no error I get no results.

But if I try it with nuget:

nuget sources add -Name Test -Source https://poshcode.pkgs.visualstudio.com/_packaging/PowerShell/nuget/v2

nuget list -source Test

It works fine, and I get a list of modules...

YOU'RE DOING IT WRONG

The problem is that PowerShellGet isn't looking up the credentials against the feed address.

Instead, it's looking up (or rather, relying on nuget to) the credentials against the final url with all the parameters in it. So we can make the Find-Module command work if we change the stored endpoint to this:

$ENV:VSS_NUGET_EXTERNAL_FEED_ENDPOINTS = @'
{
   "endpointCredentials": [{
      "endpoint":"https://poshcode.pkgs.visualstudio.com/_packaging/PowerShell/nuget/v2/FindPackagesById()?id='*'&$skip={0}&$top={1}",
      "username":"Jaykul",
      "password":"thisshouldbeareaallylongpattokenstringyougetfromtheweb"
   }]}
'@

And then re-run:

Find-Module * -Repository Test

But this is because I've hard-coded the token to a particular search.

Of course the result is that when we don't use the environment variable (i.e. Remove-Item ENV:VSS_NUGET_EXTERNAL_FEED_ENDPOINTS) and do a Find-Module we get a "devicelogin" from the credential provider, but the credentials are cached for that specific URL, so if we Find-Module for any other specific module, or try anything else (like Install-Module), they will fail to find the cached credentials, and require going through the "devicelogin" again...

The worst part is that each time we go through the device login, you're creating a new PAT token that you'll never re-use...

Please, fix it again.

alerickson commented 5 years ago

Getting out a PackageMangement release today that will fix this!

tristanbarcelon commented 5 years ago

I just got version 2.1.5 of PowershellGet but I'm still not truly able to use it with AzureDevops feeds at all as described in here. Here's what I did in my repro:

1). Install Windows 10 1903 from scratch 2). Run elevated powershell session 3). Set-ExecutionPolicy -ExecutionPolicy RemoteSigned 4). Set-PSRepository -Name PSGallery -InstallationPolicy Truste 5). Install-Module -Name Package PowershellGet -Force (to install 2.1.5) 6). close powershell session and start a new one 7). Register-PSRepository -Name PrivatePSGallery -SourceLocation 'https://pkgs.dev.azure.com/myaccount/_packaging/myfeed/nuget/v2' -InstallationPolicy trusted -PackageManagementProvider nuget -Credential $vscred ($vscred is a System.Management.Automation.PSCredential object created using my Azure Devops PAT, myaccount and myfeed are our private Azure Devops account and feed names) With this version, I get past Register-PSRepository. Previously I'd get an error about SourceLocation being an invalid web uri. 8). Install-Module -Name 'MyModule' -Repository PrivatePSGallery -Credential $vscred but I receive an error instead.

image

What am I doing wrong here? I am passing a Credential parameter. Isn't this a supported use case for version 2.1.4 or newer?

I obtained a new token and used that instead to create $vscred. My issue appears to have been caused by a token which expired on June 14th. Sorry for the confusion.

SydneyhSmith commented 5 years ago

Thanks for all the feedback, and patience--we've released updates to both PackageManagement and PowerShellGet yesterday to fix this and I wanted to see if that had resolved the issues you are facing?

CC: @alerickson

jasonchester commented 5 years ago

still having a lot of trouble using Publish-Script and Publish module from an azure devops pipeline...

here's the relevant snippets from my azure-pipelines.yml

jobs:
  - job: PublishScripts
    pool:
      vmImage: 'vs2017-win2016'
    steps:
    - pwsh: |
        Install-Module -Name PackageManagement -Force -AllowClobber
        Install-Module -Name PowerShellGet -Force -AllowClobber
      displayName: Update PackageManagement

    - pwsh: |
        Import-Module -Name 'PackageManagement' -RequiredVersion '1.4.2'
        Import-Module -Name 'PowerShellGet' -RequiredVersion '2.1.5'

        $galleryName = '****'
        $galleryUrl = 'https://pkgs.dev.azure.com/****/_packaging/****/nuget/v2'

        #get azuredevops credentials
        $password = ConvertTo-SecureString $env:SYSTEM_ACCESSTOKEN -AsPlainText -Force
        $creds = New-Object System.Management.Automation.PSCredential $env:BUILD_REQUESTEDFOREMAIL, $password

        $regparams = @{
          Name = $galleryName
          InstallationPolicy = 'Trusted'
          SourceLocation = $galleryUrl
          PublishLocation = $galleryUrl
          ScriptPublishLocation = $galleryUrl
          Credential = $creds
        }

        Register-PSRepository @regparams -Verbose -Debug
      displayName: Register-PSGallery
      env:
        SYSTEM_ACCESSTOKEN: $(System.AccessToken)

even though I have specified the Credential, I am getting the device flow prompt

Starting: Register-PSGallery |  
-- | --
  | ============================================================================== |  
  | Task         : PowerShell |  
  | Description  : Run a PowerShell script on Linux, macOS, or Windows |  
  | Version      : 2.151.1 |  
  | Author       : Microsoft Corporation |  
  | Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/powershell |  
  | ============================================================================== |  
  | Generating script. |  
  | ========================== Starting Command Output =========================== |  
  | "C:\Program Files\PowerShell\6\pwsh.exe" -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command ". 'D:\a\_temp\ead52bd0-2e54-436b-ab73-50ea0d28499e.ps1'" |  
  | [Minimal] [CredentialProvider]DeviceFlow: https://pkgs.dev.azure.com/***/_packaging/***/nuget/v2 |  
  | [Minimal] [CredentialProvider]To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code EZVXTDLDQ to authenticate. |  
  | ##[error]The operation was canceled. |  
  | Finishing: Register-PSGallery'
jasonchester commented 5 years ago

When specifying the credentials using the environment variable, I also get an error.

jobs:
  - job: PublishScripts
    pool:
      vmImage: 'vs2017-win2016'

    steps:
    - pwsh: |
        Install-Module -Name PackageManagement -Force -AllowClobber
        Install-Module -Name PowerShellGet -Force -AllowClobber
      displayName: Update PackageManagement

    - pwsh: |
        Import-Module -Name 'PackageManagement' -RequiredVersion '1.4.2'
        Import-Module -Name 'PowerShellGet' -RequiredVersion '2.1.5'

        $Env:VSS_NUGET_EXTERNAL_FEED_ENDPOINTS = @"
        {"endpointCredentials":[{"endpoint":"https://pkgs.dev.azure.com/***/_packaging/***/nuget/v2","username":"ignored","password":"$($Env:SYSTEM_ACCESSTOKEN)"}]}
        "@

        $galleryName = '***'
        $galleryUrl = 'https://pkgs.dev.azure.com/***/_packaging/***/nuget/v2'

        $regparams = @{
          Name = $galleryName
          InstallationPolicy = 'Trusted'
          SourceLocation = $galleryUrl
          PublishLocation = $galleryUrl
          ScriptPublishLocation = $galleryUrl
        }

        Register-PSRepository @regparams -Verbose -Debug
      displayName: Register-PSGallery
      env:
        SYSTEM_ACCESSTOKEN: $(System.AccessToken)
2019-06-24T19:52:15.8143537Z ##[section]Starting: Register-PSGallery
2019-06-24T19:52:15.8248034Z ==============================================================================
2019-06-24T19:52:15.8248088Z Task         : PowerShell
2019-06-24T19:52:15.8248123Z Description  : Run a PowerShell script on Linux, macOS, or Windows
2019-06-24T19:52:15.8248159Z Version      : 2.151.1
2019-06-24T19:52:15.8248354Z Author       : Microsoft Corporation
2019-06-24T19:52:15.8248390Z Help         : https://docs.microsoft.com/azure/devops/pipelines/tasks/utility/powershell
2019-06-24T19:52:15.8248426Z ==============================================================================
2019-06-24T19:52:16.6774021Z Generating script.
2019-06-24T19:52:16.7214837Z ========================== Starting Command Output ===========================
2019-06-24T19:52:16.7469045Z ##[command]"C:\Program Files\PowerShell\6\pwsh.exe" -NoLogo -NoProfile -NonInteractive -ExecutionPolicy Unrestricted -Command ". 'D:\a\_temp\4741cb7d-7a6f-4067-a038-58b6c3b68f4f.ps1'"
2019-06-24T19:52:27.8573592Z DEBUG: Ping-Endpoint: location=https://pkgs.dev.azure.com/***/_packaging/***/nuget/v2, statuscode=401, resolvedLocation=https://pkgs.dev.azure.com/BrandMuscle/_packaging/BMIPowershellGallery/nuget/v2
2019-06-24T19:52:28.4476205Z DEBUG: 00:00:00.0000001 Calling New() : MethodName = 'GetDynamicOptions'
2019-06-24T19:52:28.4479098Z DEBUG: 00:00:00.0000678 Name: ***
2019-06-24T19:52:28.4484211Z DEBUG: 00:00:00.0000856 Location: https://pkgs.dev.azure.com/***/_packaging/***/nuget/v2
2019-06-24T19:52:28.4489770Z DEBUG: 00:00:00.0001024 Verbose: True
2019-06-24T19:52:28.4490534Z DEBUG: 00:00:00.0001150 Trusted: True
2019-06-24T19:52:28.4490718Z DEBUG: 00:00:00.0001262 Debug: True
2019-06-24T19:52:28.4495673Z DEBUG: 00:00:00.0011488 INVOKING PowerShell Fn Get-DynamicOptions with args Provider that has length 1
2019-06-24T19:52:28.4496932Z DEBUG: 00:00:00.0108756 In PowerShellGet Provider - 'Get-DynamicOptions'.
2019-06-24T19:52:28.4500613Z DEBUG: 00:00:00.0298542 Done calling powershell «Get-DynamicOptions» «PSModule»
2019-06-24T19:52:28.4503769Z DEBUG: 00:00:00.0329736 Calling New() : MethodName = 'GetDynamicOptions'
2019-06-24T19:52:28.4507427Z DEBUG: 00:00:00.0329975 Name: ***
2019-06-24T19:52:28.4508555Z DEBUG: 00:00:00.0330100 Location: https://pkgs.dev.azure.com/***/_packaging/***/nuget/v2
2019-06-24T19:52:28.4561839Z DEBUG: 00:00:00.0330219 Verbose: True
2019-06-24T19:52:28.4561946Z DEBUG: 00:00:00.0330341 Trusted: True
2019-06-24T19:52:28.4562000Z DEBUG: 00:00:00.0330444 Debug: True
2019-06-24T19:52:28.4562079Z DEBUG: 00:00:00.0455502 INVOKING PowerShell Fn Get-DynamicOptions with args Source that has length 1
2019-06-24T19:52:28.4562135Z DEBUG: 00:00:00.0461873 In PowerShellGet Provider - 'Get-DynamicOptions'.
2019-06-24T19:52:28.4562374Z DEBUG: 00:00:00.0612812 Done calling powershell «Get-DynamicOptions» «PSModule»
2019-06-24T19:52:28.4739235Z DEBUG: 00:00:00.0904803 Calling New() : MethodName = 'ResolvePackageSources'
2019-06-24T19:52:28.4747556Z DEBUG: 00:00:00.0911310 Name: ***
2019-06-24T19:52:28.4763422Z DEBUG: 00:00:00.0919322 PackageManagementProvider: NuGet
2019-06-24T19:52:28.4763516Z DEBUG: 00:00:00.0927072 Debug: True
2019-06-24T19:52:28.4776409Z DEBUG: 00:00:00.0933152 ProviderName: PowerShellGet
2019-06-24T19:52:28.4776797Z DEBUG: 00:00:00.0938633 ScriptPublishLocation: https://pkgs.dev.azure.com/***/_packaging/***/nuget/v2
2019-06-24T19:52:28.4913906Z DEBUG: 00:00:00.0944756 PublishLocation: https://pkgs.dev.azure.com/***/_packaging/***/nuget/v2
2019-06-24T19:52:28.5099152Z DEBUG: 00:00:00.0997836 MessageResolver: Microsoft.PowerShell.PackageManagement.Cmdlets.GetMessageString
2019-06-24T19:52:28.5099329Z DEBUG: 00:00:00.1151458 Verbose: True
2019-06-24T19:52:28.5114411Z DEBUG: 00:00:00.1207929 Location: https://pkgs.dev.azure.com/BrandMuscle/_packaging/BMIPowershellGallery/nuget/v2
2019-06-24T19:52:28.5117941Z DEBUG: 00:00:00.1236430 Trusted: True
2019-06-24T19:52:28.5140050Z DEBUG: 00:00:00.1302943 INVOKING PowerShell Fn Resolve-PackageSource with args  that has length 0
2019-06-24T19:52:28.5213906Z DEBUG: 00:00:00.1368437 In PowerShellGet Provider - 'Resolve-PackageSource'.
2019-06-24T19:52:28.5881822Z VERBOSE: Repository details, Name = 'PSGallery', Location = 'https://www.powershellgallery.com/api/v2'; IsTrusted = 'False'; IsRegistered = 'True'.
2019-06-24T19:52:28.5889863Z DEBUG: 00:00:00.2055089 Yielding package source for PSGallery at location https://www.powershellgallery.com/api/v2
2019-06-24T19:52:28.5921091Z DEBUG: 00:00:00.2083705 Done calling powershell «Resolve-PackageSource» «PSModule»
2019-06-24T19:52:28.7081217Z PackageManagement\Register-PackageSource : Unhandled Exception - Message:'One or more errors occurred. (PowerShell is in NonInteractive mode. Read and Prompt functionality is not available.)' Name:'AggregateException' Stack Trace:'   at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification)
2019-06-24T19:52:28.7082498Z    at Microsoft.PowerShell.PackageManagement.Cmdlets.RegisterPackageSource.ProcessRecordAsync()
2019-06-24T19:52:28.7082618Z    at Microsoft.PowerShell.PackageManagement.Cmdlets.AsyncCmdlet.<>c__DisplayClass83_0.<AsyncRun>b__0()'
2019-06-24T19:52:28.7083008Z At C:\Users\VssAdministrator\Documents\PowerShell\Modules\PowerShellGet\2.1.5\PSModule.psm1:11323 char:17
2019-06-24T19:52:28.7083961Z + ...     $null = PackageManagement\Register-PackageSource @PSBoundParamete ...
2019-06-24T19:52:28.7084444Z +                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
2019-06-24T19:52:28.7084889Z + CategoryInfo          : InvalidOperation: (Microsoft.PowerShel\u2026gisterPackageSource:RegisterPackageSource) [Register-PackageSource], Exception
2019-06-24T19:52:28.7085373Z + FullyQualifiedErrorId : UnhandledException,Microsoft.PowerShell.PackageManagement.Cmdlets.RegisterPackageSource
2019-06-24T19:52:28.7085716Z  
2019-06-24T19:52:28.8216220Z ##[error]PowerShell exited with code '1'.
2019-06-24T19:52:28.8556689Z ##[section]Finishing: Register-PSGallery
jasonchester commented 5 years ago

As a workaround for this broken azure devops / azure artifacts powershell repository experience you can use a local repo and the builtin nuget commands in Azure DevOps to publish.

You will need to handle the dependencies by publishing them first or specifying as externalDependencies.

I created a gist here of a pipeline that uses this workaround. https://gist.github.com/jasonchester/12597c0af92e65b728644155a19f05a3

Jaykul commented 5 years ago

@SydneyhSmith the new release is so much better, but still broken.

EVERY TIME I search for a module that is not in the authenticated feed, I get prompted for credentials.

Simple repro:

Assuming I have installed the Nuget Credential provider using the script from their github in an elevated console:

#requires -RunAsAdministrator
iwr https://raw.githubusercontent.com/microsoft/artifacts-credprovider/master/helpers/installcredprovider.ps1 -out ~\Downloads\installcredprovider.ps1
~\Downloads\installcredprovider.ps1 -AddNetfx

And I have updated to the current versions of both relevant modules:

Install-Module PackageManagement -RequiredVersion "1.4.2" -SkipPublisherCheck
Install-Module PowerShellGet -RequiredVersion "2.1.5"

In a new window ...

Register-PSRepository -Name PoshCode -SourceLocation https://pkgs.dev.azure.com/poshcode/_packaging/PowerShell/nuget/v2 -InstallationPolicy Trusted

# Find a module that's in that feed, it works! Huzzah!
Find-Module ModuleBuilder

But then ...

# Find a module that's NOT in that feed:
Find-Module PowerShellGet

At this point, it appears that whenever it fails to find a module in the Azure feed that requires credentials, it re-prompts for credentials, and regenerates a new PAT token.

If I again search for a module that does exist in my feed, it works fine. But the next time I search for one that does not exist in my feed, it prompts for credentials!

Jaykul commented 5 years ago

Ok, update ... After re-installing the credential helper (with -Force) I've been able to get this to work now ...

# -Force to install latest credential provider
iwr https://raw.githubusercontent.com/microsoft/artifacts-credprovider/master/helpers/installcredprovider.ps1 -out ~\Downloads\installcredprovider.ps1
~\Downloads\installcredprovider.ps1 -AddNetfx -Force

# Update to latest modules
Find-Module PackageManagement, PowerShellGet -Repository PSGallery | ForEach {
Install-Module -Name $_.Name -RequiredVersion $_.Version -Repository PSGallery -SkipPublisherCheck -AllowClobber
}

After exiting all PowerShell windows and re-opening, I was able to register and search for modules both in my feed and the PSGallery feed.

Register-PSRepository -Name PoshCode -SourceLocation https://pkgs.dev.azure.com/poshcode/_packaging/PowerShell/nuget/v2

Additionally, putting JSON in ENV:VSS_NUGET_EXTERNAL_FEED_ENDPOINTS seems to work now too.

Jaykul commented 5 years ago

Ugh. Ok, maybe not. I don't understand. I tested this in PS5 earlier today and it worked.

Just now I ran a Find-Module without thinking in PS6, and it went back to requesting credentials when it couldn't find a module in our repo -- and NOW, PS5 seem broken again too. I don't understand at all.

nhudacin commented 5 years ago

I'm not sure this should be closed. I'm hitting all the same errors you're hitting.

SydneyhSmith commented 5 years ago

@nhudacin we just published PowerShellGet and PackageManagement to our int gallery (www.poshtestgallery.com/api/v2) to resolve these errors. Please feel free to test these out and see if behavior is working appropriately. The credential provider should only be prompted if the -credential parameter is not passed in.

You can register the int gallery and install the respective modules by running:

Register-PSRepository -Name PoshTestGallery -SourceLocation https://www.poshtestgallery.com/api/v2
Install-Module PowerShellGet, PackageManagement -Repository PoshTestGallery -Force
nhudacin commented 5 years ago

Thanks @SydneyhSmith. I installed the two modules from your int gallery, closed all of my powershell windows, and opened up a fresh one. I'm getting all sorts of errors now:

~> Register-PSRepository -Name 'internal' -SourceLocation 'https://pkgs.dev.azure.com/my_org/_packaging/powershell/nuget/v2'   -InstallationPolicy Trusted
Start-Process : This command cannot be run due to the error: The system cannot find the file specified.
At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\2.2\PSModule.psm1:9330 char:5
+     Start-Process $filename -ArgumentList "$arguments -V minimal" `
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [Start-Process], InvalidOperationException
    + FullyQualifiedErrorId : InvalidOperationException,Microsoft.PowerShell.Commands.StartProcessCommand

Start-Process : This command cannot be run due to the error: The system cannot find the file specified.
At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\2.2\PSModule.psm1:9337 char:5
+     Start-Process $filename -ArgumentList "$argumentsNoRetry -V verbo ...
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [Start-Process], InvalidOperationException
    + FullyQualifiedErrorId : InvalidOperationException,Microsoft.PowerShell.Commands.StartProcessCommand

Exception calling "Match" with "2" argument(s): "Value cannot be null.
Parameter name: input"
At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\2.2\PSModule.psm1:9346 char:5
+     $username = [System.Text.RegularExpressions.Regex]::Match($conten ...
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : ArgumentNullException

Exception calling "Match" with "2" argument(s): "Value cannot be null.
Parameter name: input"
At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\2.2\PSModule.psm1:9347 char:5
+     $password = [System.Text.RegularExpressions.Regex]::Match($conten ...
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : ArgumentNullException

Start-Process : This command cannot be run due to the error: The system cannot find the file specified.
At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\2.2\PSModule.psm1:9330 char:5
+     Start-Process $filename -ArgumentList "$arguments -V minimal" `
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [Start-Process], InvalidOperationException
    + FullyQualifiedErrorId : InvalidOperationException,Microsoft.PowerShell.Commands.StartProcessCommand

Start-Process : This command cannot be run due to the error: The system cannot find the file specified.
At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\2.2\PSModule.psm1:9337 char:5
+     Start-Process $filename -ArgumentList "$argumentsNoRetry -V verbo ...
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [Start-Process], InvalidOperationException
    + FullyQualifiedErrorId : InvalidOperationException,Microsoft.PowerShell.Commands.StartProcessCommand

Exception calling "Match" with "2" argument(s): "Value cannot be null.
Parameter name: input"
At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\2.2\PSModule.psm1:9346 char:5
+     $username = [System.Text.RegularExpressions.Regex]::Match($conten ...
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : ArgumentNullException

Exception calling "Match" with "2" argument(s): "Value cannot be null.
Parameter name: input"
At C:\Program Files\WindowsPowerShell\Modules\PowerShellGet\2.2\PSModule.psm1:9347 char:5
+     $password = [System.Text.RegularExpressions.Regex]::Match($conten ...
+     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [], MethodInvocationException
    + FullyQualifiedErrorId : ArgumentNullException

Register-PSRepository : The specified repository 'internal' is unauthorized and cannot be registered. Try running
with -Credential.
At line:1 char:1
+ Register-PSRepository -Name 'internal' -SourceLocation 'https:// ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : RepositoryCannotBeRegistered,Register-PSRepository
alerickson commented 5 years ago

@nhudacin you’ll need the credential provider itself installed (see: https://devblogs.microsoft.com/powershell/using-powershellget-with-azure-artifacts/#). That error isn't the most helpful but if you don’t have the credential provider installed you’ll need to use the -credential parameter

clientFDJ commented 5 years ago

I fully agree with Jaykul : it is not really working as it should, is it ?

We are talking about these cmdlets when the the parameter "-Credential" is not used. Find-Module/Script Install-Module/Script Update-Module/Script Save-Module/Script Publish-Module/Script

For this reason, until it works properly, I have done the following :

  1. set NUGET_CREDENTIALPROVIDER_SESSIONTOKENCACHE_ENABLED to "true" (to cache the flow URL token)
  2. set the NUGET_PLUGIN_PATHS to a truncated path (so that the PackageManagement won't try to set the flow URL token itself)
  3. defined my own function that uses the SourceLocation of a PSRepository to get its credential, either a. via Get-CredsFromCredentialProvider (temporarily restoring the NUGET_PLUGIN_PATHS variable to a complete path) or b. via the environment variable VSS_NUGET_EXTERNAL_FEED_ENDPOINTS and the Convertfrom-JSON Cmdlet.

Now, I pass this credential returned by my function every time I browse my private PSRepository. If I don't pass the credential, I can get a warning, but I don't loose my token !

Find-Module -Repository myPSRepository -Credential (get-myCredential myPSRepository)
Publish-Module -Name myModule -NuGetApiKey (new-GUID) -Repository myPSRepository -Credential (get-myCredential myPSRepository)
SydneyhSmith commented 5 years ago

@clientFDJ thanks for reporting this, sorry that you are still hitting these issues...you mentioned that

I loose the flow URL token way too often. For example, I should not loose the credential for my private Repository when I browse PSGallery.

You are correct that this is certainly not intended...in fact calls to the Gallery do not use the credential provider so this is a bit of a strange behavior we are investigating... Are you able to tell if the cache is clearing itself? The credentials should be storing at $env:UserProfile\AppData\Local\MicrosoftCredentialProvider in the file sessiontokencache.dat ...when you experience losing the credential, are you getting re-prompted with the device flow url? Any specific re-pro steps here would be very helpful...could you also please provide the output of Get-Installed Module PowerShellGet, PackageManagement ? Thanks for all your help!

tcpaddock commented 5 years ago

I'm not able to get anything to work without -Credential being passed. I tried Jaykul's install method with setting VSS_NUGET_EXTERNAL_FEED_ENDPOINTS.

Register-PSRepository -Name "Test" -SourceLocation "https://pkgs.dev.azure.com/<Organization>/_packaging/<Feed>/nuget/v2" -InstallationPolicy Trusted

returns

Register-PSRepository : The specified repository 'Test' is unauthorized and cannot be registered. Try running with -Credential.
At line:1 char:1
+ Register-PSRepository -Name "Test" -SourceLocation "https://pkgs.dev. ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [Write-Error], WriteErrorException
    + FullyQualifiedErrorId : RepositoryCannotBeRegistered,Register-PSRepository

If I pass -Credential, it registers the repo. Once registered with -Credential, I try

Find-Module -Repository Test

which returns

WARNING: Unable to resolve package source 'https://pkgs.dev.azure.com/<Organization>/_packaging/<Feed>/nuget/v2'.

If I pass -Credential, it returns my modules. Since I can use a credential object, I know my PAT is good. What am I doing wrong?

Module versions:

Version Name             
------- ----             
2.2     PowerShellGet    
1.4.3   PackageManagement
SydneyhSmith commented 5 years ago

@tcpaddock sorry to hear that you are hitting issues, thanks for commenting on this issue. Could you try running Find-Module * -Repository Test i.e. the same as above but using the wildcard character, and see what it returns?

tcpaddock commented 5 years ago

@SydneyhSmith with Find-Module * -Repository Test I get

WARNING: Cannot access 'https://pkgs.dev.azure.com/<Organization>/_packaging/<Feed>/nuget/v2'. Are you missing 'Credential' parameter in the cmdlet?
WARNING: Unable to resolve package source 'https://pkgs.dev.azure.com/<Organization>/_packaging/<Feed>/nuget/v2'.
tcpaddock commented 5 years ago

I logged in to Azure DevOps through IE and cached credentials. I'm no longer receiving access denied. Find-Module now returns

WARNING: Query Url https://pkgs.dev.azure.com/<Organization>/_packaging/<Feed>/nuget/v2 is invalid.

There are some private functions like Resolve-Location and Ping-Endpoint that only look at the credential object being passed through. At some level, it looks like a credential object needs to be built from the VSS_NUGET_EXTERNAL_FEED_ENDPOINTS environment variable to be passed to these support functions.

Jaykul commented 5 years ago

Yeah. This is broken again in 2.2 and 1.4.3

I have PackageManagement v1.4.2 and PowerShellGet v2.1.5 and I'm able to register a feed, get that giant wall of nasty error looking stuff that ends with a "devicelogin" URL and token ... do the login, and then it works.

When I install and load the new versions (PowerShellGet v2.2 and PackageManagement v1.4.3) querying still works against the previously registered feed, but if I try to Register-PSRepository a new Azure DevOps feed, I'm getting the same bogus error that tcpaddock mentioned

Obviously I can't successfully pass credentials, they require MFA or a token...

ld0614 commented 4 years ago

Is there any update of this? When using version 2.2.1 and 1.4.4 and the device authentication method the modules can't be found or installed.

When using 2.1.5 and 1.4.2 it is possible to find modules however I am unable to download modules

molinch commented 4 years ago

Any news on that topic? It still seems buggy to me, I'm using Powershell core on Linux. At some point it showed the device login but not anymore. I often get this, but it doesn't seem to be a blocking issue:

PS /home/fabienm> Register-PSRepository -Name "PowershellAzureDevopsServices" -SourceLocation "https://pkgs.dev.azure.com/***/PartnerInfra/_packaging/InternalPowershellGallery/nuget/v2" -PublishLocation "https://pkgs.dev.azure.com/***/PartnerInfra/_packaging/InternalPowershellGallery/nuget/v2" -InstallationPolicy Trusted
Unknown option: -V
.NET Core SDK (3.0.100)
Usage: dotnet [runtime-options] [path-to-application] [arguments]

Execute a .NET Core application.
...

Furthermore when trying to find a module I constantly get:

WARNING: Cannot access 'https://pkgs.dev.azure.com/***/PartnerInfra/_packaging/InternalPowershellGallery/nuget/v2'. Are you missing 'Credential' parameter in the cmdlet?
WARNING: Unable to resolve package source 'https://pkgs.dev.azure.com/***/PartnerInfra/_packaging/InternalPowershellGallery/nuget/v2'```

If I manually pass the credentials with -Credential then it works. However I then need to pass them to each and every call which is a huge burden

@SydneyhSmith Shouldn't we get the devicelogin, once only, and then subsequent calls are automatically allowed?

SydneyhSmith commented 4 years ago

@molinch thanks for commenting on this, would it be possible for you to give us the output of running Find-Module with the debug parameter (after the repository has been registered)? Thanks!

spencerstewart commented 4 years ago

I'm also trying to adopt Azure Artifacts as a NuGet repository of PowerShell modules for our team.

From a fresh Windows 10 install, I've been using the following commands to get fellow user up and running.

Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy RemoteSigned
Install-PackageProvider -Name NuGet -MinimumVersion 2.8.5.201 -Scope CurrentUser -Force
Install-Module PowerShellGet -Scope CurrentUser -Force -AllowClobber

# Install Credential Provider
$script = Invoke-WebRequest -Uri https://raw.githubusercontent.com/microsoft/artifacts-credprovider/master/helpers/installcredprovider.ps1 -UseBasicParsing
$TempInstallScript = New-TemporaryFile
$TempInstallScript = $TempInstallScript.ToString().Replace($TempInstallScript.Extension,".ps1")
Out-File -InputObject $script.Content -FilePath $TempInstallScript
Invoke-Command -ScriptBlock {powershell.exe -file $TempInstallScript -AddNetFx -Force}
Remove-Item -Path $TempInstallScript -Force

# Add NuGet plugin path as environment variable for current user's PS profile
Add-Content -Path $PROFILE.CurrentUserAllHosts -Value '$Env:NUGET_PLUGIN_PATHS = "$($env:UserProfile)/.nuget\plugins\netfx\CredentialProvider.Microsoft\CredentialProvider.Microsoft.exe"'

# Register the Repository
Register-PSRepository TestRepo -SourceLocation "https://pkgs.dev.azure.com/<organization>/_packaging/<Feed>/nuget/v2" -PublishLocation "https://pkgs.dev.azure.com/<organization>/_packaging/<Feed>/nuget/v2"

Write-Output "Open a new PowerShell Console to find and install modules from the new repo"

After running the previous commands and opening a new PowerShell console, I can successfully run Find-Module -Repository TestRepo and view the private modules.

I thought I was out of the woods, but I cannot install the module. The following error occurs: image

When I use the -Credential flag and a PAT, I can install the module. Requiring my peers to use PAT authentication to install and update PowerShell modules is not going to be widely adopted, however. Any help or tips from other users would be helpful or even whether it's a good idea to use Azure Artifacts in this way.

Module Versions:

Version Module
1.4.6 PackageManagement
2.2.3 PowerShellGet
lymedo commented 4 years ago

What is the solution to this issue? I'm trying to connect to an Azure Artifact from a DevOps pipeline but can't do it without receiving the Device Flow prompt.

randomchance commented 4 years ago

Same here, is there a functioning work around or a path forward?

nhudacin commented 4 years ago

Ok... This has been working for me since my initial comment. When I passed it over to a colleague, they couldn't get Install-Module/Update-Module to work with our Powershell feed in Azure Devops. I spent a significant amount of time troubleshooting why my box worked but his didn't. Here's what I have, it's ugly but it works. I was able to test this on (several) Windows10 VMs to confirm.

The order is VERY important here because as @Jaykul called out above, you must Register-PSRepository using specific versions of PowershellGet & PackageManagement (2.1.5 & 1.4.2 respectively).

If you've already monkey'ed with this stuff and tried to sign in with the device workflow before, you MUST reset this before proceeding:

Ordered Step by Step

This will need to be executed in an admin console and it's called out when you should close & re-open this console. If not called out, then run the commands in the same console.

  1. Setup/Install the EXACT versions of the modules you'll need:
Set-ExecutionPolicy -Scope CurrentUser -ExecutionPolicy Bypass -Force

Install-PackageProvider -Name NuGet -MinimumVersion 2.8.5.201 -Force

Install-Module PowerShellGet -RequiredVersion 2.1.5 -Force -AllowClobber
Install-Module PackageManagement -RequiredVersion 1.4.2 -Force -AllowClobber

# use both modules
Import-Module PowershellGet -RequiredVersion 2.1.5 -Force -ErrorAction Stop
Import-Module PackageManagement  -RequiredVersion 1.4.2 -Force -ErrorAction Stop
  1. Install the credential provider that does the device code login for Azure Devops. This was pulled from above.
iwr https://raw.githubusercontent.com/microsoft/artifacts-credprovider/master/helpers/installcredprovider.ps1 -out ~\Downloads\installcredprovider.ps1
~\Downloads\installcredprovider.ps1 -AddNetfx -Force
  1. Close & Re-open your admin Powershell console. You'll need to register your PSRepository at this step. You should get the device code login workflow here. If you don't, then something is probably wrong.
$Env:NUGET_PLUGIN_PATHS = "$($env:UserProfile)/.nuget\plugins\netfx\CredentialProvider.Microsoft\CredentialProvider.Microsoft.exe"
Register-PSRepository -Name 'MyPoshFeed' -SourceLocation 'https://pkgs.dev.azure.com/my_org/_packaging/powershell/nuget/v2' -InstallationPolicy Trusted
  1. If you don't have nuget.exe available, you'll need to download it here. Adding the source to nuget allows the PSRepository to not throw errors around finding the package source. I THINK this is similar to the workaround above by setting the $env:VSS_NUGET_EXTERNAL_FEED_ENDPOINTS variable, but I cannot confirm.
.\downloads\nuget sources add -Name InternalPowershellv2 -Source  https://pkgs.dev.azure.com/my_org/_packaging/powershell/nuget/v2
  1. At this point, you'd think that everything works but you are soo wrong. You need to update to the latest PackageManagement & PowershellGet versions! Remember, this is so that Install/Update module works. You MUST register-psrepository on the versions above!
Update-Module PackageManagement
Update-Module PowershellGet
  1. Finally - The kicker.... You need to close all of your powershell windows and then download/install .NET Core (SDK). https://dotnet.microsoft.com/download. This was to get around the credential provider lookup/limitation.

At this point you can now use Azure Devops like you expect to be able to use them. Unfortunately for me and others, this is not a very clean process and will limit the adoption across my internal teams to use Azure Devops as a Powershell module repo.

This information was sourced from as many issues and blog posts I could find around this and I spent several days creating and destroying fresh Windows 10 VM's to confirm the process. Even 1 step out of order throws the whole darn thing off.

lymedo commented 4 years ago

I've finally managed to get this working in a Azure DevOps Release pipeline by including a 'NuGet authenicate' task. This now works without a Device Flow prompt in the task output:

  1. Enable 'Allow scripts to access the OAuth token' on the Agent job.
  2. Add a 'NuGet authenticate' task - if the only authenticated feeds you use are Azure Artifacts feeds in your organization, you can use the NuGet authenticate task without specifying any inputs.
steps:
- task: NuGetAuthenticate@0
  displayName: 'NuGet Authenticate'
  continueOnError: true
  1. Add a Powershell script and add the Environment Variable - SYSTEM_ACCESSTOKEN : $(System.AccessToken)

Script:

$accessToken = ConvertTo-SecureString $env:SYSTEM_ACCESSTOKEN -AsPlainText -Force
$credsAzureDevopsServices = New-Object System.Management.Automation.PSCredential $accessToken , $accessToken
$artifactLocation = "https://pkgs.dev.azure.com/<organisation>/<project>/_packaging/<feed>/nuget/v2"

$registerRepoParams = @{
          Name = "PowershellAzureDevopsServices"
          InstallationPolicy = 'Trusted'
          SourceLocation = $artifactLocation
          PublishLocation = $artifactLocation
          ScriptPublishLocation = $artifactLocation
          Credential = $credsAzureDevopsServices 
        }
Register-PSRepository @registerRepoParams 
Find-Module <module-name> -Repository "PowershellAzureDevopsServices" -Credential $credsAzureDevopsServices

If you are trying to connect from a local machine, make sure you have installed the cred provider https://github.com/microsoft/artifacts-credprovider/blob/master/README.md#setup and that 'netcore' and 'netfx' have been copied to %UserProfile%/.nuget/plugins/

Hope this helps others.

deadlydog commented 4 years ago

Until this issue is properly addressed by PowerShellGet, I've created the AzureArtifactsPowerShellModuleHelper PowerShell module, which is available on the PowerShell Gallery here. Basically as long as you have the VSS_NUGET_EXTERNAL_FEED_ENDPOINTS environment variable on the local machine, you can use this module to find, install, and update PowerShell modules from you Azure Artifacts without worrying about providing credentials; they'll be automatically injected into all of the calls for you.

It also includes a convenient Install-AndUpdateAzureArtifactsModule helper cmdlet to help address issue PowerShell/PowerShellGet#37 until it's fixed properly natively in PowerShellGet.

alerickson commented 4 years ago

@deadlydog that's great! That's in the vein of what, I was thinking should be done to address this issue in v3. Only read from the VSS_NUGET_EXTERNAL_FEED_ENDPOINTS env var and incorporate cmdlets to allow for installation and configuration. Would you be interested incorporating any of that work into v3?

deadlydog commented 4 years ago

@alerickson Yeah I'd be open to working it into v3. Is there a specific branch I should be looking at incorporating it into? Any other info I should know? Thanks.

alerickson commented 4 years ago

@deadlydog we'll have a fork with v3 up in the next few weeks. I can reach out to you as soon as that's up. There isn't anything else in particular you should know right now.

jonathaneckman commented 4 years ago

I tried most of the solutions mentioned in this thread to complete the workflow described in the blog post. I was finally able to install an existing module using the helper @deadlydog created. I haven't been able to find a way to publish a module, even when passing -Credentials.

Publish-PSArtifactUtility : Failed to publish module 'MyModule': 'dotnet cli failed to nuget push info : Pushing
MyModule.nupkg to 'https://pkgs.dev.azure.com/myorg/_packaging/myfeed/nuget/v2'...
info :   PUT https://pkgs.dev.azure.com/myorg/_packaging/myfeed/nuget/v2/
warn : The plugin credential provider could not acquire credentials. Authentication may require manual action.
Consider re-running the command with --interactive for `dotnet`, /p:NuGetInteractive="true" for MSBuild or removing
the -NonInteractive switch for `NuGet`

Any tips?

alerickson commented 4 years ago

@jonathaneckman what version of dotnet, PowerShell, and PowerShellGet are you using?

jonathaneckman commented 4 years ago

@alerickson

alerickson commented 4 years ago

@jonathaneckman thanks! A couple of folks have been running into publishing issues lately, could you let me know if the suggestion proposed here works for you?

avishnyakov commented 4 years ago

Oh, what a thread. Noticed one of my issues referenced here.

After battling https://github.com/PowerShell/PowerShellGet/issues/584 for several days to get a custom module published locally and then exploring Azure artifacts as a distribution channel custom modules, here are some learning:

Regarding Azure artifacts as a distribution channel:

Overall, I gave up and packed a custom module as a docker container to make sure that:

Well, the only challenge is limited workloads which come via mapped folders to do the work and ability to actually install the module on the target system. But can only be done with downgrading dotnet at the moment with PS Core.

randomchance commented 4 years ago

The easiest and most consistent way I have found to publish modules to Azure Artifacts is packing and pushing the module with the FSharp package manager paket - since it is a replacement for nuget it bypasses the problems with inconsistent versions and requirements, and it is a single lightweight executable.

I wrote a build task for Invoke-Build that generates a paket.template file from the module manifest, and use that to create the .nupkg file.

Some things to note:

The full workflow I'm using is:

  1. Configure and verify the connection to AzureArtifacts with paket config
  2. paket init - technically this only needs to be done once.
  3. paket install - technically this only needs to be done once.
  4. build the module
  5. New-PaketTemplate or update the old paket.template file with the new version number
  6. paket pack
  7. paket push
simonsabin commented 4 years ago

Seem to have hit this today. I can't find anyway to enable command line auth. I've created a PAT and whatever route I do, Environment variable, credential in PSreposirtory, credential in ALL module calls, I get prompted for the devicelogin flow.

I swear this was working yesterday, so wondering what has changed.

Jaykul commented 4 years ago

If you get a warning similar to this one, don't even bother signing in: image

Jaykul commented 4 years ago

Fwiw, we've resolved our publishing problems by using the PSPublishHelper module to package it, and then pushing the package with the nuget client.

ChrisAtRelativity commented 4 years ago

Seeing the same behavior Unknown option: -V as @molinch, also on Linux (Ubuntu 18.04)

Repeatable when spinning up a linux container with VSS_NUGET_EXTERNAL_FEED_ENDPOINTS env set and the wget -qO- https://aka.ms/install-artifacts-credprovider.sh | bash credential provider installed following instructions from @SydneyhSmith 's post last year.

Credentials must be provided for every command (Register... Install-Module...) and the Unknown option: -V is "thrown" although appears non-blocking as modules are installed.

ChrisAtRelativity commented 4 years ago

When trying PowershellGet 3.0 beta8, I also had to provided a Credential for Install-PSResource. In addition, there are case-sensitivity issues on Linux as of the beta8 prerelease. https://github.com/PowerShell/PowerShellGet/issues/218

218 is resolved in Beta10. Credential still appears to be required in Beta10.

sheppyh commented 3 years ago

Just wondering if this is planned to be fixed in v2 or if we need to wait for v3?

Damn, this issue has been going on for almost 4 years and is stunting adoption of Azure Artifacts where I work

sheppyh commented 3 years ago

Managed to workaround PackageManagement\Install-Package : Unable to resolve package source with Register-PackageSource -Name CorporateRepo -Location $Url -ProviderName NuGet -SkipValidate as per https://stackoverflow.com/questions/60973101/azure-powershell-module-properly-pushed-to-artifacts-feed-repository-cannot-be

k-weaver commented 3 years ago

Managed to workaround PackageManagement\Install-Package : Unable to resolve package source with Register-PackageSource -Name CorporateRepo -Location $Url -ProviderName NuGet -SkipValidate as per https://stackoverflow.com/questions/60973101/azure-powershell-module-properly-pushed-to-artifacts-feed-repository-cannot-be

This worked for me. Adding the -SkipValidate was the trick.

andyfeller commented 2 years ago

@SydneyhSmith : would you have an update on where this falls on the roadmap?

The only work around I've found for this is by adding in the PowerShell repository as a dotnet nuget source, too. I'll be honest as I've really struggled trying to get a barebones PowerShell CI pipeline and project setup together as part of https://github.com/andyfeller/powershell-poc. I really need an easy and smooth way for all of this to just work in as straight forward of a way as possible for newcomers.

StevenBucher98 commented 2 years ago

Thanks @andyfeller, we are integrating credential persistence and will likely have this out with the next release. We are integrating with Secret Management with PowerShellGet.

Here is some blog posts describing Secret Management:

andyfeller commented 2 years ago

Thanks, @StevenBucher98! ✨

This came up on my radar while trying to put this PoC together but just trying to get something working, so I hadn't delved deeper, you know? Let me dig into this and see how to incorporate to GitHub Actions.

itsme112358 commented 2 years ago

Don't know if anybody at Microsoft still cares, but the issue persists. I have a deployment group of buildservers and another of developer machines. They are identical in OS and most of the setup. After spending hours reading the docs and redoing everything many times I also found that passing credentials with every command is the only way. Well if it's so, it's so...

BUT I furthermore found out that on my buildservers registering a repository behaves differently on dev vs buildservers. I run the exact same script on both machines:

#Param
$ArtifactSource = $env:PackageFeedEndpointV2
$RegisterAs = $env:PowerShellRepositoryName

$Credential = [pscredential]::new($env:COMPUTERNAME,($env:SYSTEM_ACCESSTOKEN | ConvertTo-SecureString -AsPlainText -Force))

Register-PsRepository -Name $RegisterAs -SourceLocation $ArtifactSource -PublishLocation $ArtifactSource -InstallationPolicy Trusted -Credential $Credential

On my buildservers it runs just fine. On the dev machines it tries to use DeviceFlow four times (times out every time) before adding the repository anyway (successfully). The main difference I can find is that on dev machines Visual Studio is installed. On the build servers it isn't. I cannot say for sure that's the issue but that is what I suspect to be the issue...

I also found the fix via Add-Packagesource here: https://andrewmatveychuk.com/how-to-access-private-powershell-repository-from-azure-pipeline/

I still think it's sad to see that the issue has persisted for years, costs the community loads of time and frustration and nobody is bothering to fix it (reliably)...