find-psresource [ 3.0.0-beta10] not listing all modules with artifactory repository

[adrian-andersson]

Running find-psresource has strange behaviour when the NUGET repository is hosted in Artifactory

Our repository, 'powershelf', has 760 artifacts over 53 unique modules. I can list all of them with version 2.2.5

With powershellget beta, I only see 4

PS 5.1 with PowerShellGet 2.2.5:

ps:\> (find-module -repository powershelf -Credential $creds).count 

53

PS 7.1.0-rc.1 with PowerShellGet 3.0.0-beta10:

ps:\> (Find-PSResource -Repository powershelf -Credential $creds).count
4

When I search for a specific module not listed within the generic find, I get my module just fine.

Additional odd behavior, when Publishing with the publish-psresource, I can not find it by name, only with no name specified, or a * specified for the name. This resolves after about 30 minutes, may be a caching issue

[SydneyhSmith]

Thanks @adrian-andersson we havent finished the wildcard implementation for v3 nuget feeds (like artifactory) so this likely what is causing the issue. We are interested that 4 made it through--it would be interesting to know if there is some commonality between those 4 that are returned....we have a few theories as to what may be happening (it may only be hitting the first layer of the repository index) but this would help us diagnose the issue...thanks!

[adrian-andersson]

Hi @SydneyhSmith I did a comparison and noted that find-psresource command is returning the first 4 modules in alphabetical order, and I can get the same result with find-module -repository powershelf -Credential $creds|sort-object -Property name |select-object -first 4

Happy to close and revisit this at a later date

[SydneyhSmith]

Thanks @adrian-andersson we still plan to address this

[StevenBucher98]

Hi @adrian-andersson, is this still an issue with the latest preview?

[ghost]

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment.

[adrian-andersson]

Thanks @StevenBucher98

Seems in this beta, with Find-PSResource -name is now a mandatory param. Wildcards don't seem to work with Artifactory repositories with nuget feeds v2 or v3 though. As a result, Unable to 100% verify this is resolved. It works for psgallery though.

Wildcards on artifactory feeds is a pretty big mess. Example using my artifactory repo (Powershelf) and comparing to PSGallery:

Wildcards are clearly not working. It might be an artifactory limitation so I will keep experimenting

get-module powershellget

ModuleType Version    PreRelease Name                                ExportedCommands
---------- -------    ---------- ----                                ----------------
Binary     3.0.12     beta       PowerShellGet                       {Find-PSResource, Get-PSResource, Get-PSResourceR…

ps:\> find-psresource -Repository psgallery -Name carb*

Name                     Version  Prerelease Repository Description
----                     -------  ---------- ---------- -----------
Carbon                   2.11.0.0            PSGallery  Carbon is a PowerShell module for automating the configuration…
Carbon.Core              1.0.0.0             PSGallery  Functions that make doing things in PowerShell a little easier…
Carbon.Cryptography      2.2.0.0             PSGallery  Makes encrypting and decrypting strings and other security wor…
Carbon.Windows.Installer 1.0.0.0             PSGallery  The Carbon.Windows.Installer module is a Windows-only module t…

ps:\> find-psresource -Repository powershelf -Name carb* -Credential $creds

ps:\> find-psresource -Repository powershelf -Name carbon -Credential $creds

Name   Version Prerelease Repository Description
----   ------- ---------- ---------- -----------
Carbon 2.9.4.0            powershelf Carbon is a PowerShell module for automating the configuration Windows 7, 8, 2008, and 2012 and automation the installation and configuration of Windows applications, websites, and services. It can …

ps:\> find-psresource -Repository powershelf -Name * -Credential $creds

Name   Version Prerelease Repository Description
----   ------- ---------- ---------- -----------
Carbon 2.9.4.0            powershelf Carbon is a PowerShell module for automating the configuration Windows 7, 8, 2008, and 2012 and automation the installation and configuration of Windows applications, websites, and services. It can …

ps:\> find-psresource -Repository powershelf -Name psrapid -Credential $creds

Name    Version Prerelease Repository Description
----    ------- ---------- ---------- -----------
psrapid 1.0.2.0            powershelf A PowerShell API Framework

ps:\> find-psresource -Repository powershelf -Name psrap* -Credential $creds

[fbe75380fbd4]

Hi,

I confirm that this issue is still happening on 3.0.16-beta16 with Find-PSResource and Artifactory using wildcard.

> Find-PSResource -Name * -Repository Test -Verbose
VERBOSE: Searching in repository Test
VERBOSE: No packages found in repository: Test.

> Find-PSResource -Name PwshCore -Repository Test -Verbose
VERBOSE: Searching in repository Test

Name     Version Prerelease Repository Description
----     ------- ---------- ---------- -----------
PwshCore 4.1.4.0            Test       PowerShell wrapper for Rest API

This situation is a blocker for moving onto version 3.0 at the moment. Is there any way to contribute to your testing?

[aproposts]

I can confirm that this is also an issue for GitLab package registries at least as current as v15.2.2-ee.

[indented-automation]

I'm seeing the same for nuget feeds under Nexus OSS 3.38 configured as PS repositories.

[alerickson]

@aproposts @indented-automation could you let us know the url for your registries? Also are any packages at all being returned?

[aproposts]

Sorry for the slow reply, I saw @alerickson's question last week then got busy and forgot.

https://gitlab-internal.domain.com/api/v4/projects/135/packages/nuget/index.json

No packages are returned when using wildcards, but can be found if the exact name is given.

[alerickson]

@anamnavi we should test this with your changes in #1144

[SydneyhSmith]

Find all was not supported for v3 repositories because many have upstreams which could have expensive unintended consequences, additionally there was not native api support so we prioritized getting base features working before adding the customization logic for v3 support. We can special case for artifactory (and other repos) and plan to do this-- just need to finish up a few other bugs first 😄

[SydneyhSmith]

Would love to know from folks on this issue if you expect upstream sources to be picked up in a find all call?

[adrian-andersson]

Hi @SydneyhSmith I have switched roles/companies, no longer have access to Artifactory to test, sorry!.

[rickandersen-forte]

FWIW, I can get a (partial) list of modules from a GitHub Repo iff I specify the Name with one or more letters followed by a wildcard. For example:

Find-PSResource -Repository MyRepo -Name A*

I'm running PowerShell v7.3.8. Below are my package management-related versions:

Microsoft.PowerShell.PSResourceGet 1.0.0 Microsoft.PowerShell.SecretStore 1.0.6 PackageManagement 1.4.4 Microsoft.PowerShell.SecretManagement 1.1.2 PowerShellGet 3.0.22 beta22

(I realize this isn't Artifactory, but it may be related.)