More tests for non-PSGallery v2 endpoints

[alerickson]

Summary of the new feature / enhancement

Add more tests for non-PSGallery v2 endpoints (likely using ADO repositories)

Proposed technical implementation details (optional)

No response

[sean-r-williams]

@alerickson could we get some commitment from the PowerShell team to include 3rd-party providers in PSResourceGet's test suite? I think this was briefly discussed in #1535 but the text of this issue suggests ADO, not Artifactory, would be used here.

Ideally, the test harness for PSRG should include reasonable complement of 1P/3P providers (i.e. something other than ADO/Azure Artifacts, GitHub Packages, or ACR). A quick scan of GitHub issues suggests the following providers have usage within the community and varying levels of support within PSResourceGet:

• JFrog Artifactory (#1404 #1416 #1424 #1485)
• Sonatype Nexus (#1466)
• ProGet (#1403 #1413)
• GitLab (#1406)

[alerickson]

@sean-r-williams @SydneyhSmith and I are trying to work on this, but it's been a bit difficult to get funding for paid repository services. More than happy to add any that are free for the time being.

[sean-r-williams]

@alerickson Totally understand any budget constraints your team might be facing.

ProGet and GitLab both appear to have free-tier/selfhost options that do not require paid licenses, and using it strictly for NuGet integration testing likely wouldn't need any enterprise features.

For the remainder (e.g. Artifactory), I imagine the amount of engineer-cycles saved in early exposure of compat bugs (compared to reactive work to fix those bugs, review fixes from others, etc.) and faster shipment of deliverables would far offset the monetary cost.

Budget notwithstanding, clearly documenting scope-of-support (i.e. which 1P/3P providers are tested in CI/otherwise expected to work) would help manage customer expectations in this space. Currently, the [implicit] expectation is that, in PSResourceGet replacing PowerShellGet, the module will work in all areas that PowerShellGet did. Artifactory (and presumably the other providers mentioned earlier) all worked with PowerShellGet - quirks included.

If the team's limited resources mean engineering-system improvements (like CI) can only focus on specific 1P/3P providers (and other providers are therefore on a best-effort/community-support basis), then documenting this scope enables your users to make an informed decision and prevent any surprises during integration.

[oed-metzb]

@sean-r-williams

ProGet and GitLab both appear to have free-tier/selfhost options that do not require paid licenses, and using it strictly for NuGet integration testing likely wouldn't need any enterprise features.

In addition: Sonatype Nexus is also offered in an OSS version (and is also used by us in this way, in relation to #1466 ) If further information is required, I am also happy to provide it.

[alerickson]

@sean-r-williams we've documented the repositories which we support here: https://learn.microsoft.com/en-us/powershell/gallery/powershellget/supported-repositories?view=powershellget-3.x

The expectation with Artifactory (and with all repositories) was that users would be using the latest endpoint available (in Artifactory's case v3), hence why the v2 support in PSResourceGet focused on repositories that do not support v3 endpoints (namely the PowerShell Gallery).

However, we want to make the transition from using PowerShellGet 2.x to PSResourceGet as smooth as possible, which is why we are prioritizing ensuring the v2 endpoints for these repositories can support user scenarios as well.

In general, we prioritize repositories that are used the most, but since PowerShellGet/PSResourceGet does not collect telemetry we only know about this through user feedback. So the more folks are able to open issues about the repositories they're using and like other issues that are currently open, the more likely we are to prioritize those scenarios.