Hex Test Failure: unknown or incorrect license key

[thavaahariharangit]

Hex test suite is failing due to unknown or incorrect license key.

https://github.com/dependabot/dependabot-core/actions/runs/9321367540/job/25660190764?pr=9868

Findings

1. Test trying to access the private registry : https://dependabot-private.fly.dev/ and failing due to unknown or incorrect license key
2. Looks like that registry was set up as part of https://github.com/dependabot/dependabot-core/pull/5043 by a contributor and now no longer exists, and we don't own it
3. The source for that private hex registry still exists at https://github.com/sorentwo/dependabot-private-repo, so we should be able to set up a new instance of it. (Primary focus of this ticket.)
4. We could also look into setting up request stubs or VCR for these tests, so that they do not rely on a private registry being available, but that only works if the native tooling isn't making these requests.

Action Required

1. Ensuring the hex tests use a stub
2. Addressing the tests running on draft PRs

Issue Identified in below PR.

• https://github.com/dependabot/dependabot-core/pull/9868/

[thavaahariharangit]

Temporary fix is provided here

[jurre]

but that should probably be a separate issue in itself.

Is this not that issue? When I wrote this, I meant that the permanent fix should not be considered a part of the Rubocop changes.

[GarryHurleyJr]

In order to pass the hex tests, I had to not only annotate them as skipped, but also comment them out. This is not an ideal solution. I commented out the entire context block in order to make them pass. This really needs a permanent solution as it is affecting other pull requests. I will add that these tests pass for me locally but not when they are marked as skipped.

[jurre]

In order to pass the hex tests, I had to not only annotate them as skipped, but also comment them out.

I doubt that that's accurate, if the skipped test is still getting ran that seems like an issue with how it is skipped.

[jurre]

We should look into running that Hex Registry Server on localhost in CI as part of the build process rather than having to rely on an externally hosted version. I think that we can't rely on stubbing or VCR here because a native Elixir process needs to access it. Running the server locally could be a viable option instead.

[sachin-sandhu]

@jurre , Was looking at registry hosting externally, is it a viable option in this context as dependabot is kind of open source. https://hex.pm/pricing

[jeffwidman]

Was looking at registry hosting externally, is it a viable option in this context as dependabot is kind of open source. https://hex.pm/pricing

FWIW, we do have test accounts that hex.pm has generously provided us. It helps that it's in their business interest to ensure that :dependabot: works with their product.

That said, there's a chance this may not be really relevant here--depends on if a self-hosted Hex registry server is identical to this or not.

We should look into running that Hex Registry Server on localhost in CI as part of the build process rather than having to rely on an externally hosted version. I think that we can't rely on stubbing or VCR here because a native Elixir process needs to access it. Running the server locally could be a viable option instead.

While this is technically correct, given the low usage here, I do think that if it's possible to use hex.pm for these tests, we should just do that. I realize it adds an external dependency, but we've done that for other registry services. IIRC, we dynamically check if a secret is set for the registry, and if it is the test proceeds, and if not the test skips. That way the test runs in CI but not in local development. And if hex.pm is down, then the Hex ecosystem has bigger problems.