Credentials ignored for custom CodeQL registries

[kjcolley7]

I know this isn't officially supported or even documented yet, but I've found the CodeQL package manager and ability to publish QL Packs to be very interesting and useful so far. I've created a global CodeQL package manager config file at ~/.codeql/qlconfig.yml and pointed it at a private (company-internal) Docker registry.

~/.codeql/qlconfig.yml:

registries:
  - url: <my private docker registry url>
    packages:
      - **
registryKind: docker
credentials:
  "<my private docker registry url>":
    token: <redacted authentication token>

However, whenever I try to publish a package there, the CodeQL CLI crashes with a null pointer exception:

Oops! A fatal internal error occurred.
java.lang.NullPointerException
    at com.semmle.cli2.pack.docker.DockerPackageRegistry.checkVersion(DockerPackageRegistry.java:219)
    at com.semmle.cli2.pack.docker.DockerPackageRegistry.publishPackageImpl(DockerPackageRegistry.java:88)
    at com.semmle.cli2.pack.PackageRegistry.publishPackage(PackageRegistry.java:128)
    at com.semmle.cli2.pack.PackageManager.publishPackage(PackageManager.java:246)
    at com.semmle.cli2.pack.PackPublishCommand.executeSubcommand(PackPublishCommand.java:99)
    at com.semmle.cli2.picocli.SubcommandCommon.call(SubcommandCommon.java:456)
    at com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:205)
    at com.semmle.cli2.picocli.SubcommandMaker.runMain(SubcommandMaker.java:214)
    at com.semmle.cli2.CodeQL.main(CodeQL.java:98)

I haven't dug very deep into the reason behind the crash, but I did find this bit of code after decompiling the CodeQL CLI:

src/com/semmle/frontend/config/CodeQLConfig.java:

public final class CodeQLConfig {
  //...
  public Optional<RegistryCredentials> findCredentials(URI uri) {
    uri = ensureEndsInSlash(uri);
    if (!"https://ghcr.io/v2/".equals(uri.toString()))
      return Optional.empty(); 
    return Optional.ofNullable(this.credentials.get(uri.toString()));
  }
  //...
}

It seems that the CodeQL CLI artificially prevents you from adding authentication credentials to custom private Docker registries, which seems counterintuitive. If someone has added a credentials key to their qlconfig.yml file, why ignore it? I think this check should be removed to allow authenticating to private Docker registries.

[aeisenberg]

Thanks for your interest in our packaging work. As you mention here, the qlconfig.yml file is not meant to be publicly used. See more context in my response in https://github.com/github/codeql-cli-binaries/issues/92.

We do not yet support external registries and when we do, we will need to implement a more sophisticated mechanism for authentication and authorization since credentials should not be stored on disk.

I will change the exception message at that point in the code to explicitly state that this feature is unsupported.

[aeisenberg]

What sort of requirements do you have? One of the blockers with this piece of work is that GHES does not yet ship with its own container registry, and so we would be targeting third party docker registries that all have slightly different mechanisms for authentication (at least initially). What sort of registry are you hoping to use?

[kjcolley7]

For now, I'm trying to use it with Artifactory. This registry supports docker login from the CLI. Different repos are exposed on different port numbers, so I would at least need to be able to set the host name and port. I think my company plans to keep GHCR disabled even when it becomes available on GHES because we use Artifactory for that purpose instead.

[aeisenberg]

Thanks for the context.