Authentication Issues with JFrog Container Registry as Fallback for Cloudflare Serverless Registry

[anonsaber]

Description:

We are currently using JFrog Container Registry license 7.59.12 rev 75912900 as a fallback source for our serverless registry setup.

Issue:

When we disable authentication on JCR, the fallback mechanism works flawlessly. However, once authentication is enabled, we encounter persistent 401 Unauthorized errors, regardless of whether we use a password or a token for authentication.

Code Snippet:

const params = new URLSearchParams({
  service: ctx.service,
  // explicitly include that we don't want an offline_token.
  scope: `repository:${this.url.pathname.slice(1)}/image:pull,push`,
  client_id: "r2registry",
  grant_type: "password",
  username: this.configuration.username,
  password: this.password(),
});

let res = await fetch(ctx.realm, {
  headers: {
    "Content-Type": "application/x-www-form-urlencoded",
    "User-Agent": "Docker-Client/24.0.5 (linux)",
  },
  method: "POST",
  body: params.toString(),
});

Additional Information:

The same authentication logic works seamlessly with Google Container Registry (GCR) and Docker Hub.

We've ensured that the username and password/token are correctly configured and tested them separately with direct API calls.

Why does this code result in a 401 error with JFrog Container Registry, while it functions correctly with GCR and Docker Hub? Could there be specific configurations or headers required by JCR that we might be missing?

Any insights or suggestions would be greatly appreciated. Thank you!

[gabivlj]

Hello! Thank you for submitting the issue. I will take a look, it wouldn't surprise me if there is an edge-case with our authentication here. Can you confirm if you're using the Cloud or local version of JFrog?

[anonsaber]

@gabivlj Hi! Initially, we were using the local version, but then we switched to the cloud version because we were concerned there might be a bug in the local version. However, both versions are exhibiting the same behavior.

[gabivlj]

Can you show the fallback configuration?

[gabivlj]

Hello, I fixed the issue. Let me know if this PR works for you: https://github.com/cloudflare/serverless-registry/pull/55

Please, also make sure that the "registry" property includes your namespace in the path as well (only the url doesn't work).

[anonsaber]

@gabivlj Thank you so much for your effort in addressing the issue. It seems that the problem has been resolved in the cloud version, but unfortunately, it still persists in the self-hosted version. Initially, the verification seemed to succeed, but subsequent operations failed.

here is the configuration file I am using:

name = "r2-registry"

workers_dev = true
main = "./index.ts"
compatibility_date = "2022-04-18"
compatibility_flags = ["streams_enable_constructors"]

[env.production]
r2_buckets = [
  { binding = "REGISTRY", bucket_name = "r2-registry" }
]

[env.production.vars]
REGISTRIES_JSON = "[{ \"registry\": \"https://jcr-example.motofans.club\", \"password_env\": \"REGISTRY_TOKEN\", \"username\": \"cftest\" }]"

I've also included the server-side error logs that were generated during the process:

  "logs": [
    {
      "message": [
        "https://jcr-example.motofans.club",
        "Oauth 404/401/405... Falling back to simple token authentication, see https://distribution.github.io/distribution/spec/auth/token"
      ],
      "level": "debug",
      "timestamp": 1727341977424
    },
    {
      "message": [
        "sending authentication parameters:",
        "https://jcr-example.motofans.club:443/artifactory/api/docker/null/v2/token?service=jcr-example.motofans.club%3A443&scope=repository%3A%2Fimage%3Apull%2Cpush&client_id=r2registry&grant_type=password"
      ],
      "level": "log",
      "timestamp": 1727341977424
    },
    {
      "message": [
        "Authenticated with registry https://jcr-example.motofans.club successfully, got token that expires in undefined seconds"
      ],
      "level": "debug",
      "timestamp": 1727341978141
    },
    {
      "message": [
        "https://jcr-example.motofans.club:443/v2//based/1panel/kubepi/manifests/v1.7.0",
        "->",
        403,
        "getting manifest:",
        ""
      ],
      "level": "warn",
      "timestamp": 1727341978875
    }
  ],

I hope this information helps in identifying the issue. Please let me know if there's anything else you need from my side.

Thank you once again for your assistance!

[gabivlj]

@anonsaber thank you for confirming the cloud version works! I am a bit confused on how to setup the host version. Could you add a simple setup guide for the host version that I can follow to repro locally?

[anonsaber]

@gabivlj OK, I will share a docker-compose YAML file next week.

[gabivlj]

Oops, closed without meaning to (I think after merge it closed the issue). @anonsaber, sorry!

[anonsaber]

@gabivlj Oops, it was my issue.

I tried again, I discovered that my local version is working now. Due to time constraints, I forgot that I had issued a short-term test token.

Thank you for your help; everything is working fine now.

---

Just for testing purposes, you can start an instance with an embedded database using the following docker-compose.yaml:

version: '3.8'

services:
  artifactory-jcr:
    image: jcr.motofans.club/releases-docker.jfrog.io/jfrog/artifactory-jcr:7.90.1
    container_name: artifactory-jcr
    network_mode: host
    environment:
      - JF_ROUTER_ENTRYPOINTS_EXTERNALPORT=8082
    volumes:
      - /ares-data/jfrog/artifactory/var:/var/opt/jfrog/artifactory
      - /etc/localtime:/etc/localtime:ro
    restart: always
    ulimits:
      nproc: 65535
      nofile:
        soft: 32000
        hard: 40000

And for the server configuration, use the following Nginx configuration:

server {
    listen 443 ssl;

    server_name jcr-example.motofans.club;
    if ($http_x_forwarded_proto = '') {
        set $http_x_forwarded_proto  $scheme;
    }

    ## Application specific logs
    access_log /var/log/nginx/jcr-example.motofans.club-access.log;
    error_log /var/log/nginx/jcr-example.motofans.club-error.log;

    rewrite ^/$ /ui/ redirect;
    rewrite ^/ui$ /ui/ redirect;
    chunked_transfer_encoding on;
    client_max_body_size 0;
    location / {
    proxy_read_timeout  2400s;
    proxy_pass_header   Server;
    proxy_cookie_path   ~*^/.* /;
    proxy_buffer_size 128k;
    proxy_buffers 40 128k;
    proxy_busy_buffers_size 128k;
    proxy_pass          http://127.0.0.1:8082;
    proxy_set_header    X-JFrog-Override-Base-Url $http_x_forwarded_proto://$host:$server_port;
    proxy_set_header    X-Forwarded-Port  $server_port;
    proxy_set_header    X-Forwarded-Proto $http_x_forwarded_proto;
    proxy_set_header    Host              $http_host;
    proxy_set_header    X-Forwarded-For   $proxy_add_x_forwarded_for;
    add_header X-Content-Type-Options "nosniff" always;
    add_header Strict-Transport-Security always;

        location ~ ^/artifactory/ {
            proxy_pass    http://127.0.0.1:8181;
        }
    }
}

server {
    listen 80;
    if ($host = jcr-example.motofans.club) {
        return 301 https://$host$request_uri;
    }
    server_name jcr-example.motofans.club;
    return 404;
}

This configuration will help you set up the server with the specified settings.