`Insufficient scope`

[ashtonian]

Description

Steps to reproduce

I attempted to spin up portus and push an image. The registry is setup via traefik at https://registry.mysite.com and portus is available https://portus.mysite.com.

The registry logs show 401 errors about invalid scope, I'm assuming something with oauth but I can't figure out what, any help would be appreciated.

Deployment information

Deployment method: docker

Configuration:

version: "3.7"

services:
  portus:
    image: opensuse/portus:2.4
    # env_file:
    #   - ./portus.env
    environment:
      - PORTUS_MACHINE_FQDN_VALUE=portus.mysite.com
      - PORTUS_DB_HOST=db
      - PORTUS_DB_DATABASE=portus_production
      - PORTUS_DB_PASSWORD=${DATABASE_PASSWORD}
      - PORTUS_DB_POOL=5
      - PORTUS_SECRET_KEY_BASE=${SECRET_KEY_BASE}
      - PORTUS_KEY_PATH=/certificates/portus.key
      - PORTUS_PASSWORD=${PORTUS_PASSWORD}
      - PORTUS_CHECK_SSL_USAGE_ENABLED='false'
      - RAILS_SERVE_STATIC_FILES='true'
    # ports:
    #   - 3000:3000
    depends_on:
      - db
    links:
      - db
    volumes:
      - secrets:/certificates:ro
    networks:
      - portus
      - public
    labels:
      traefik.enable: "true"
      traefik.docker.network: "public"
      traefik.backend: "portus"
      traefik.frontend.rule: "Host:portus.mysite.com"
      traefik.port: "3000"
      traefik.protocol: "http"
    deploy:
      labels:
        traefik.enable: "true"
        traefik.docker.network: "public"
        traefik.backend: "portus"
        traefik.frontend.rule: "Host:portus.mysite.com"
        traefik.port: "3000"
        traefik.protocol: "http"

  background:
    image: opensuse/portus:2.4
    depends_on:
      - portus
      - db
    environment:
      # Theoretically not needed, but cconfig's been buggy on this...
      - CCONFIG_PREFIX=PORTUS
      - PORTUS_MACHINE_FQDN_VALUE=portus.mysite.com

      # DB. The password for the database should definitely not be here. You are
      # probably better off with Docker Swarm secrets.
      - PORTUS_DB_HOST=db
      - PORTUS_DB_DATABASE=portus_production
      - PORTUS_DB_PASSWORD=${DATABASE_PASSWORD}
      - PORTUS_DB_POOL=5

      # Secrets. It can possibly be handled better with Swarm's secrets.
      - PORTUS_SECRET_KEY_BASE=${SECRET_KEY_BASE}
      - PORTUS_KEY_PATH=/certificates/portus.key
      - PORTUS_PASSWORD=${PORTUS_PASSWORD}

      - PORTUS_BACKGROUND=true
    links:
      - db
    # env_file:
    #   - ./portus.env
    volumes:
      - secrets:/certificates:ro
    networks:
      - portus

  db:
    image: library/mariadb:10.0.33
    command: mysqld --character-set-server=utf8 --collation-server=utf8_unicode_ci --init-connect='SET NAMES UTF8;' --innodb-flush-log-at-trx-commit=0
    # env_file:
    #   - ./portus.env
    environment:
      - MYSQL_DATABASE=portus_production
      - MYSQL_ROOT_PASSWORD=${DATABASE_PASSWORD}
    volumes:
      - mariadb:/var/lib/mysql
    networks:
      - portus

  registry:
    image: library/registry:2
    # env_file:
    #   - ./portus.env
    environment:
      # REGISTRY_HTTP_ADDR: registry.mysite.com
      # Authentication
      REGISTRY_AUTH_TOKEN_REALM: https://portus.mysite.com/v2/token
      REGISTRY_AUTH_TOKEN_SERVICE: registry.mysite.com
      REGISTRY_AUTH_TOKEN_ISSUER: portus.mysite.com
      REGISTRY_AUTH_TOKEN_ROOTCERTBUNDLE: /secrets/portus.crt

      # Portus endpoint
      REGISTRY_NOTIFICATIONS_ENDPOINTS: >
        - name: portus
          url: https://portus.mysite.com/v2/webhooks/events
          timeout: 2000ms
          threshold: 5
          backoff: 1s
    volumes:
      - registry:/var/lib/registry
      - secrets:/secrets:ro
      - ./config.yml:/etc/docker/registry/config.yml:ro
    ports:
      # - 5000:5000
      - 5001:5001 # required to access debug service
    links:
      - portus:portus
    networks:
      - portus
      - public
    labels:
      traefik.enable: "true"
      traefik.docker.network: "public"
      traefik.backend: "registry"
      traefik.frontend.rule: "Host:registry.mysite.com"
      traefik.port: "5000"
      traefik.protocol: "http"
    deploy:
      labels:
        traefik.enable: "true"
        traefik.docker.network: "public"
        traefik.backend: "registry"
        traefik.frontend.rule: "Host:registry.mysite.com"
        traefik.port: "5000"
        traefik.protocol: "http"

volumes:
  secrets:
    driver: local
    driver_opts:
      type: "none"
      o: "bind,rw"
      device: "/mnt/workspace/portus/secrets"

  mariadb:
  registry:

networks:
  public:
    external: true
  portus:

[Jean-Baptiste-Lasselle]

hi @Ashtonian : I have those last days worked hard on portus, got to the same point. And I found something :

• I tried pushing images, image after image,using different tags
• The scope terminology made me think about the repositories concept, present in quay.io as well. Say you have one repository inside your docker registry, and you named it dkronio. Then you would push an image to in that registry, to that repository named dkronio (so not in another repository), using a tag like docker push docker.mycompany.io:5000/myinfra/dkronio:1.8.3. Here, Portus uses the terminology namespace for the myinfra component of the later tag, and the repository name is dkronio.
• And this scope thing...It's like I don't have the right to push an image of dkronio to a repository inside the freshly created myinfra namespace. I then see in Portus, that two namespaces were created by default :
  • one associated to the registry , named like my registry's uri docker.mycompany.io:5000/
  • one associated to my user, the user I created when I signed up first in the Portus webapp.
• So suddenly I have an idea : If I have any permission, on any namespace , then that'd be on the one created for my user.
• Let the username I chose at signup be zorro, then, to a repository inside the zorro namespace push an image to that one, i'll have to push my image using a tag like :

docker pull docker pull dkron/dkron
export IMAGE_ID=$(docker images|grep dkron | awk '{print $3}' )
docker tag $IMAGE_ID docker.mycompany.io:5000/zorro/dkronio:1.8.3
docker push docker.mycompany.io:5000/zorro/dkronio:1.8.3

Well try that with your own username, never the less, it worked in my case.

All in all, I think this scope thing has to be about permissions management in permissions, and I consider that because I never had any issue while pushing to a private docker registry, with any "full tag path" of the form :

docker.mycompany.io:5000/whatever/ifeellike:1.8.3`  
# "whatever" and "ifeellike" are usually  completely free of choice.

Btw, thank you for the traefik setup, I was so thinking about doing that next, and completely get rid of the twisted nginx conf,to endup with a clear traefik.io, and a drastically simplified,and clear nginx.conf

[Jean-Baptiste-Lasselle]

HI again @Ashtonian :

I had more results on pushing images

Alright, here are my new results, using my latest provisioning recipe of portus :

• I use opensuse/portus:2.4.3 (update : also tested with opensuse/portus:2.5 with same fine results) and library/registry:2.6
• when i first access the web app https://portus.mycompany.io:3000/ I create a user named garcia
• Then, in the web app, I am logged in as garcia, and being an admin (first user has to be an admin), I configure the registry : I tell the portus app how to connect to my private docker registry. There is a VERY important point, and probably where the problem resides on your side.
• So here is the very important point :
  • When in the portus webapp, you configure the registry, there is an advanced button. See it? Well click on it, and you see another configuration parameter, namely External Registry Name this :

• What is so important here, is that the value you give to that parameter HAS TO MATCH ONE VARIABLE IN DOCKER-COMPOSE.YML, the REGISTRY_AUTH_TOKEN_SERVICE variable, as of this note by @adnoh (thank yu so much @adnoh !! )

But that's not every thing : now let's push anywhere

Ok, so now that you have your problem solved, of course, we want to push images to our repo (dying to actually).

So here is on example I tested exactly as I describe below, and worked as you will understand, just as portus is expected to behave :

• In the portus web app, still logged in as garcia, I create a team, which I name garciasteam. I note after that, that portus gives me the info that I (garcia), am owner, of that team. Ok, just natural.
• Then, still in the portus web app, and still logged in as garcia, I create a namespace attached the team garciasteam, namespace I chose to name garciashood.
• Being there, I wanna have dinner, so I just have a quick overview of all those things I just created, and I notice one thing :
  • Any namespace has 3 possible choices for one settings that looks like permissions : "locked" (no one can push or pull from the repo), "open to team" (only team members can pull from repo), or "public". :

• The namespace I just created, namely garciashood, is locked : portus probably locks down freshly created namespaces by default.
  • Great, immediatly, I unlocked the garciahood namespace, and now let's push :
• one image to the namespace created for my user, garcia
• one image to the namespace garciahood I created myself, for my team garciasteam
• one a freely chosen namespace, here below zorro, which I never created when logged in the portus web app
  • And here are the results, which conforms to anyone's expectations :

johnbl@poste-devops-typique:~$ docker pull node
Using default tag: latest
latest: Pulling from library/node
844c33c7e6ea: Pull complete 
ada5d61ae65d: Pull complete 
f8427fdf4292: Pull complete 
f025bafc4ab8: Pull complete 
7a9577c07934: Pull complete 
9b4289f800f5: Pull complete 
c74d80ccdeab: Pull complete 
b418965736e5: Pull complete 
fb4cff8b8d55: Pull complete 
Digest: sha256:a4ee833346b09f24095868f6a9d2c7781b6ac319821f912df05f71c6f5a4259c
Status: Downloaded newer image for node:latest
johnbl@poste-devops-typique:~$ export TEST_IMAGE_ID=$(docker images | grep node| awk '{print $3}')
johnbl@poste-devops-typique:~$ docker tag $TEST_IMAGE_ID oci-registry.pegasusio.io:5000/garciashood/node-latest:0.0.5
johnbl@poste-devops-typique:~$ docker tag $TEST_IMAGE_ID oci-registry.pegasusio.io:5000/garcia/node-latest:0.0.5
johnbl@poste-devops-typique:~$ docker login oci-registry.pegasusio.io:5000 --username jbl
Password: 
WARNING! Your password will be stored unencrypted in /home/johnbl/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
johnbl@poste-devops-typique:~$ docker push oci-registry.pegasusio.io:5000/garciashood/node-latest:0.0.5
The push refers to repository [oci-registry.pegasusio.io:5000/garciashood/node-latest]
eb6930092ccc: Pushed 
e07b73aa5089: Pushed 
bc9e904364b4: Pushed 
553039093d83: Pushed 
2e517d68c391: Pushed 
5f3a5adb8e97: Pushed 
73bfa217d66f: Pushed 
91ecdd7165d3: Pushed 
e4b20fcc48f4: Pushed 
0.0.5: digest: sha256:737b3a051de3db388aac1d4ef2e7cf6b96e6dcceb3e1f700c01e8c250d7d5500 size: 2215
johnbl@poste-devops-typique:~$ docker push oci-registry.pegasusio.io:5000/zorro/node-latest:0.0.5The push refers to repository [oci-registry.pegasusio.io:5000/zorro/node-latest]
eb6930092ccc: Preparing 
e07b73aa5089: Preparing 
bc9e904364b4: Preparing 
553039093d83: Preparing 
2e517d68c391: Preparing 
5f3a5adb8e97: Waiting 
73bfa217d66f: Waiting 
91ecdd7165d3: Waiting 
e4b20fcc48f4: Waiting 
denied: requested access to the resource is denied
johnbl@poste-devops-typique:~$ docker push oci-registry.pegasusio.io:5000/garcia/node-latest:0.0.5
The push refers to repository [oci-registry.pegasusio.io:5000/garcia/node-latest]
eb6930092ccc: Pushed 
e07b73aa5089: Pushed 
bc9e904364b4: Pushed 
553039093d83: Pushed 
2e517d68c391: Pushed 
5f3a5adb8e97: Pushed 
73bfa217d66f: Pushed 
91ecdd7165d3: Pushed 
e4b20fcc48f4: Pushed 
0.0.5: digest: sha256:737b3a051de3db388aac1d4ef2e7cf6b96e6dcceb3e1f700c01e8c250d7d5500 size: 2215
johnbl@poste-devops-typique:~$ 

Another and most important test

• create another user vercingetorix, make him an admin,
• create his own team gaulois, and a namespace for that team, alesia,
• open access to namespace alesia, to logged in members (of the team) only,
• logged in as garcia, try and push an image to alesia : which should fail.
• Then logged in as vercingetorix, add garcia to the gaulois team, with Viewer role and try again the same docker push. Should fail, because users with role Viewer can only docker pull. So we try docker pull, and it should work.
• Then logged in as vercingetorix, and modify garcia's role, as member of the gaulois team, from Viewer to Contributor role, and try again the same docker push. Should work this time.As well as docker pulls.
• finally, i'll be in Heaven, if portus RESTful API (hopefully OpenAPI compliant), would be mature enough to completely automate the above described test. I'll give news about that if asked.

The Permissions rules stated in portus' documentation :

Every Team member has a role, pick among the following three :

• Viewer: viewers can only pull from the repositories owned by the team.
• Contributor: contributors can both pull and push from the repositories owned by the team.
• Owner: owners have the same permissions as contributors, but they can also manage the list of team members. Owners can: add/remove team members and edit the role of team members.

(But there are no teams when we start working with portus, so : )

Push policies

Push policies are regulated in the user_permission.push_images.policy option in the portus configuration file /srv/Portus/config/config.yml, described here. It may take one of the following values:

• allow-teams: this is the default value and it will simply apply all the rules that have been stated above on this page. That is, push policy will be regulated through team permissions. This way, Portus administrators and team owners and contributors will be able to push to the namespaces owned by a given team.
• allow-personal: this way Portus will restrict push access to only administrators of Portus. That being said, users will still have their own personal namespace at their disposal.
• admin-only: when used, it will restrict push access to only Portus administrators. Users won’t even have a personal namespace. Use this option if you want to ensure that only Portus administrators can submit Docker images to your private registry.

Note that when either allow-personal or admin-only have been selected, then owners, contributors and viewers of a team have the same permissions on team-owned namespace: only pull access.

And About he Web UI, which is a problem in portus, well... I'll surely give news about that too. You know, i love super secret projects too :) ...

[Jean-Baptiste-Lasselle]

hi @Ashtonian I think possible I found what's going wrong :

In your docker-compose.yml, for the registry, where is this :

command: ["/bin/sh", "/etc/docker/registry/init"]

?

It's important because it makes "trusted" (by the registry), the SSL/TLS https cetificate of your token issuer, namely your portus service at https://portus.mycompany.io/v2/token. The Certificate, in this recipe, is used three times :

• once to make an authority trusted (by users's computers and firefoxes) : that's the authority who signed the SSL Certtificate , lets' call it WebCertificate.crt of the the https://portus.mycompany.io web app. To make that authority trusted, we need the CA 's own certificate, let's call it PortusCAAuthority.crt which has to either be signed by an already trusted CA, or self-signed (root ca). Well actually the offical portus docker-compose recipe wants that we use WebCertificate.crt as PortusCAAuthority.crt. The same certificate
• a second time to make an authority trusted (by the registry webhook client who sends the nofications to the protus web app) : that's the authority who signed the SSL Certtificate , we called it WebCertificate.crt of the the https://portus.mycompany.io web app. Again, To make that authority trusted, we need its own certificate, again being WebCertificate.crt, which is self-signed. Note that if you use a certificate signed by an external authority, so not self-signed, then you will need that CA's own public SSL Certificate, exactly it's the examples/compose/registry/init script , mapped inside the regisry conainer to "/etc/docker/registry/init", that will need to find the signing CA's certificate, like thawte.crt.
• and a last time to make an authority trusted (by the docker clients you use to docker push images) : to prove identity of the registry service to the docker client. Well there, its the docker registry service that has the same SSL Certifcate, used as SSL Certificate (see registry/config.yml). Oh but there's a part you miss perhaps here : it is that it's on the machine from which you docker push, that the registry's certifcate has to be trusted. So okkay, you need to do that :

# on the machine where you docker push
cp  WebCertificate.crt /usr/local/share/ca-certificates/
sudo update-ca-certificates

All in all, you still have to make sure your namespaces, users, and permissions are set accordingly to the rules I detailed above, before trying to docker push.

[ashtonian]

hi @Jean-Baptiste-Lasselle sorry for the delay in reply, I had other things to work on and this was going to take some time. So I verified what you said and debugged it down to the ssl root certificate. This is problematic because because I'm using traefik (lets encrypt) and this handles the certs in its own format not immediately accessible. I was thinking about having a separate container run lets encrypt and dump them and share that for services that need that, however I found https://github.com/ldez/traefik-certs-dumper which I think will solve the problem. Using that to expose the certs from traefik in a raw format the registry/portus should be able to consume them and solve this problem. I'll try and resolve and post a compose file.

[ashtonian]

After correctly configuring portus and registry to read the dumped traefik certificate in .key and .crt format and it just worked.

Here is an traefik v2 compose file with all related services for portus and registry. Closing.

version: "3.7"

services:
  portus:
    image: opensuse/portus:2.4.3
    # env_file:
    #   - ./portus.env
    environment:
      - PORTUS_MACHINE_FQDN_VALUE=${PORTUS_DOMAIN}
      - PORTUS_DB_HOST=db
      - PORTUS_DB_DATABASE=portus_production
      - PORTUS_DB_PASSWORD=${DATABASE_PASSWORD}
      - PORTUS_DB_POOL=5
      - PORTUS_SECRET_KEY_BASE=${SECRET_KEY_BASE}
      - PORTUS_KEY_PATH=/certificates/${PORTUS_DOMAIN}/privatekey.key
      - PORTUS_PASSWORD=${PORTUS_PASSWORD}
      - PORTUS_CHECK_SSL_USAGE_ENABLED=false
      - PORTUS_SIGNUP_ENABLED=false
      - RAILS_SERVE_STATIC_FILES=true

      - PORTUS_GRAVATAR_ENABLED=true
      - PORTUS_DELETE_ENABLED=true
      - PORTUS_DELETE_CONTRIBUTORS=false
      - PORTUS_DELETE_GARBAGE_COLLECTOR_ENABLED=true
      - PORTUS_DELETE_GARBAGE_COLLECTOR_OLDER_THAN=30
      - PORTUS_DELETE_GARBAGE_COLLECTOR_KEEP_LATEST=5

      - PORTUS_OAUTH_GITHUB_ENABLED=true
      - PORTUS_OAUTH_GITHUB_CLIENT_ID=${PORTUS_OAUTH_GITHUB_CLIENT_ID}
      - PORTUS_OAUTH_GITHUB_CLIENT_SECRET=${PORTUS_OAUTH_GITHUB_CLIENT_SECRET}
      - PORTUS_OAUTH_GITHUB_ORGANIZATION=yourteam
      # - PORTUS_OAUTH_GITHUB_TEAM=''
      # - PORTUS_OAUTH_GITHUB_DOMAIN=''
      - PORTUS_ANONYMOUS_BROWSING_ENABLED=false

      #       - PORTUS_SECURITY_CLAIR_SERVER=http://clair:6060
    # ports:
    #   - 3000:3000
    depends_on:
      - db
    links:
      - db
    volumes:
      - traefik_certs_raw:/certificates:ro
      # - secrets:/certificates:ro
    networks:
      - portus
      - public
    labels:
      - "traefik.enable=true"
      # - "traefik.http.middlewares.sslHeaders.headers.SSLHost=${PORTUS_DOMAIN}"
      - "traefik.http.routers.portus.rule=Host(`${PORTUS_DOMAIN}`)"
      - "traefik.http.routers.portus.middlewares=https_redirect, sslHeaders"
      - "traefik.http.routers.portus.service=portus"
      - "traefik.http.routers.portus.tls=true"
      - "traefik.http.routers.portus.tls.certresolver=le"
      - "traefik.http.services.portus.loadbalancer.server.port=3000"
      - "traefik.http.services.portus.loadbalancer.server.scheme=http"
      - "traefik.http.middlewares.https_redirect.redirectscheme.scheme=https" # Standard move to default when traefik fixes behavior
      - "traefik.http.middlewares.https_redirect.redirectscheme.permanent=true"
      # - "traefik.http.middlewares.sslHeaders.headers.framedeny=true"
      # - "traefik.http.middlewares.sslHeaders.headers.sslredirect=true"
      # - "traefik.http.middlewares.sslHeaders.headers.STSSeconds=315360000"
      # - "traefik.http.middlewares.sslHeaders.headers.browserXSSFilter=true"
      # - "traefik.http.middlewares.sslHeaders.headers.contentTypeNosniff=true"
      # - "traefik.http.middlewares.sslHeaders.headers.forceSTSHeader=true"
      # - "traefik.http.middlewares.sslHeaders.headers.STSIncludeSubdomains=true"
      # - "traefik.http.middlewares.sslHeaders.headers.STSPreload=true"
    deploy:
      labels:
        - "traefik.enable=true"
        # - "traefik.http.middlewares.sslHeaders.headers.SSLHost=${PORTUS_DOMAIN}"
        - "traefik.http.routers.portus.rule=Host(`${PORTUS_DOMAIN}`)"
        - "traefik.http.routers.portus.middlewares=https_redirect, sslHeaders"
        - "traefik.http.routers.portus.service=portus"
        - "traefik.http.routers.portus.tls=true"
        - "traefik.http.routers.portus.tls.certresolver=le"
        - "traefik.http.services.portus.loadbalancer.server.port=3000"
        - "traefik.http.services.portus.loadbalancer.server.scheme=http"
        # - "traefik.http.middlewares.https_redirect.redirectscheme.scheme=https" # Standard move to default when traefik fixes behavior
        # - "traefik.http.middlewares.https_redirect.redirectscheme.permanent=true"
        # - "traefik.http.middlewares.sslHeaders.headers.framedeny=true"
        # - "traefik.http.middlewares.sslHeaders.headers.sslredirect=true"
        # - "traefik.http.middlewares.sslHeaders.headers.STSSeconds=315360000"
        # - "traefik.http.middlewares.sslHeaders.headers.browserXSSFilter=true"
        # - "traefik.http.middlewares.sslHeaders.headers.contentTypeNosniff=true"
        # - "traefik.http.middlewares.sslHeaders.headers.forceSTSHeader=true"
        # - "traefik.http.middlewares.sslHeaders.headers.STSIncludeSubdomains=true"
        # - "traefik.http.middlewares.sslHeaders.headers.STSPreload=true"
  background:
    image: opensuse/portus:2.4.3
    depends_on:
      - portus
      - db
    environment:
      # Theoretically not needed, but cconfig's been buggy on this...
      - CCONFIG_PREFIX=PORTUS
      - PORTUS_MACHINE_FQDN_VALUE=${PORTUS_DOMAIN}
      - PORTUS_DB_HOST=db
      - PORTUS_DB_DATABASE=portus_production
      - PORTUS_DB_PASSWORD=${DATABASE_PASSWORD}
      - PORTUS_DB_POOL=5
      - PORTUS_SECRET_KEY_BASE=${SECRET_KEY_BASE}
      - PORTUS_KEY_PATH=/certificates/${PORTUS_DOMAIN}/privatekey.key
      - PORTUS_PASSWORD=${PORTUS_PASSWORD}
      #       - PORTUS_SECURITY_CLAIR_SERVER=http://clair:6060
      # - PORTUS_CHECK_SSL_USAGE_ENABLED=false
      - PORTUS_GRAVATAR_ENABLED=true
      - PORTUS_DELETE_ENABLED=true
      - PORTUS_DELETE_CONTRIBUTORS=false
      - PORTUS_DELETE_GARBAGE_COLLECTOR_ENABLED=true
      - PORTUS_DELETE_GARBAGE_COLLECTOR_OLDER_THAN=30
      - PORTUS_DELETE_GARBAGE_COLLECTOR_KEEP_LATEST=5

      - PORTUS_OAUTH_GITHUB_ENABLED=true
      - PORTUS_OAUTH_GITHUB_CLIENT_ID=${PORTUS_OAUTH_GITHUB_CLIENT_ID}
      - PORTUS_OAUTH_GITHUB_CLIENT_SECRET=${PORTUS_OAUTH_GITHUB_CLIENT_SECRET}
      - PORTUS_OAUTH_GITHUB_ORGANIZATION=yourteam
      # - PORTUS_OAUTH_GITHUB_TEAM=''
      # - PORTUS_OAUTH_GITHUB_DOMAIN=''
      - PORTUS_ANONYMOUS_BROWSING_ENABLED=false

      - PORTUS_BACKGROUND=true
      - PORTUS_BACKGROUND_REGISTRY_ENABLED=true
      - PORTUS_BACKGROUND_SYNC_ENABLED=true
      - PORTUS_BACKGROUND_SYNC_STRATEGY=update-delete
    links:
      - db
    # env_file:
    #   - ./portus.env
    volumes:
      - traefik_certs_raw:/certificates:ro
    networks:
      - portus

  db:
    image: library/mariadb:10.0.33
    command: mysqld --character-set-server=utf8 --collation-server=utf8_unicode_ci --init-connect='SET NAMES UTF8;' --innodb-flush-log-at-trx-commit=0
    # env_file:
    #   - ./portus.env
    environment:
      - MYSQL_DATABASE=portus_production
      - MYSQL_ROOT_PASSWORD=${DATABASE_PASSWORD}
    volumes:
      - mariadb:/var/lib/mysql
    networks:
      - portus

  # clair: TODO:
  #   image: quay.io/coreos/clair
  #   restart: unless-stopped
  #   depends_on:
  #     - postgres
  #   links:
  #     - postgres
  #     - portus
  #   ports:
  #     - "6060-6061:6060-6061"
  #   volumes:
  #     - /tmp:/tmp
  #     - ./clair/clair.yml:/clair.yml
  #   command: [-config, /clair.yml]

  registry:
    image: library/registry:2.7.1
    # env_file:
    #   - ./portus.env
    environment:
      # REGISTRY_HTTP_ADDR: ${REGISTRY_DOMAIN}
      # Authentication
      REGISTRY_AUTH_TOKEN_REALM: https://${PORTUS_DOMAIN}/v2/token
      REGISTRY_AUTH_TOKEN_SERVICE: ${REGISTRY_DOMAIN}
      REGISTRY_AUTH_TOKEN_ISSUER: ${PORTUS_DOMAIN}
      REGISTRY_AUTH_TOKEN_ROOTCERTBUNDLE: /certificates/${PORTUS_DOMAIN}/certificate.crt

      # Portus endpoint
      REGISTRY_NOTIFICATIONS_ENDPOINTS: >
        - name: portus
          url: https://${PORTUS_DOMAIN}/v2/webhooks/events
          timeout: 2000ms
          threshold: 5
          backoff: 1s
    volumes:
      - traefik_certs_raw:/certificates:ro
      - registry:/var/lib/registry
      - secrets:/secrets:ro
      - ./config.yml:/etc/docker/registry/config.yml:ro
    ports:
      # - 5000:5000
      - 5001:5001 # required to access debug service
    links:
      - portus:portus
    networks:
      - portus
      - public
    labels:
      - "traefik.enable=true"
      # - "traefik.http.middlewares.sslHeaders.headers.SSLHost=${REGISTRY_DOMAIN}"
      - "traefik.http.routers.registry.rule=Host(`${REGISTRY_DOMAIN}`)"
      - "traefik.http.routers.registry.middlewares=https_redirect, sslHeaders"
      - "traefik.http.routers.registry.service=registry"
      - "traefik.http.routers.registry.tls=true"
      - "traefik.http.routers.registry.tls.certresolver=le"
      - "traefik.http.services.registry.loadbalancer.server.port=5000"
      - "traefik.http.services.registry.loadbalancer.server.scheme=http"
      # - "traefik.http.middlewares.https_redirect.redirectscheme.scheme=https" # Standard move to default when traefik fixes behavior
      # - "traefik.http.middlewares.https_redirect.redirectscheme.permanent=true"
      # - "traefik.http.middlewares.sslHeaders.headers.framedeny=true"
      # - "traefik.http.middlewares.sslHeaders.headers.sslredirect=true"
      # - "traefik.http.middlewares.sslHeaders.headers.STSSeconds=315360000"
      # - "traefik.http.middlewares.sslHeaders.headers.browserXSSFilter=true"
      # - "traefik.http.middlewares.sslHeaders.headers.contentTypeNosniff=true"
      # - "traefik.http.middlewares.sslHeaders.headers.forceSTSHeader=true"
      # - "traefik.http.middlewares.sslHeaders.headers.STSIncludeSubdomains=true"
      # - "traefik.http.middlewares.sslHeaders.headers.STSPreload=true"
    deploy:
      labels:
      - "traefik.enable=true"
      # - "traefik.http.middlewares.sslHeaders.headers.SSLHost=${REGISTRY_DOMAIN}"
      - "traefik.http.routers.registry.rule=Host(`${REGISTRY_DOMAIN}`)"
      - "traefik.http.routers.registry.middlewares=https_redirect, sslHeaders"
      - "traefik.http.routers.registry.service=registry"
      - "traefik.http.routers.registry.tls=true"
      - "traefik.http.routers.registry.tls.certresolver=le"
      - "traefik.http.services.registry.loadbalancer.server.port=5000"
      - "traefik.http.services.registry.loadbalancer.server.scheme=http"
      # - "traefik.http.middlewares.https_redirect.redirectscheme.scheme=https" # Standard move to default when traefik fixes behavior
      # - "traefik.http.middlewares.https_redirect.redirectscheme.permanent=true"
      # - "traefik.http.middlewares.sslHeaders.headers.framedeny=true"
      # - "traefik.http.middlewares.sslHeaders.headers.sslredirect=true"
      # - "traefik.http.middlewares.sslHeaders.headers.STSSeconds=315360000"
      # - "traefik.http.middlewares.sslHeaders.headers.browserXSSFilter=true"
      # - "traefik.http.middlewares.sslHeaders.headers.contentTypeNosniff=true"
      # - "traefik.http.middlewares.sslHeaders.headers.forceSTSHeader=true"
      # - "traefik.http.middlewares.sslHeaders.headers.STSIncludeSubdomains=true"
      # - "traefik.http.middlewares.sslHeaders.headers.STSPreload=true"

  traefik:
    ports:
      - "80:80"
      - "443:443"
      - "8183:8080"
    image: traefik:2.1
    logging: *default-logging
    volumes:
      - traefik_certs:/certs
    networks:
      - public
      - dockersock
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.http_catchall.rule=HostRegexp(`{any:.+}`)"
      - "traefik.http.routers.http_catchall.entrypoints=web"
      - "traefik.http.routers.http_catchall.middlewares=https_redirect"
      - "traefik.http.middlewares.https_redirect.redirectscheme.scheme=https"
      - "traefik.http.middlewares.https_redirect.redirectscheme.permanent=true"
      - "traefik.http.middlewares.sslHeaders.headers.framedeny=true"
      - "traefik.http.middlewares.sslHeaders.headers.sslredirect=true"
      - "traefik.http.middlewares.sslHeaders.headers.STSSeconds=315360000"
      - "traefik.http.middlewares.sslHeaders.headers.browserXSSFilter=true"
      - "traefik.http.middlewares.sslHeaders.headers.contentTypeNosniff=true"
      - "traefik.http.middlewares.sslHeaders.headers.forceSTSHeader=true"
      - "traefik.http.middlewares.sslHeaders.headers.STSIncludeSubdomains=true"
      - "traefik.http.middlewares.sslHeaders.headers.STSPreload=true"
    deploy:
      labels:
        - "traefik.enable=true"
        - "traefik.http.routers.http_catchall.rule=HostRegexp(`{any:.+}`)"
        - "traefik.http.routers.http_catchall.entrypoints=web"
        - "traefik.http.routers.http_catchall.middlewares=https_redirect,sslHeaders"
        - "traefik.http.middlewares.https_redirect.redirectscheme.scheme=https"
        - "traefik.http.middlewares.https_redirect.redirectscheme.permanent=true"
        - "traefik.http.middlewares.sslHeaders.headers.framedeny=true"
        - "traefik.http.middlewares.sslHeaders.headers.sslredirect=true"
        - "traefik.http.middlewares.sslHeaders.headers.STSSeconds=315360000"
        - "traefik.http.middlewares.sslHeaders.headers.browserXSSFilter=true"
        - "traefik.http.middlewares.sslHeaders.headers.contentTypeNosniff=true"
        - "traefik.http.middlewares.sslHeaders.headers.forceSTSHeader=true"
        - "traefik.http.middlewares.sslHeaders.headers.STSIncludeSubdomains=true"
        - "traefik.http.middlewares.sslHeaders.headers.STSPreload=true"
    command:
      - "--api.dashboard=true"
      - "--api.insecure=true"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.websecure.address=:443"
      - "--providers.docker=true"
      - "--certificatesResolvers.le.acme.email=youremail.com"
      - "--certificatesResolvers.le.acme.storage=/certs/acme.json"
      - "--certificatesResolvers.le.acme.httpChallenge.entryPoint=web"
      - "--certificatesResolvers.le.acme.tlsChallenge=true"
      # - "--certificatesResolvers.le.acme.caServer=https://acme-staging-v02.api.letsencrypt.org/directory" # lets encrypt staging, remove after verified deployment
      - "--providers.docker.endpoint=tcp://mgmt_docker-proxy:2375"
      # - "--providers.docker.useBindPortIP=true"
      - "--providers.docker.exposedByDefault=false"
      - "--providers.docker.network=public"
      # - "--providers.docker.swarmMode=true"

  certDumper:
    image: ldez/traefik-certs-dumper:v2.7.0
    command:
      - "file"
      - "--version=v2"
      - "--source=/certs/acme.json"
      - "--domain-subdir=true"
      - "--dest=/dump/"
      - "--watch=true"
      # - "--crt-ext=.pem"
      # - "--key-ext=.pem"
    volumes:
      - traefik_certs:/certs
      - traefik_certs_raw:/dump

volumes:
  secrets:
    driver: local
    driver_opts:
      type: "none"
      o: "bind,rw"
      device: "/mnt/workspace/portus/secrets"
  traefik_certs_raw:
    driver: local
    driver_opts:
      type: "none"
      o: "bind,ro"
      device: "/mnt/workspace/traefik_certs_raw/"

  mariadb:
  registry:

networks:
  public:
    external: true
  dockersock:
    external: true
  portus:

[Jean-Baptiste-Lasselle]

hi @Jean-Baptiste-Lasselle sorry for the delay in reply, I had other things to work on and this was going to take some time. So I verified what you said and debugged it down to the ssl root certificate. This is problematic because because I'm using traefik (lets encrypt) and this handles the certs in its own format not immediately accessible. I was thinking about having a separate container run lets encrypt and dump them and share that for services that need that, however I found https://github.com/ldez/traefik-certs-dumper which I think will solve the problem. Using that to expose the certs from traefik in a raw format the registry/portus should be able to consume them and solve this problem. I'll try and resolve and post a compose file.

So it was from here that I had already seen your github username! :D SO much thank you for sharing all these infos about the certificates n traefik, I from the start planned to switch to traefik, but I didn' t want to do that before I understand completely every part of the network communication between portus background and registry.

[Jean-Baptiste-Lasselle]

@Ashtonian that is really excellent work on certificates and traefik, you know I have to tell you about that that I crossed roads a year and a half ago with some guys at carrefour in France, working with Google, and this guy kept saying all around that traefik can't work with SSL Certificates;and is forbidden inside all infra at carrefour. So ridiculous. I wanted to say to really tell you thank you for your work on SSL and traefik with portus, valuable.

[Jean-Baptiste-Lasselle]

@Ashtonian Hi again, I wanna mean a huuuuge thank you about the traefik cert dumper technique, it's like really great !

[Jean-Baptiste-Lasselle]

hi @Jean-Baptiste-Lasselle sorry for the delay in reply, I had other things to work on and this was going to take some time. So I verified what you said and debugged it down to the ssl root certificate. This is problematic because because I'm using traefik (lets encrypt) and this handles the certs in its own format not immediately accessible. I was thinking about having a separate container run lets encrypt and dump them and share that for services that need that, however I found https://github.com/ldez/traefik-certs-dumper which I think will solve the problem. Using that to expose the certs from traefik in a raw format the registry/portus should be able to consume them and solve this problem. I'll try and resolve and post a compose file.

So, just to be sure I understand everything in your config :

• The SSL certificates you use in your config, are issued by Let's Encrypt,
• and Traefik is configured with an ACME provider, https://docs.traefik.io/https/acme/ , am I right ?
• The certificates are requested and automatically renewed by traefik itself, acting as would a certbot,
• And those certificates are dumped into files, thanks to https://github.com/ldez/traefik-certs-dumper , so containers have access to them through docker volume sharing, in your case the traefik_certs_raw volume.
• Do you confirm when you run your docker-compose.yml, you have a ./traefik_certs/ in the folder where your docker-compose.yml file is ?
• And If I understand well, what the cert dumper does, is reading the content of the file /certs/acme.json, and generate actual *.key and *.crt/*.cert files into the /dump folder inside the container. Am I right there ?

[Jean-Baptiste-Lasselle]

@Ashtonian I also really like you secret dedicated docker volume definition. I'll use that too from now on, in addition with HashiCorp Vault as Secret Manager

[Jean-Baptiste-Lasselle]

Hi @Ashtonian, I dumped this about PORTUS_SECRET_KEY_BASE ,as a thank you for making me understand your traefik-cert-dumper, sharing it with community.