Closed ashtonian closed 4 years ago
hi @Ashtonian :
I have those last days worked hard on portus
, got to the same point. And I found something :
quay.io
as well. Say you have one repository inside your docker registry, and you named it dkronio
. Then you would push an image to in that registry, to that repository named dkronio
(so not in another repository), using a tag like docker push docker.mycompany.io:5000/myinfra/dkronio:1.8.3
. Here, Portus
uses the terminology namespace
for the myinfra
component of the later tag, and the repository name is dkronio
.dkronio
to a repository
inside the freshly created myinfra
namespace
. I then see in Portus
, that two namespaces
were created by default :
docker.mycompany.io:5000/
Portus
webapp. zorro
, then, to a repository inside the zorro
namespace push an image to that one, i'll have to push my image using a tag like : docker pull docker pull dkron/dkron
export IMAGE_ID=$(docker images|grep dkron | awk '{print $3}' )
docker tag $IMAGE_ID docker.mycompany.io:5000/zorro/dkronio:1.8.3
docker push docker.mycompany.io:5000/zorro/dkronio:1.8.3
Well try that with your own username, never the less, it worked in my case.
All in all, I think this scope thing has to be about permissions management in permissions, and I consider that because I never had any issue while pushing to a private docker registry, with any "full tag path" of the form :
docker.mycompany.io:5000/whatever/ifeellike:1.8.3`
# "whatever" and "ifeellike" are usually completely free of choice.
Btw, thank you for the traefik setup, I was so thinking about doing that next, and completely get rid of the twisted nginx conf,to endup with a clear traefik.io, and a drastically simplified,and clear nginx.conf
HI again @Ashtonian :
Alright, here are my new results, using my latest provisioning recipe of portus :
opensuse/portus:2.4.3
(update : also tested with opensuse/portus:2.5
with same fine results) and library/registry:2.6
garcia
garcia
, and being an admin (first user has to be an admin), I configure the registry : I tell the portus
app how to connect to my private docker registry. There is a VERY important point, and probably where the problem resides on your side.portus
webapp, you configure the registry, there is an advanced
button. See it? Well click on it, and you see another configuration parameter, namely External Registry Name
this :
REGISTRY_AUTH_TOKEN_SERVICE
variable, as of this note by @adnoh (thank yu so much @adnoh !! ) Ok, so now that you have your problem solved, of course, we want to push images to our repo (dying to actually).
So here is on example I tested exactly as I describe below, and worked as you will understand, just as portus
is expected to behave :
portus
web app, still logged in as garcia
, I create a team, which I name garciasteam
. I note after that, that portus
gives me the info that I (garcia
), am owner, of that team. Ok, just natural.portus
web app, and still logged in as garcia
, I create a namespace attached the team garciasteam
, namespace I chose to name garciashood
.locked
" (no one can push or pull from the repo), "open to team
" (only team members can pull from repo), or "public
". : garciashood
, is locked : portus probably locks down freshly created namespaces by default.
garciahood
namespace, and now let's push : garcia
garciahood
I created myself, for my team garciasteam
zorro
, which I never created when logged in the portus
web app
johnbl@poste-devops-typique:~$ docker pull node
Using default tag: latest
latest: Pulling from library/node
844c33c7e6ea: Pull complete
ada5d61ae65d: Pull complete
f8427fdf4292: Pull complete
f025bafc4ab8: Pull complete
7a9577c07934: Pull complete
9b4289f800f5: Pull complete
c74d80ccdeab: Pull complete
b418965736e5: Pull complete
fb4cff8b8d55: Pull complete
Digest: sha256:a4ee833346b09f24095868f6a9d2c7781b6ac319821f912df05f71c6f5a4259c
Status: Downloaded newer image for node:latest
johnbl@poste-devops-typique:~$ export TEST_IMAGE_ID=$(docker images | grep node| awk '{print $3}')
johnbl@poste-devops-typique:~$ docker tag $TEST_IMAGE_ID oci-registry.pegasusio.io:5000/garciashood/node-latest:0.0.5
johnbl@poste-devops-typique:~$ docker tag $TEST_IMAGE_ID oci-registry.pegasusio.io:5000/garcia/node-latest:0.0.5
johnbl@poste-devops-typique:~$ docker login oci-registry.pegasusio.io:5000 --username jbl
Password:
WARNING! Your password will be stored unencrypted in /home/johnbl/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store
Login Succeeded
johnbl@poste-devops-typique:~$ docker push oci-registry.pegasusio.io:5000/garciashood/node-latest:0.0.5
The push refers to repository [oci-registry.pegasusio.io:5000/garciashood/node-latest]
eb6930092ccc: Pushed
e07b73aa5089: Pushed
bc9e904364b4: Pushed
553039093d83: Pushed
2e517d68c391: Pushed
5f3a5adb8e97: Pushed
73bfa217d66f: Pushed
91ecdd7165d3: Pushed
e4b20fcc48f4: Pushed
0.0.5: digest: sha256:737b3a051de3db388aac1d4ef2e7cf6b96e6dcceb3e1f700c01e8c250d7d5500 size: 2215
johnbl@poste-devops-typique:~$ docker push oci-registry.pegasusio.io:5000/zorro/node-latest:0.0.5The push refers to repository [oci-registry.pegasusio.io:5000/zorro/node-latest]
eb6930092ccc: Preparing
e07b73aa5089: Preparing
bc9e904364b4: Preparing
553039093d83: Preparing
2e517d68c391: Preparing
5f3a5adb8e97: Waiting
73bfa217d66f: Waiting
91ecdd7165d3: Waiting
e4b20fcc48f4: Waiting
denied: requested access to the resource is denied
johnbl@poste-devops-typique:~$ docker push oci-registry.pegasusio.io:5000/garcia/node-latest:0.0.5
The push refers to repository [oci-registry.pegasusio.io:5000/garcia/node-latest]
eb6930092ccc: Pushed
e07b73aa5089: Pushed
bc9e904364b4: Pushed
553039093d83: Pushed
2e517d68c391: Pushed
5f3a5adb8e97: Pushed
73bfa217d66f: Pushed
91ecdd7165d3: Pushed
e4b20fcc48f4: Pushed
0.0.5: digest: sha256:737b3a051de3db388aac1d4ef2e7cf6b96e6dcceb3e1f700c01e8c250d7d5500 size: 2215
johnbl@poste-devops-typique:~$
vercingetorix
, make him an admin, gaulois
, and a namespace for that team, alesia
, alesia
, to logged in members (of the team) only,garcia
, try and push an image to alesia
: which should fail. vercingetorix
, add garcia
to the gaulois
team, with Viewer
role and try again the same docker push. Should fail, because users with role Viewer
can only docker pull. So we try docker pull, and it should work.vercingetorix
, and modify garcia
's role, as member of the gaulois
team, from Viewer
to Contributor
role, and try again the same docker push. Should work this time.As well as docker pulls. portus
RESTful API (hopefully OpenAPI compliant), would be mature enough to completely automate the above described test. I'll give news about that if asked.portus
' documentation :Every Team member has a
role
, pick among the following three :
Viewer
: viewers can only pull from the repositories owned by the team.Contributor
:contributors
can both pull and push from the repositories owned by the team.Owner
: owners have the same permissions ascontributors
, but they can also manage the list of team members. Owners can: add/remove team members and edit the role of team members.(But there are no teams when we start working with
portus
, so : )Push policies
Push policies are regulated in the
user_permission.push_images.policy
option in theportus
configuration file/srv/Portus/config/config.yml
, described here. It may take one of the following values:
allow-teams
: this is the default value and it will simply apply all the rules that have been stated above on this page. That is, push policy will be regulated through team permissions. This way,Portus
administrators and team owners and contributors will be able to push to the namespaces owned by a given team.allow-personal
: this way Portus will restrict push access to only administrators of Portus. That being said, users will still have their own personal namespace at their disposal.admin-only
: when used, it will restrict push access to only Portus administrators. Users won’t even have a personal namespace. Use this option if you want to ensure that onlyPortus
administrators can submit Docker images to your private registry.Note that when either allow-personal or admin-only have been selected, then owners, contributors and viewers of a team have the same permissions on team-owned namespace: only pull access.
And About he Web UI, which is a problem in portus, well... I'll surely give news about that too. You know, i love super secret projects too :) ...
hi @Ashtonian I think possible I found what's going wrong :
In your docker-compose.yml
, for the registry
, where is this :
command: ["/bin/sh", "/etc/docker/registry/init"]
?
It's important because it makes "trusted" (by the registry), the SSL/TLS https cetificate of your token issuer, namely your portus service at https://portus.mycompany.io/v2/token. The Certificate, in this recipe, is used three times :
WebCertificate.crt
of the the https://portus.mycompany.io web app. To make that authority trusted, we need the CA 's own certificate, let's call it PortusCAAuthority.crt
which has to either be signed by an already trusted CA, or self-signed (root ca). Well actually the offical portus
docker-compose recipe wants that we use WebCertificate.crt
as PortusCAAuthority.crt
. The same certificateregistry
webhook client who sends the nofications to the protus web app) : that's the authority who signed the SSL Certtificate , we called it WebCertificate.crt
of the the https://portus.mycompany.io web app. Again, To make that authority trusted, we need its own certificate, again being WebCertificate.crt
, which is self-signed. Note that if you use a certificate signed by an external authority, so not self-signed, then you will need that CA's own public SSL Certificate, exactly it's the examples/compose/registry/init
script , mapped inside the regisry conainer to "/etc/docker/registry/init", that will need to find the signing CA's certificate, like thawte.crt
. docker push
images) : to prove identity of the registry
service to the docker client. Well there, its the docker registry service that has the same SSL Certifcate, used as SSL Certificate (see registry/config.yml
). Oh but there's a part you miss perhaps here : it is that it's on the machine from which you docker push
, that the registry's certifcate has to be trusted. So okkay, you need to do that : # on the machine where you docker push
cp WebCertificate.crt /usr/local/share/ca-certificates/
sudo update-ca-certificates
All in all, you still have to make sure your namespaces, users, and permissions are set accordingly to the rules I detailed above, before trying to docker push
.
hi @Jean-Baptiste-Lasselle sorry for the delay in reply, I had other things to work on and this was going to take some time. So I verified what you said and debugged it down to the ssl root certificate. This is problematic because because I'm using traefik (lets encrypt) and this handles the certs in its own format not immediately accessible. I was thinking about having a separate container run lets encrypt and dump them and share that for services that need that, however I found https://github.com/ldez/traefik-certs-dumper which I think will solve the problem. Using that to expose the certs from traefik in a raw format the registry/portus should be able to consume them and solve this problem. I'll try and resolve and post a compose file.
After correctly configuring portus and registry to read the dumped traefik certificate in .key
and .crt
format and it just worked.
Here is an traefik v2 compose file with all related services for portus and registry. Closing.
version: "3.7"
services:
portus:
image: opensuse/portus:2.4.3
# env_file:
# - ./portus.env
environment:
- PORTUS_MACHINE_FQDN_VALUE=${PORTUS_DOMAIN}
- PORTUS_DB_HOST=db
- PORTUS_DB_DATABASE=portus_production
- PORTUS_DB_PASSWORD=${DATABASE_PASSWORD}
- PORTUS_DB_POOL=5
- PORTUS_SECRET_KEY_BASE=${SECRET_KEY_BASE}
- PORTUS_KEY_PATH=/certificates/${PORTUS_DOMAIN}/privatekey.key
- PORTUS_PASSWORD=${PORTUS_PASSWORD}
- PORTUS_CHECK_SSL_USAGE_ENABLED=false
- PORTUS_SIGNUP_ENABLED=false
- RAILS_SERVE_STATIC_FILES=true
- PORTUS_GRAVATAR_ENABLED=true
- PORTUS_DELETE_ENABLED=true
- PORTUS_DELETE_CONTRIBUTORS=false
- PORTUS_DELETE_GARBAGE_COLLECTOR_ENABLED=true
- PORTUS_DELETE_GARBAGE_COLLECTOR_OLDER_THAN=30
- PORTUS_DELETE_GARBAGE_COLLECTOR_KEEP_LATEST=5
- PORTUS_OAUTH_GITHUB_ENABLED=true
- PORTUS_OAUTH_GITHUB_CLIENT_ID=${PORTUS_OAUTH_GITHUB_CLIENT_ID}
- PORTUS_OAUTH_GITHUB_CLIENT_SECRET=${PORTUS_OAUTH_GITHUB_CLIENT_SECRET}
- PORTUS_OAUTH_GITHUB_ORGANIZATION=yourteam
# - PORTUS_OAUTH_GITHUB_TEAM=''
# - PORTUS_OAUTH_GITHUB_DOMAIN=''
- PORTUS_ANONYMOUS_BROWSING_ENABLED=false
# - PORTUS_SECURITY_CLAIR_SERVER=http://clair:6060
# ports:
# - 3000:3000
depends_on:
- db
links:
- db
volumes:
- traefik_certs_raw:/certificates:ro
# - secrets:/certificates:ro
networks:
- portus
- public
labels:
- "traefik.enable=true"
# - "traefik.http.middlewares.sslHeaders.headers.SSLHost=${PORTUS_DOMAIN}"
- "traefik.http.routers.portus.rule=Host(`${PORTUS_DOMAIN}`)"
- "traefik.http.routers.portus.middlewares=https_redirect, sslHeaders"
- "traefik.http.routers.portus.service=portus"
- "traefik.http.routers.portus.tls=true"
- "traefik.http.routers.portus.tls.certresolver=le"
- "traefik.http.services.portus.loadbalancer.server.port=3000"
- "traefik.http.services.portus.loadbalancer.server.scheme=http"
- "traefik.http.middlewares.https_redirect.redirectscheme.scheme=https" # Standard move to default when traefik fixes behavior
- "traefik.http.middlewares.https_redirect.redirectscheme.permanent=true"
# - "traefik.http.middlewares.sslHeaders.headers.framedeny=true"
# - "traefik.http.middlewares.sslHeaders.headers.sslredirect=true"
# - "traefik.http.middlewares.sslHeaders.headers.STSSeconds=315360000"
# - "traefik.http.middlewares.sslHeaders.headers.browserXSSFilter=true"
# - "traefik.http.middlewares.sslHeaders.headers.contentTypeNosniff=true"
# - "traefik.http.middlewares.sslHeaders.headers.forceSTSHeader=true"
# - "traefik.http.middlewares.sslHeaders.headers.STSIncludeSubdomains=true"
# - "traefik.http.middlewares.sslHeaders.headers.STSPreload=true"
deploy:
labels:
- "traefik.enable=true"
# - "traefik.http.middlewares.sslHeaders.headers.SSLHost=${PORTUS_DOMAIN}"
- "traefik.http.routers.portus.rule=Host(`${PORTUS_DOMAIN}`)"
- "traefik.http.routers.portus.middlewares=https_redirect, sslHeaders"
- "traefik.http.routers.portus.service=portus"
- "traefik.http.routers.portus.tls=true"
- "traefik.http.routers.portus.tls.certresolver=le"
- "traefik.http.services.portus.loadbalancer.server.port=3000"
- "traefik.http.services.portus.loadbalancer.server.scheme=http"
# - "traefik.http.middlewares.https_redirect.redirectscheme.scheme=https" # Standard move to default when traefik fixes behavior
# - "traefik.http.middlewares.https_redirect.redirectscheme.permanent=true"
# - "traefik.http.middlewares.sslHeaders.headers.framedeny=true"
# - "traefik.http.middlewares.sslHeaders.headers.sslredirect=true"
# - "traefik.http.middlewares.sslHeaders.headers.STSSeconds=315360000"
# - "traefik.http.middlewares.sslHeaders.headers.browserXSSFilter=true"
# - "traefik.http.middlewares.sslHeaders.headers.contentTypeNosniff=true"
# - "traefik.http.middlewares.sslHeaders.headers.forceSTSHeader=true"
# - "traefik.http.middlewares.sslHeaders.headers.STSIncludeSubdomains=true"
# - "traefik.http.middlewares.sslHeaders.headers.STSPreload=true"
background:
image: opensuse/portus:2.4.3
depends_on:
- portus
- db
environment:
# Theoretically not needed, but cconfig's been buggy on this...
- CCONFIG_PREFIX=PORTUS
- PORTUS_MACHINE_FQDN_VALUE=${PORTUS_DOMAIN}
- PORTUS_DB_HOST=db
- PORTUS_DB_DATABASE=portus_production
- PORTUS_DB_PASSWORD=${DATABASE_PASSWORD}
- PORTUS_DB_POOL=5
- PORTUS_SECRET_KEY_BASE=${SECRET_KEY_BASE}
- PORTUS_KEY_PATH=/certificates/${PORTUS_DOMAIN}/privatekey.key
- PORTUS_PASSWORD=${PORTUS_PASSWORD}
# - PORTUS_SECURITY_CLAIR_SERVER=http://clair:6060
# - PORTUS_CHECK_SSL_USAGE_ENABLED=false
- PORTUS_GRAVATAR_ENABLED=true
- PORTUS_DELETE_ENABLED=true
- PORTUS_DELETE_CONTRIBUTORS=false
- PORTUS_DELETE_GARBAGE_COLLECTOR_ENABLED=true
- PORTUS_DELETE_GARBAGE_COLLECTOR_OLDER_THAN=30
- PORTUS_DELETE_GARBAGE_COLLECTOR_KEEP_LATEST=5
- PORTUS_OAUTH_GITHUB_ENABLED=true
- PORTUS_OAUTH_GITHUB_CLIENT_ID=${PORTUS_OAUTH_GITHUB_CLIENT_ID}
- PORTUS_OAUTH_GITHUB_CLIENT_SECRET=${PORTUS_OAUTH_GITHUB_CLIENT_SECRET}
- PORTUS_OAUTH_GITHUB_ORGANIZATION=yourteam
# - PORTUS_OAUTH_GITHUB_TEAM=''
# - PORTUS_OAUTH_GITHUB_DOMAIN=''
- PORTUS_ANONYMOUS_BROWSING_ENABLED=false
- PORTUS_BACKGROUND=true
- PORTUS_BACKGROUND_REGISTRY_ENABLED=true
- PORTUS_BACKGROUND_SYNC_ENABLED=true
- PORTUS_BACKGROUND_SYNC_STRATEGY=update-delete
links:
- db
# env_file:
# - ./portus.env
volumes:
- traefik_certs_raw:/certificates:ro
networks:
- portus
db:
image: library/mariadb:10.0.33
command: mysqld --character-set-server=utf8 --collation-server=utf8_unicode_ci --init-connect='SET NAMES UTF8;' --innodb-flush-log-at-trx-commit=0
# env_file:
# - ./portus.env
environment:
- MYSQL_DATABASE=portus_production
- MYSQL_ROOT_PASSWORD=${DATABASE_PASSWORD}
volumes:
- mariadb:/var/lib/mysql
networks:
- portus
# clair: TODO:
# image: quay.io/coreos/clair
# restart: unless-stopped
# depends_on:
# - postgres
# links:
# - postgres
# - portus
# ports:
# - "6060-6061:6060-6061"
# volumes:
# - /tmp:/tmp
# - ./clair/clair.yml:/clair.yml
# command: [-config, /clair.yml]
registry:
image: library/registry:2.7.1
# env_file:
# - ./portus.env
environment:
# REGISTRY_HTTP_ADDR: ${REGISTRY_DOMAIN}
# Authentication
REGISTRY_AUTH_TOKEN_REALM: https://${PORTUS_DOMAIN}/v2/token
REGISTRY_AUTH_TOKEN_SERVICE: ${REGISTRY_DOMAIN}
REGISTRY_AUTH_TOKEN_ISSUER: ${PORTUS_DOMAIN}
REGISTRY_AUTH_TOKEN_ROOTCERTBUNDLE: /certificates/${PORTUS_DOMAIN}/certificate.crt
# Portus endpoint
REGISTRY_NOTIFICATIONS_ENDPOINTS: >
- name: portus
url: https://${PORTUS_DOMAIN}/v2/webhooks/events
timeout: 2000ms
threshold: 5
backoff: 1s
volumes:
- traefik_certs_raw:/certificates:ro
- registry:/var/lib/registry
- secrets:/secrets:ro
- ./config.yml:/etc/docker/registry/config.yml:ro
ports:
# - 5000:5000
- 5001:5001 # required to access debug service
links:
- portus:portus
networks:
- portus
- public
labels:
- "traefik.enable=true"
# - "traefik.http.middlewares.sslHeaders.headers.SSLHost=${REGISTRY_DOMAIN}"
- "traefik.http.routers.registry.rule=Host(`${REGISTRY_DOMAIN}`)"
- "traefik.http.routers.registry.middlewares=https_redirect, sslHeaders"
- "traefik.http.routers.registry.service=registry"
- "traefik.http.routers.registry.tls=true"
- "traefik.http.routers.registry.tls.certresolver=le"
- "traefik.http.services.registry.loadbalancer.server.port=5000"
- "traefik.http.services.registry.loadbalancer.server.scheme=http"
# - "traefik.http.middlewares.https_redirect.redirectscheme.scheme=https" # Standard move to default when traefik fixes behavior
# - "traefik.http.middlewares.https_redirect.redirectscheme.permanent=true"
# - "traefik.http.middlewares.sslHeaders.headers.framedeny=true"
# - "traefik.http.middlewares.sslHeaders.headers.sslredirect=true"
# - "traefik.http.middlewares.sslHeaders.headers.STSSeconds=315360000"
# - "traefik.http.middlewares.sslHeaders.headers.browserXSSFilter=true"
# - "traefik.http.middlewares.sslHeaders.headers.contentTypeNosniff=true"
# - "traefik.http.middlewares.sslHeaders.headers.forceSTSHeader=true"
# - "traefik.http.middlewares.sslHeaders.headers.STSIncludeSubdomains=true"
# - "traefik.http.middlewares.sslHeaders.headers.STSPreload=true"
deploy:
labels:
- "traefik.enable=true"
# - "traefik.http.middlewares.sslHeaders.headers.SSLHost=${REGISTRY_DOMAIN}"
- "traefik.http.routers.registry.rule=Host(`${REGISTRY_DOMAIN}`)"
- "traefik.http.routers.registry.middlewares=https_redirect, sslHeaders"
- "traefik.http.routers.registry.service=registry"
- "traefik.http.routers.registry.tls=true"
- "traefik.http.routers.registry.tls.certresolver=le"
- "traefik.http.services.registry.loadbalancer.server.port=5000"
- "traefik.http.services.registry.loadbalancer.server.scheme=http"
# - "traefik.http.middlewares.https_redirect.redirectscheme.scheme=https" # Standard move to default when traefik fixes behavior
# - "traefik.http.middlewares.https_redirect.redirectscheme.permanent=true"
# - "traefik.http.middlewares.sslHeaders.headers.framedeny=true"
# - "traefik.http.middlewares.sslHeaders.headers.sslredirect=true"
# - "traefik.http.middlewares.sslHeaders.headers.STSSeconds=315360000"
# - "traefik.http.middlewares.sslHeaders.headers.browserXSSFilter=true"
# - "traefik.http.middlewares.sslHeaders.headers.contentTypeNosniff=true"
# - "traefik.http.middlewares.sslHeaders.headers.forceSTSHeader=true"
# - "traefik.http.middlewares.sslHeaders.headers.STSIncludeSubdomains=true"
# - "traefik.http.middlewares.sslHeaders.headers.STSPreload=true"
traefik:
ports:
- "80:80"
- "443:443"
- "8183:8080"
image: traefik:2.1
logging: *default-logging
volumes:
- traefik_certs:/certs
networks:
- public
- dockersock
labels:
- "traefik.enable=true"
- "traefik.http.routers.http_catchall.rule=HostRegexp(`{any:.+}`)"
- "traefik.http.routers.http_catchall.entrypoints=web"
- "traefik.http.routers.http_catchall.middlewares=https_redirect"
- "traefik.http.middlewares.https_redirect.redirectscheme.scheme=https"
- "traefik.http.middlewares.https_redirect.redirectscheme.permanent=true"
- "traefik.http.middlewares.sslHeaders.headers.framedeny=true"
- "traefik.http.middlewares.sslHeaders.headers.sslredirect=true"
- "traefik.http.middlewares.sslHeaders.headers.STSSeconds=315360000"
- "traefik.http.middlewares.sslHeaders.headers.browserXSSFilter=true"
- "traefik.http.middlewares.sslHeaders.headers.contentTypeNosniff=true"
- "traefik.http.middlewares.sslHeaders.headers.forceSTSHeader=true"
- "traefik.http.middlewares.sslHeaders.headers.STSIncludeSubdomains=true"
- "traefik.http.middlewares.sslHeaders.headers.STSPreload=true"
deploy:
labels:
- "traefik.enable=true"
- "traefik.http.routers.http_catchall.rule=HostRegexp(`{any:.+}`)"
- "traefik.http.routers.http_catchall.entrypoints=web"
- "traefik.http.routers.http_catchall.middlewares=https_redirect,sslHeaders"
- "traefik.http.middlewares.https_redirect.redirectscheme.scheme=https"
- "traefik.http.middlewares.https_redirect.redirectscheme.permanent=true"
- "traefik.http.middlewares.sslHeaders.headers.framedeny=true"
- "traefik.http.middlewares.sslHeaders.headers.sslredirect=true"
- "traefik.http.middlewares.sslHeaders.headers.STSSeconds=315360000"
- "traefik.http.middlewares.sslHeaders.headers.browserXSSFilter=true"
- "traefik.http.middlewares.sslHeaders.headers.contentTypeNosniff=true"
- "traefik.http.middlewares.sslHeaders.headers.forceSTSHeader=true"
- "traefik.http.middlewares.sslHeaders.headers.STSIncludeSubdomains=true"
- "traefik.http.middlewares.sslHeaders.headers.STSPreload=true"
command:
- "--api.dashboard=true"
- "--api.insecure=true"
- "--entrypoints.web.address=:80"
- "--entrypoints.websecure.address=:443"
- "--providers.docker=true"
- "--certificatesResolvers.le.acme.email=youremail.com"
- "--certificatesResolvers.le.acme.storage=/certs/acme.json"
- "--certificatesResolvers.le.acme.httpChallenge.entryPoint=web"
- "--certificatesResolvers.le.acme.tlsChallenge=true"
# - "--certificatesResolvers.le.acme.caServer=https://acme-staging-v02.api.letsencrypt.org/directory" # lets encrypt staging, remove after verified deployment
- "--providers.docker.endpoint=tcp://mgmt_docker-proxy:2375"
# - "--providers.docker.useBindPortIP=true"
- "--providers.docker.exposedByDefault=false"
- "--providers.docker.network=public"
# - "--providers.docker.swarmMode=true"
certDumper:
image: ldez/traefik-certs-dumper:v2.7.0
command:
- "file"
- "--version=v2"
- "--source=/certs/acme.json"
- "--domain-subdir=true"
- "--dest=/dump/"
- "--watch=true"
# - "--crt-ext=.pem"
# - "--key-ext=.pem"
volumes:
- traefik_certs:/certs
- traefik_certs_raw:/dump
volumes:
secrets:
driver: local
driver_opts:
type: "none"
o: "bind,rw"
device: "/mnt/workspace/portus/secrets"
traefik_certs_raw:
driver: local
driver_opts:
type: "none"
o: "bind,ro"
device: "/mnt/workspace/traefik_certs_raw/"
mariadb:
registry:
networks:
public:
external: true
dockersock:
external: true
portus:
hi @Jean-Baptiste-Lasselle sorry for the delay in reply, I had other things to work on and this was going to take some time. So I verified what you said and debugged it down to the ssl root certificate. This is problematic because because I'm using traefik (lets encrypt) and this handles the certs in its own format not immediately accessible. I was thinking about having a separate container run lets encrypt and dump them and share that for services that need that, however I found https://github.com/ldez/traefik-certs-dumper which I think will solve the problem. Using that to expose the certs from traefik in a raw format the registry/portus should be able to consume them and solve this problem. I'll try and resolve and post a compose file.
So it was from here that I had already seen your github username! :D SO much thank you for sharing all these infos about the certificates n traefik, I from the start planned to switch to traefik, but I didn' t want to do that before I understand completely every part of the network communication between portus background and registry.
@Ashtonian that is really excellent work on certificates and traefik, you know I have to tell you about that that I crossed roads a year and a half ago with some guys at carrefour in France, working with Google, and this guy kept saying all around that traefik can't work with SSL Certificates;and is forbidden inside all infra at carrefour. So ridiculous. I wanted to say to really tell you thank you for your work on SSL and traefik with portus, valuable.
@Ashtonian Hi again, I wanna mean a huuuuge thank you about the traefik cert dumper technique, it's like really great !
hi @Jean-Baptiste-Lasselle sorry for the delay in reply, I had other things to work on and this was going to take some time. So I verified what you said and debugged it down to the ssl root certificate. This is problematic because because I'm using traefik (lets encrypt) and this handles the certs in its own format not immediately accessible. I was thinking about having a separate container run lets encrypt and dump them and share that for services that need that, however I found https://github.com/ldez/traefik-certs-dumper which I think will solve the problem. Using that to expose the certs from traefik in a raw format the registry/portus should be able to consume them and solve this problem. I'll try and resolve and post a compose file.
So, just to be sure I understand everything in your config :
Traefik
is configured with an ACME provider, https://docs.traefik.io/https/acme/ , am I right ? traefik
itself, acting as would a certbot
, traefik_certs_raw
volume.docker-compose.yml
, you have a ./traefik_certs/
in the folder where your docker-compose.yml
file is ?/certs/acme.json
, and generate actual *.key
and *.crt/*.cert
files into the /dump
folder inside the container. Am I right there ? @Ashtonian I also really like you secret dedicated docker volume definition. I'll use that too from now on, in addition with HashiCorp Vault as Secret Manager
Hi @Ashtonian, I dumped this about PORTUS_SECRET_KEY_BASE
,as a thank you for making me understand your traefik-cert-dumper
, sharing it with community.
Description
Steps to reproduce
I attempted to spin up portus and push an image. The registry is setup via traefik at https://registry.mysite.com and portus is available https://portus.mysite.com.
The registry logs show 401 errors about invalid scope, I'm assuming something with oauth but I can't figure out what, any help would be appreciated.
Deployment information
Deployment method: docker
Configuration: