search

SUSE / Portus

Authorization service and frontend for Docker registry (v2)
http://port.us.org/
Apache License 2.0
3k stars 470 forks source link

Portus example for compose insecure does not work (unable to configure authorization) #1980

Open Mario-Hofstaetter opened 5 years ago

Mario-Hofstaetter commented 5 years ago

Description

I tried running Portus using the docker-compose.insecure.yml because its an isolated network, the registry container exits immediately. After providing a certificate (see below), the container stays up, but connection between portus and registry fails. What MACHINE_FQDN do I have to provide? The FQDN of the docker host does not work (see below)

Steps to reproduce

  1. Clone Portus git repository, go to ./examples/compose
  2. execute docker-compose -f docker-compose.insecure.yml up -d
  3. Check container state docker-compose ps , compose_registry_1 has exited

- Notice: `open /secrets/portus.crt: no such file or directory`
- The compose file has this bind mount: `- ./secrets:/secrets:ro`
- The [./secrets/](https://github.com/SUSE/Portus/tree/master/examples/compose/secrets) directory is empty, except an `.gitignore` containing `portus.*`
- try creating an empty file: `touch ./secrets/portus.crt` and run compose again, error message does not change... ?
- run `docker rm compose_registry_1` , run docker-compose up, check logs again:

registry_1 | time="2018-09-27T17:29:29Z" level=info msg="debug server listening 0.0.0.0:5001" registry_1 | time="2018-09-27T17:29:29Z" level=warning msg="No HTTP secret provided - generated random secret. This may cause problems with uploads if multiple registries are behind a load-balancer. To provide a shared secret, fill in http.secret in the configuration file or set the REGISTRY_HTTP_SECRET environment variable." go.version=go1.7.6 instance.id=66bc9ffa-f529-4922-8e03-b0e5c38f3684 version=v2.6.2 registry_1 | time="2018-09-27T17:29:29Z" level=info msg="configuring endpoint portus (http://172.17.0.1:3000/v2/webhooks/events), timeout=2s, headers=map[]" go.version=go1.7.6 instance.id=66bc9ffa-f529-4922-8e03-b0e5c38f3684 version=v2.6.2 registry_1 | time="2018-09-27T17:29:29Z" level=info msg="redis not configured" go.version=go1.7.6 instance.id=66bc9ffa-f529-4922-8e03-b0e5c38f3684 version=v2.6.2 registry_1 | time="2018-09-27T17:29:29Z" level=info msg="Starting upload purge in 42m0s" go.version=go1.7.6 instance.id=66bc9ffa-f529-4922-8e03-b0e5c38f3684 version=v2.6.2 registry_1 | panic: unable to configure authorization (token): token auth requires at least one token signing root certificate registry_1 | registry_1 | goroutine 1 [running]: registry_1 | panic(0xb4ccc0, 0xc4203c4050) registry_1 | /usr/local/go/src/runtime/panic.go:500 +0x1a1 registry_1 | github.com/docker/distribution/registry/handlers.NewApp(0x1066800, 0xc4203ca2d0, 0xc4203b2000, 0x1066800) registry_1 | /go/src/github.com/docker/distribution/registry/handlers/app.go:302 +0x1b6a registry_1 | github.com/docker/distribution/registry.NewRegistry(0x7f65137c6068, 0xc4203ca2d0, 0xc4203b2000, 0xc, 0x0, 0x0) registry_1 | /go/src/github.com/docker/distribution/registry/registry.go:86 +0x213 registry_1 | github.com/docker/distribution/registry.glob..func1(0x108e180, 0xc420392ff0, 0x1, 0x1) registry_1 | /go/src/github.com/docker/distribution/registry/registry.go:55 +0x106 registry_1 | github.com/docker/distribution/vendor/github.com/spf13/cobra.(Command).execute(0x108e180, 0xc420392fa0, 0x1, 0x1, 0x108e180, 0xc420392fa0) registry_1 | /go/src/github.com/docker/distribution/vendor/github.com/spf13/cobra/command.go:495 +0x190 registry_1 | github.com/docker/distribution/vendor/github.com/spf13/cobra.(Command).Execute(0x108e320, 0xc4201e3f40, 0xc4200001a0) registry_1 | /go/src/github.com/docker/distribution/vendor/github.com/spf13/cobra/command.go:560 +0x3c3 registry_1 | main.main() registry_1 | /go/src/github.com/docker/distribution/cmd/registry/main.go:24 +0x2d


- try to generate some certificate, [like this](https://serverfault.com/questions/224122/what-is-crt-and-key-files-and-how-to-generate-them), putting it to `./secrets/portus.crt`
- **the registry container now stays up, why is this necessary for the "insecure" environment?**

Actually, for the following part, I could open another issue if you'd like:

- Create admin user in Portus -> OK
- Configuring the registry in Portus fails, no matter what ip or hostname I use (with Port :5000)
**`Errno::ECONNREFUSED: connection refused`**
(tried FQDN, short hostname, ipv4, 0.0.0.0
- `http://<docker-host>:5000/` is accessible in browser, so registry is running

**Do I have to set `MACHINE_FQDN` in `.env` ? This is not clear**
-> Tried using the docker host fqdn, short hostname, ip-adresses, does not make a difference

### Deployment information

**Deployment method**: Using the current [docker-compose.insecure.yml](https://github.com/SUSE/Portus/blob/master/examples/compose/docker-compose.insecure.yml)

**Configuration**:

```yml
version: "2"

services:
  portus:
    image: opensuse/portus:head
    environment:
      - PORTUS_MACHINE_FQDN_VALUE=${MACHINE_FQDN}

      # DB. The password for the database should definitely not be here. You are
      # probably better off with Docker Swarm secrets.
      - PORTUS_DB_HOST=db
      - PORTUS_DB_DATABASE=portus_production
      - PORTUS_DB_PASSWORD=${DATABASE_PASSWORD}
      - PORTUS_DB_POOL=5

      # Secrets. It can possibly be handled better with Swarm's secrets.
      - PORTUS_SECRET_KEY_BASE=${SECRET_KEY_BASE}
      - PORTUS_KEY_PATH=/certificates/portus.key
      - PORTUS_PASSWORD=${PORTUS_PASSWORD}

      # SSL
      - PORTUS_CHECK_SSL_USAGE_ENABLED='false'

      # Since we have no nginx in insecure mode, portus have to
      # serve the static files
      - RAILS_SERVE_STATIC_FILES='true'
    ports:
      - 3000:3000
    depends_on:
      - db
    links:
      - db
    volumes:
      - ./secrets:/certificates:ro

  background:
    image: opensuse/portus:head
    depends_on:
      - portus
      - db
    environment:
      # Theoretically not needed, but cconfig's been buggy on this...
      - CCONFIG_PREFIX=PORTUS
      - PORTUS_MACHINE_FQDN_VALUE=${MACHINE_FQDN}

      # DB. The password for the database should definitely not be here. You are
      # probably better off with Docker Swarm secrets.
      - PORTUS_DB_HOST=db
      - PORTUS_DB_DATABASE=portus_production
      - PORTUS_DB_PASSWORD=${DATABASE_PASSWORD}
      - PORTUS_DB_POOL=5

      # Secrets. It can possibly be handled better with Swarm's secrets.
      - PORTUS_SECRET_KEY_BASE=${SECRET_KEY_BASE}
      - PORTUS_KEY_PATH=/certificates/portus.key
      - PORTUS_PASSWORD=${PORTUS_PASSWORD}

      - PORTUS_BACKGROUND=true
    links:
      - db
    volumes:
      - ./secrets:/certificates:ro

  db:
    image: library/mariadb:10.0.23
    command: mysqld --character-set-server=utf8 --collation-server=utf8_unicode_ci --init-connect='SET NAMES UTF8;' --innodb-flush-log-at-trx-commit=0
    environment:
      - MYSQL_DATABASE=portus_production

      # Again, the password shouldn't be handled like this.
      - MYSQL_ROOT_PASSWORD=${DATABASE_PASSWORD}
    volumes:
      - /var/lib/portus/mariadb:/var/lib/mysql

  registry:
    image: library/registry:2.6
    environment:
      # Authentication
      REGISTRY_AUTH_TOKEN_REALM: http://${MACHINE_FQDN}:3000/v2/token
      REGISTRY_AUTH_TOKEN_SERVICE: ${MACHINE_FQDN}:5000
      REGISTRY_AUTH_TOKEN_ISSUER: ${MACHINE_FQDN}
      REGISTRY_AUTH_TOKEN_ROOTCERTBUNDLE: /secrets/portus.crt

      # Portus endpoint
      REGISTRY_NOTIFICATIONS_ENDPOINTS: >
        - name: portus
          url: http://${MACHINE_FQDN}:3000/v2/webhooks/events
          timeout: 2000ms
          threshold: 5
          backoff: 1s
    volumes:
      - /var/lib/portus/registry:/var/lib/registry
      - ./secrets:/secrets:ro
      - ./registry/config.yml:/etc/docker/registry/config.yml:ro
    ports:
      - 5000:5000
      - 5001:5001 # required to access debug service
    links:
- portus:portus

Portus version: latest docker image

quickstar commented 5 years ago

Same issue here! I think it could be related to the following error msg:

panic: unable to configure authorization (token): unable to open token auth root certificate bundle file "/secrets/portus.crt": open /secrets/portus.crt: no such file or directory

Unfortunately I don't know why it cannot find the /secrets folder! As far as i can tell portus creates that volume and the registry container tries to attach it as well!

Any ideas?

Mario-Hofstaetter commented 5 years ago

Same issue here! I think it could be related to the following error msg:

panic: unable to configure authorization (token): unable to open token auth root certificate bundle file "/secrets/portus.crt": open /secrets/portus.crt: no such file or directory

Unfortunately I don't know why it cannot find the /secrets folder! As far as i can tell portus creates that volume and the registry container tries to attach it as well!

Any ideas?

Read my post again, I was able to overcome this by doing the following:

But then, i was not able to connect portus to the registry container (see my first post)

quickstar commented 5 years ago

ok I got it to work by creating a valid self signed certificate in the ./secrets directory.

openssl genrsa 1024 > portus.key
chmod 400 portus.key
openssl req -new -x509 -nodes -sha1 -days 365 -key portus.key -out portus.crt
Mario-Hofstaetter commented 5 years ago

ok I got it to work by creating a valid self signed certificate in the ./secrets directory.

openssl genrsa 1024 > portus.key
chmod 400 portus.key
openssl req -new -x509 -nodes -sha1 -days 365 -key portus.key -out portus.crt

You were able to setup portus completely? So the connection to the registry was ok in the portus web interface? Would you be so kind and sind me your docker-compose.yml file? Maybe I overlooked something.

quickstar commented 5 years ago

You were able to setup portus completely? So the connection to the registry was ok in the portus web interface?

Yes, that's now working fine

Would you be so kind and sind me your docker-compose.yml file? Maybe I overlooked something.

I didn't change anything. The only thing i had to make sure, was that the certificate was generated.

mssola commented 5 years ago

the registry container now stays up, why is this necessary for the "insecure" environment?

This is needed because the registry actually requires it. You can check the docs from the Docker registry, but a tl;dr would be that the registry needs to validate that the JWT token being passed actually belongs to Portus. I guess this could be better documented, since you are not the first person to raise this documentation issue...

Do I have to set MACHINE_FQDN in .env ? This is not clear

We could be more clear on the documentation, for sure. You need to set the MACHINE_FQDN environment variable in whatever way you see fit: either on the .env file, or in the docker-compose.insecure.yml file. The FQDN to be set on this setup is the one created by Docker itself (e.g. in my setup, on Linux, this is the IP from the docker0 bridge created by the Docker daemon). We used to do some magic to figure out this IP on previous examples, but it was hard to maintain.

Axel13fr commented 5 years ago

Hi Gents, I followed all the workarounds mentionned in this post but it still doesn't work. Portus doesn't manage to connect to the db. It seems that the db creates a server socket on something empty as it appears in the log. Any ideas ?

Attaching to compose_db_1, compose_portus_1, compose_registry_1, compose_background_1
db_1          | 181106 15:27:17 [Note] mysqld (mysqld 10.0.23-MariaDB-1~jessie) starting as process 1 ...
db_1          | 181106 15:27:17 [Note] InnoDB: Using mutexes to ref count buffer pool pages
db_1          | 181106 15:27:17 [Note] InnoDB: The InnoDB memory heap is disabled
db_1          | 181106 15:27:17 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
db_1          | 181106 15:27:17 [Note] InnoDB: Memory barrier is not used
db_1          | 181106 15:27:17 [Note] InnoDB: Compressed tables use zlib 1.2.8
db_1          | 181106 15:27:17 [Note] InnoDB: Using Linux native AIO
db_1          | 181106 15:27:17 [Note] InnoDB: Using CPU crc32 instructions
db_1          | 181106 15:27:17 [Note] InnoDB: Initializing buffer pool, size = 256.0M
db_1          | 181106 15:27:17 [Note] InnoDB: Completed initialization of buffer pool
db_1          | 181106 15:27:17 [Note] InnoDB: Highest supported file format is Barracuda.
db_1          | 181106 15:27:17 [Note] InnoDB: 128 rollback segment(s) are active.
registry_1    | time="2018-11-06T15:27:18Z" level=info msg="debug server listening 0.0.0.0:5001" 
db_1          | 181106 15:27:17 [Note] InnoDB: Waiting for purge to start
registry_1    | time="2018-11-06T15:27:18Z" level=warning msg="No HTTP secret provided - generated random secret. This may cause problems with uploads if multiple registries are behind a load-balancer. To provide a shared secret, fill in http.secret in the configuration file or set the REGISTRY_HTTP_SECRET environment variable." go.version=go1.7.6 instance.id=5e7809d3-8814-4342-b030-cf9d1e67bfdb version=v2.6.2 
registry_1    | time="2018-11-06T15:27:18Z" level=info msg="configuring endpoint portus (http://172.17.0.1:3000/v2/webhooks/events), timeout=2s, headers=map[]" go.version=go1.7.6 instance.id=5e7809d3-8814-4342-b030-cf9d1e67bfdb version=v2.6.2 
db_1          | 181106 15:27:17 [Note] InnoDB:  Percona XtraDB (http://www.percona.com) 5.6.26-76.0 started; log sequence number 2234355
registry_1    | time="2018-11-06T15:27:18Z" level=info msg="Starting upload purge in 14m0s" go.version=go1.7.6 instance.id=5e7809d3-8814-4342-b030-cf9d1e67bfdb version=v2.6.2 
registry_1    | time="2018-11-06T15:27:18Z" level=info msg="redis not configured" go.version=go1.7.6 instance.id=5e7809d3-8814-4342-b030-cf9d1e67bfdb version=v2.6.2 
registry_1    | time="2018-11-06T15:27:18Z" level=info msg="listening on [::]:5000" go.version=go1.7.6 instance.id=5e7809d3-8814-4342-b030-cf9d1e67bfdb version=v2.6.2 
db_1          | 181106 15:27:17 [Note] Plugin 'FEEDBACK' is disabled.
db_1          | 181106 15:27:17 [Note] Server socket created on IP: '::'.
db_1          | 181106 15:27:17 [Warning] 'proxies_priv' entry '@% root@b26fdd6fc7dd' ignored in --skip-name-resolve mode.
db_1          | 181106 15:27:17 [Note] mysqld: ready for connections.
db_1          | Version: '10.0.23-MariaDB-1~jessie'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  mariadb.org binary distribution
background_1  | [Mailer config] Host:     172.17.0.1
background_1  | [Mailer config] Protocol: https://
portus_1      | Waiting for mariadb to be ready in 5 seconds
background_1  | [Database] Not ready yet. Waiting...
background_1  | [Database] Not ready yet. Waiting...
portus_1      | Waiting for mariadb to be ready in 5 seconds
background_1  | [Database] Not ready yet. Waiting...
portus_1      | Waiting for mariadb to be ready in 5 seconds
background_1  | [Database] Not ready yet. Waiting...
background_1  | [Database] Not ready yet. Waiting...
portus_1      | Waiting for mariadb to be ready in 5 seconds
background_1  | [Database] Not ready yet. Waiting...
portus_1      | Waiting for mariadb to be ready in 5 seconds
background_1  | [Database] Not ready yet. Waiting...
background_1  | [Database] Not ready yet. Waiting...
portus_1      | Waiting for mariadb to be ready in 5 seconds
background_1  | [Database] Not ready yet. Waiting...
portus_1      | Waiting for mariadb to be ready in 5 seconds
background_1  | [Database] Not ready yet. Waiting...
background_1  | [Database] Not ready yet. Waiting...
portus_1      | Waiting for mariadb to be ready in 5 seconds
background_1  | [Database] Not ready yet. Waiting...
portus_1      | Waiting for mariadb to be ready in 5 seconds
background_1  | [Database] Not ready yet. Waiting...
portus_1      | Waiting for mariadb to be ready in 5 seconds
background_1  | [Database] Not ready yet. Waiting...
background_1  | [Database] Not ready yet. Waiting...
portus_1      | Waiting for mariadb to be ready in 5 seconds
background_1  | [Database] Not ready yet. Waiting...
portus_1      | Waiting for mariadb to be ready in 5 seconds
background_1  | [Database] Not ready yet. Waiting...
background_1  | [Database] Not ready yet. Waiting...
portus_1      | Waiting for mariadb to be ready in 5 seconds
background_1  | [Database] Timeout reached, exiting with error. Check the logs...
background_1  | /srv/Portus/lib/portus/db.rb:40:in `wait_until': Timeout reached for 'ready' status (Portus::DB::TimeoutReachedError)
background_1  |     from /srv/Portus/bin/background.rb:8:in `<top (required)>'
background_1  |     from /srv/Portus/vendor/bundle/ruby/2.5.0/gems/railties-4.2.10/lib/rails/commands/runner.rb:60:in `load'
background_1  |     from /srv/Portus/vendor/bundle/ruby/2.5.0/gems/railties-4.2.10/lib/rails/commands/runner.rb:60:in `<top (required)>'
background_1  |     from /srv/Portus/vendor/bundle/ruby/2.5.0/gems/railties-4.2.10/lib/rails/commands/commands_tasks.rb:123:in `require'
background_1  |     from /srv/Portus/vendor/bundle/ruby/2.5.0/gems/railties-4.2.10/lib/rails/commands/commands_tasks.rb:123:in `require_command!'
background_1  |     from /srv/Portus/vendor/bundle/ruby/2.5.0/gems/railties-4.2.10/lib/rails/commands/commands_tasks.rb:90:in `runner'
background_1  |     from /srv/Portus/vendor/bundle/ruby/2.5.0/gems/railties-4.2.10/lib/rails/commands/commands_tasks.rb:39:in `run_command!'
background_1  |     from /srv/Portus/vendor/bundle/ruby/2.5.0/gems/railties-4.2.10/lib/rails/commands.rb:17:in `<top (required)>'
background_1  |     from bin/rails:12:in `require'
background_1  |     from bin/rails:12:in `<main>'
background_1  | exit status 1
compose_background_1 exited with code 1
portus_1      | Waiting for mariadb to be ready in 5 seconds
Axel13fr commented 5 years ago

Solution was to clean database persistent files as I had a run with a different password: sudo rm -rf /var/lib/portus

kovacsbalu commented 5 years ago

Add skip-grant-tables under [mysqld] in database my.cnf