Docker container does not start

gschafra commented 3 years ago

Container seems not to start correctly (trying restarts and quitting after a bunch of restarts):

openvas_1  | Redis ready.
openvas_1  | Fixing local/share ...
openvas_1  | cp: cannot create regular file '/data/local-share/gvm/gvmd/global_schema_formats/787a4a18-dabc-11df-9486-002264764cea/rnc.xsl': File exists
openvas_1  | 8:C 14 Jun 2021 12:59:14.809 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
openvas_1  | 8:C 14 Jun 2021 12:59:14.809 # Redis version=5.0.7, bits=64, commit=00000000, modified=0, pid=8, just started
openvas_1  | 8:C 14 Jun 2021 12:59:14.809 # Configuration loaded
openvas_1  | Wait for redis socket to be created...
openvas_1  | Testing redis status...
openvas_1  | Redis ready.
openvas_1  | Fixing local/share ...
openvas_1  | cp: cannot create regular file '/data/local-share/gvm/gvmd/global_schema_formats/787a4a18-dabc-11df-9486-002264764cea/rnc.xsl': File exists
openvas_1  | 10:C 14 Jun 2021 12:59:21.048 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
openvas_1  | 10:C 14 Jun 2021 12:59:21.048 # Redis version=5.0.7, bits=64, commit=00000000, modified=0, pid=10, just started
openvas_1  | 10:C 14 Jun 2021 12:59:21.048 # Configuration loaded
openvas_1  | Wait for redis socket to be created...
openvas_1  | Testing redis status...
openvas_1  | Redis ready.

/data/local-share/gvm/gvmd/global_schema_formats/787a4a18-dabc-11df-9486-002264764cea/rnc.xsl indeed exists in/on the volume

docker-compose.yml:

version: '3'

services:
  openvas:
    image: immauss/openvas:21.04.01
    restart: always
    volumes:
      #- /data/openvas/data:/data
      - ./data:/data
    ports:
      - "8088:9392"
    environment:
      - RELAYHOST=10.0.0.20
      - QUIET=true

immauss commented 3 years ago

It looks like you may have a permissions issue with your volume location.

"openvas_1 | cp: cannot create regular file '/data/local-share/gvm/gvmd/global_schema_formats/787a4a18-dabc-11df-9486-002264764cea/rnc.xsl': File exists"

It should not have any issues overwriting that file.

check the permissions on your "./data" directory.

For a temp check, you try starting it without the volume, of your course you'll lose anything you do if you delete teh container in this mode.

What distribution are you using? Is SELinux enabled? This can cause issues sometimes as well when using local directories vs creating a docker volume. (personally, I always go with a docker volume) Check the SELinux context of the default docker storage directory (Usually /var/lib/docker) and match the permissions and SELinux context of your "./data" directory.

gschafra commented 3 years ago

Hmmm... I'm running on Windows 10 (Docker Desktop v3.4.0) 😜 with full access rights on the given directory . Explicitely defining the volume seems to work... at least on Windows 😉:

version: '3'

services:
  openvas:
    image: immauss/openvas:21.04.01
    restart: always
    volumes:
      #- /data/openvas/data:/data
      - openvas:/data
    ports:
      - "8088:9392"
    environment:
      - RELAYHOST=10.0.0.20
      - QUIET=true
volumes:
    openvas:

Thanks for the fast reply!

gschafra commented 3 years ago

Now getting follwing problem on Ubuntu Bionic (v18) with last config (see above):

openvas_1  | 8:C 15 Jun 2021 10:59:32.385 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
openvas_1  | 8:C 15 Jun 2021 10:59:32.385 # Redis version=5.0.7, bits=64, commit=00000000, modified=0, pid=8, just started
openvas_1  | 8:C 15 Jun 2021 10:59:32.385 # Configuration loaded
openvas_1  | Wait for redis socket to be created...
openvas_1  | Testing redis status...
openvas_1  | Redis ready.
openvas_1  | Creating Data and database folder...
openvas_1  | mv: cannot create hard link '/data/database/global/4060' to '/data/database/base': Operation not permitted
openvas_1  | mv: will not create hard link '/data/database/pg_logical/mappings' to directory '/data/database/pg_dynshmem'
openvas_1  | mv: will not create hard link '/data/database/pg_logical/snapshots' to directory '/data/database/pg_commit_ts'
openvas_1  | mv: cannot create hard link '/data/database/pg_notify/0000' to '/data/database/pg_multixact': Operation not permitted
openvas_1  | mv: cannot create hard link '/data/database/pg_subtrans/0000' to '/data/database/pg_stat_tmp': Operation not permitted
openvas_1  | mv: cannot create hard link '/data/database/pg_wal/000000010000000000000001' to '/data/database/pg_snapshots': Operation not permitted
openvas_1  | mv: will not create hard link '/data/database/pg_wal/archive_status' to directory '/data/database/pg_twophase'
openvas_1  | mv: cannot create hard link '/data/database/pg_xact/0000' to '/data/database/pg_serial': Operation not permitted
openvas_1  | 8:C 15 Jun 2021 10:59:37.223 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
openvas_1  | 8:C 15 Jun 2021 10:59:37.223 # Redis version=5.0.7, bits=64, commit=00000000, modified=0, pid=8, just started
openvas_1  | 8:C 15 Jun 2021 10:59:37.223 # Configuration loaded
openvas_1  | Wait for redis socket to be created...
openvas_1  | Testing redis status...
openvas_1  | Redis ready.
openvas_1  | Fixing Database folder...
openvas_1  | Fixing local/var/lib ...
openvas_1  | Fixing local/share ...
openvas_1  | Creating postgresql.conf and pg_hba.conf
openvas_1  | Starting PostgreSQL...
openvas_1  | waiting for server to start....2021-06-15 10:59:48.135 GMT [32] FATAL:  data directory "/data/database" has invalid permissions
openvas_1  | 2021-06-15 10:59:48.135 GMT [32] DETAIL:  Permissions should be u=rwx (0700) or u=rwx,g=rx (0750).
openvas_1  |  stopped waiting
openvas_1  | pg_ctl: could not start server
openvas_1  | Examine the log output.

Spec:

Distributor ID: Ubuntu
Description:    Ubuntu 18.04.5 LTS
Release:        18.04
Codename:       bionic

Client: Docker Engine - Community
 Version:           20.10.7
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        f0df350
 Built:             Wed Jun  2 11:56:40 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          19.03.14
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       5eb3275d40
  Built:            Tue Dec  1 19:18:45 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.6
  GitCommit:        d71fcd7d8303cbf684402823e425e9dd2e99285d
 runc:
  Version:          1.0.0-rc95
  GitCommit:        b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

😰

immauss commented 3 years ago

This still looks like permissions again. Lots of "Operation not permitted" errors.

immauss commented 3 years ago

Any luck?

gschafra commented 3 years ago

I'll try using user: root in the docker-compose.yml during the next week.

immauss commented 3 years ago

oh .. .and make sure you are using the "latest" tag as it has the must up to date arm images.

gschafra commented 3 years ago

Unfortunately still not working, even with user: root and named volumes in docker-compose.yml:

version: '3'

services:
  openvas:
    image: immauss/openvas:21.04.02
    restart: always
    user: root
    volumes:
      #- /data/openvas/data:/data
      - openvas:/data
    ports:
      - "8088:9392"
    environment:
      - RELAYHOST=10.0.0.20
      - QUIET=true
      - PASSWORD=testpwd
volumes:
    openvas:

Am I doing something wrong, maybe wrong volume configuration? Do I have to create the volume explicitly with specific options?

Log output:

openvas_1  | 8:C 29 Jun 2021 12:41:34.567 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
openvas_1  | 8:C 29 Jun 2021 12:41:34.567 # Redis version=5.0.7, bits=64, commit=00000000, modified=0, pid=8, just started
openvas_1  | 8:C 29 Jun 2021 12:41:34.567 # Configuration loaded
openvas_1  | Wait for redis socket to be created...
openvas_1  | Testing redis status...
openvas_1  | Redis ready.
openvas_1  | Creating Data and database folder...
openvas_1  | mv: cannot create hard link '/data/database/global/4060' to '/data/database/base': Operation not permitted
openvas_1  | mv: will not create hard link '/data/database/pg_logical/mappings' to directory '/data/database/pg_dynshmem'
openvas_1  | mv: will not create hard link '/data/database/pg_logical/snapshots' to directory '/data/database/pg_commit_ts'
openvas_1  | mv: cannot create hard link '/data/database/pg_notify/0000' to '/data/database/pg_multixact': Operation not permitted
openvas_1  | mv: cannot create hard link '/data/database/pg_subtrans/0000' to '/data/database/pg_stat_tmp': Operation not permitted
openvas_1  | mv: cannot create hard link '/data/database/pg_wal/000000010000000000000001' to '/data/database/pg_snapshots': Operation not permitted
openvas_1  | mv: will not create hard link '/data/database/pg_wal/archive_status' to directory '/data/database/pg_twophase'
openvas_1  | mv: cannot create hard link '/data/database/pg_xact/0000' to '/data/database/pg_serial': Operation not permitted
openvas_1  | 8:C 29 Jun 2021 12:41:38.673 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
openvas_1  | 8:C 29 Jun 2021 12:41:38.673 # Redis version=5.0.7, bits=64, commit=00000000, modified=0, pid=8, just started
openvas_1  | 8:C 29 Jun 2021 12:41:38.673 # Configuration loaded
openvas_1  | Wait for redis socket to be created...
openvas_1  | Testing redis status...
openvas_1  | Redis ready.
openvas_1  | Fixing Database folder...
openvas_1  | Fixing local/var/lib ...
openvas_1  | Fixing local/share ...
openvas_1  | Fixing log directory for persistent logs ....
openvas_1  | Creating postgresql.conf and pg_hba.conf
openvas_1  | Starting PostgreSQL...
openvas_1  | waiting for server to start....2021-06-29 12:41:51.320 GMT [34] FATAL:  data directory "/data/database" has invalid permissions
openvas_1  | 2021-06-29 12:41:51.320 GMT [34] DETAIL:  Permissions should be u=rwx (0700) or u=rwx,g=rx (0750).
openvas_1  | pg_ctl: could not start server
openvas_1  | Examine the log output.
openvas_1  |  stopped waiting
openvas_1  | 8:C 29 Jun 2021 12:41:53.667 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
openvas_1  | 8:C 29 Jun 2021 12:41:53.667 # Redis version=5.0.7, bits=64, commit=00000000, modified=0, pid=8, just started
openvas_1  | 8:C 29 Jun 2021 12:41:53.667 # Configuration loaded
openvas_1  | Wait for redis socket to be created...
openvas_1  | Testing redis status...
openvas_1  | Redis ready.
openvas_1  | Creating postgresql.conf and pg_hba.conf
openvas_1  | Starting PostgreSQL...
openvas_1  | waiting for server to start....2021-06-29 12:41:54.793 GMT [20] FATAL:  data directory "/data/database" has invalid permissions
openvas_1  | 2021-06-29 12:41:54.793 GMT [20] DETAIL:  Permissions should be u=rwx (0700) or u=rwx,g=rx (0750).
openvas_1  |  stopped waiting
openvas_1  | pg_ctl: could not start server
openvas_1  | Examine the log output.
openvas_openvas_1 exited with code 1

immauss commented 3 years ago

Can you try again and change the image line to: image: immauss/openvas:latest

gschafra commented 3 years ago

Seems to be the same problem with the latest image, sorry:

openvas_1  | 8:C 29 Jun 2021 18:18:14.555 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
openvas_1  | 8:C 29 Jun 2021 18:18:14.555 # Redis version=5.0.7, bits=64, commit=00000000, modified=0, pid=8, just sta                                                                                                                  rted
openvas_1  | 8:C 29 Jun 2021 18:18:14.555 # Configuration loaded
openvas_1  | Wait for redis socket to be created...
openvas_1  | Testing redis status...
openvas_1  | Redis ready.
openvas_1  | Creating Data and database folder...
openvas_1  | mv: cannot create hard link '/data/database/global/4060' to '/data/database/base': Operation not permitte                                                                                                                  d
openvas_1  | mv: will not create hard link '/data/database/pg_logical/mappings' to directory '/data/database/pg_dynshm                                                                                                                  em'
openvas_1  | mv: will not create hard link '/data/database/pg_logical/snapshots' to directory '/data/database/pg_commi                                                                                                                  t_ts'
openvas_1  | mv: cannot create hard link '/data/database/pg_notify/0000' to '/data/database/pg_multixact': Operation n                                                                                                                  ot permitted
openvas_1  | mv: cannot create hard link '/data/database/pg_subtrans/0000' to '/data/database/pg_stat_tmp': Operation                                                                                                                   not permitted
openvas_1  | mv: cannot create hard link '/data/database/pg_wal/000000010000000000000001' to '/data/database/pg_snapsh                                                                                                                  ots': Operation not permitted
openvas_1  | mv: will not create hard link '/data/database/pg_wal/archive_status' to directory '/data/database/pg_twop                                                                                                                  hase'
openvas_1  | mv: cannot create hard link '/data/database/pg_xact/0000' to '/data/database/pg_serial': Operation not pe                                                                                                                  rmitted
openvas_1  | 8:C 29 Jun 2021 18:18:18.256 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
openvas_1  | 8:C 29 Jun 2021 18:18:18.256 # Redis version=5.0.7, bits=64, commit=00000000, modified=0, pid=8, just sta                                                                                                                  rted
openvas_1  | 8:C 29 Jun 2021 18:18:18.256 # Configuration loaded
openvas_1  | Wait for redis socket to be created...
openvas_1  | Testing redis status...
openvas_1  | Redis ready.
openvas_1  | Fixing Database folder...
openvas_1  | Fixing local/var/lib ...
openvas_1  | Fixing local/share ...
openvas_1  | Fixing log directory for persistent logs ....
openvas_1  | Creating postgresql.conf and pg_hba.conf
openvas_1  | Starting PostgreSQL...
openvas_1  | waiting for server to start....2021-06-29 18:18:30.636 GMT [34] FATAL:  data directory "/data/database" h                                                                                                                  as invalid permissions
openvas_1  | 2021-06-29 18:18:30.636 GMT [34] DETAIL:  Permissions should be u=rwx (0700) or u=rwx,g=rx (0750).
openvas_1  |  stopped waiting
openvas_1  | pg_ctl: could not start server
openvas_1  | Examine the log output.

Somewhat frustrating. OS specific issue?

gschafra commented 3 years ago

Maybe it's necessary to modify access rights from within the Dockerfile according to https://github.com/docker-library/postgres/blob/master/Dockerfile-debian.template#L183? Whereby this seems to be already treated in the script.sh...

immauss commented 3 years ago

The permissions internal to the container image are already being set. I suspect this has something to do with your setup, but I'm not able to identify it.

The image is working on multiple Linux distros, Docker Desktop on Mac, and on Raspberry Pi running Kali. (I've personally tested them all. ) I don't have a windows computer with Docker Desktop Handy. I could clone a Win10 VM and install it, but that will take me some time.

The common thread in every error you have shown me is permissions. If this were a problem with the image, it would be the same on every platform.

I really want to figure this out, but I've no idea where to tell you to look for your platform.

If you just run a simple start, what do you get?

docker run -d --name openvas immauss/openvas:latest

LoZio commented 3 years ago

I'll hijack this to say that both the latest (710275929ba4) and the previous one (9d489b116b42) that was working fine, don't start anymore. Processes are up and running but nothing binds to the network. If needed for debug I have an old and a new instances running.

immauss commented 3 years ago

@LoZio Hate to say, but it sounds like you may have a local problem. I have the latest running on 3 platforms including my prod and an RPi, and working fine.
The previous image was running for weeks on my Prod with no issues. docker restart?

LoZio commented 3 years ago

Two different hosts here, ubuntu 20.04 and Debian 10, each with several containers running. The Debian 10 was running fine until last week, yesterday I just restarted the stopped container and it does not start anymore. On the 20.04 host I just did what I do each time, since I have a script: it stop the container, deletes it, pulls new images, creates and starts a new container with the old data volume. It worked fine since now. The only curious thing is that we passed the month, is there something (a certificate or date somewhere) that was due on June 30? Each docker was updated and restarted, together with the host and other containers runs with no problems. Also this one runs (except it says it is not healthy after some time, but this is another issue and does not affect the openvas) but it just does not bind the local port 9392. I read all the logs and there is no error in it, just the port is not listening. And yes, I check it entering a bash on the container.

immauss commented 3 years ago

So .... there was a problem with the script I use to pull the latest versions. Apparently github "latest" is the most recent saved release, not the highest numbered release. So when Greenbone pushed a new release of the 20.08 branch, it showed up as the "latest" release and my 21.04 build tried to install the 20.08 branch of gsa. That is fixed now as of early yesterday. So the latest 21.04 has the most recent gsa. The worst part was my build was completing instead of failing when the gsa didn't build. It sounds like you pulled one of those images. Please give the current latest a shot. -Scott

LoZio commented 3 years ago

latest: Pulling from immauss/openvas
Digest: sha256:7ce13203928684fe62c31fb2bb4638d92b6bc8f809e9c3d121297f7891410774
Status: Image is up to date for immauss/openvas:latest
docker.io/immauss/openvas:latest

I had the latest, yesterday I deleted/re-created a lot of containers so at some point I got the right one.

LoZio commented 3 years ago

Mumble.. re-run the script (that gives up to date for the image) and the new container binds and runs with the latest data volume. This is the Deb 10 install. Will try to connect the VPN and test the Ubuntu 20 install...

immauss commented 3 years ago

Sorry about that... That change caught me off guard. I test the build one day, then the next it tanked. The one that tanked was the one that got pushed to docker hub because I thought it was all good. :/

LoZio commented 3 years ago

No problem, s***t happens and I thank you for maintaining this. I always hope to help to make it better. Also I learned to never start an upgrade while writing docs and having the need for the reports :) As long as I test the other install I'll report the results.

immauss commented 3 years ago

@gschafra

Are you able to get the latest up and running? Please let me know.

-Scott

gschafra commented 3 years ago

Still evaluating/testing, sorry... I'm currently a little bit busy. Running the given config with Docker Desktop (v3.5.1) on Windows works like a charm. I am currently trying to install a test environment (Ubuntu v18 VM) to isolate the problem and not bother you further with system specific (configuration) problems. Just give me a week or so.

gschafra commented 3 years ago

O.k... on a fresh installed Ubuntu v18 it works without any problems. Seems to be a local configuration problem on the server. Sorry for the trouble caused!

immauss / openvas

Docker container does not start #47