Closed gschafra closed 3 years ago
It looks like you may have a permissions issue with your volume location.
"openvas_1 | cp: cannot create regular file '/data/local-share/gvm/gvmd/global_schema_formats/787a4a18-dabc-11df-9486-002264764cea/rnc.xsl': File exists"
It should not have any issues overwriting that file.
check the permissions on your "./data" directory.
For a temp check, you try starting it without the volume, of your course you'll lose anything you do if you delete teh container in this mode.
What distribution are you using? Is SELinux enabled? This can cause issues sometimes as well when using local directories vs creating a docker volume. (personally, I always go with a docker volume) Check the SELinux context of the default docker storage directory (Usually /var/lib/docker) and match the permissions and SELinux context of your "./data" directory.
Hmmm... I'm running on Windows 10 (Docker Desktop v3.4.0) 😜 with full access rights on the given directory . Explicitely defining the volume seems to work... at least on Windows 😉:
version: '3'
services:
openvas:
image: immauss/openvas:21.04.01
restart: always
volumes:
#- /data/openvas/data:/data
- openvas:/data
ports:
- "8088:9392"
environment:
- RELAYHOST=10.0.0.20
- QUIET=true
volumes:
openvas:
Thanks for the fast reply!
Now getting follwing problem on Ubuntu Bionic (v18) with last config (see above):
openvas_1 | 8:C 15 Jun 2021 10:59:32.385 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
openvas_1 | 8:C 15 Jun 2021 10:59:32.385 # Redis version=5.0.7, bits=64, commit=00000000, modified=0, pid=8, just started
openvas_1 | 8:C 15 Jun 2021 10:59:32.385 # Configuration loaded
openvas_1 | Wait for redis socket to be created...
openvas_1 | Testing redis status...
openvas_1 | Redis ready.
openvas_1 | Creating Data and database folder...
openvas_1 | mv: cannot create hard link '/data/database/global/4060' to '/data/database/base': Operation not permitted
openvas_1 | mv: will not create hard link '/data/database/pg_logical/mappings' to directory '/data/database/pg_dynshmem'
openvas_1 | mv: will not create hard link '/data/database/pg_logical/snapshots' to directory '/data/database/pg_commit_ts'
openvas_1 | mv: cannot create hard link '/data/database/pg_notify/0000' to '/data/database/pg_multixact': Operation not permitted
openvas_1 | mv: cannot create hard link '/data/database/pg_subtrans/0000' to '/data/database/pg_stat_tmp': Operation not permitted
openvas_1 | mv: cannot create hard link '/data/database/pg_wal/000000010000000000000001' to '/data/database/pg_snapshots': Operation not permitted
openvas_1 | mv: will not create hard link '/data/database/pg_wal/archive_status' to directory '/data/database/pg_twophase'
openvas_1 | mv: cannot create hard link '/data/database/pg_xact/0000' to '/data/database/pg_serial': Operation not permitted
openvas_1 | 8:C 15 Jun 2021 10:59:37.223 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
openvas_1 | 8:C 15 Jun 2021 10:59:37.223 # Redis version=5.0.7, bits=64, commit=00000000, modified=0, pid=8, just started
openvas_1 | 8:C 15 Jun 2021 10:59:37.223 # Configuration loaded
openvas_1 | Wait for redis socket to be created...
openvas_1 | Testing redis status...
openvas_1 | Redis ready.
openvas_1 | Fixing Database folder...
openvas_1 | Fixing local/var/lib ...
openvas_1 | Fixing local/share ...
openvas_1 | Creating postgresql.conf and pg_hba.conf
openvas_1 | Starting PostgreSQL...
openvas_1 | waiting for server to start....2021-06-15 10:59:48.135 GMT [32] FATAL: data directory "/data/database" has invalid permissions
openvas_1 | 2021-06-15 10:59:48.135 GMT [32] DETAIL: Permissions should be u=rwx (0700) or u=rwx,g=rx (0750).
openvas_1 | stopped waiting
openvas_1 | pg_ctl: could not start server
openvas_1 | Examine the log output.
Spec:
Distributor ID: Ubuntu
Description: Ubuntu 18.04.5 LTS
Release: 18.04
Codename: bionic
Client: Docker Engine - Community
Version: 20.10.7
API version: 1.40
Go version: go1.13.15
Git commit: f0df350
Built: Wed Jun 2 11:56:40 2021
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 19.03.14
API version: 1.40 (minimum version 1.12)
Go version: go1.13.15
Git commit: 5eb3275d40
Built: Tue Dec 1 19:18:45 2020
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.6
GitCommit: d71fcd7d8303cbf684402823e425e9dd2e99285d
runc:
Version: 1.0.0-rc95
GitCommit: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7
docker-init:
Version: 0.18.0
GitCommit: fec3683
😰
This still looks like permissions again. Lots of "Operation not permitted" errors.
Any luck?
I'll try using user: root
in the docker-compose.yml
during the next week.
oh .. .and make sure you are using the "latest" tag as it has the must up to date arm images.
Unfortunately still not working, even with user: root
and named volumes in docker-compose.yml
:
version: '3'
services:
openvas:
image: immauss/openvas:21.04.02
restart: always
user: root
volumes:
#- /data/openvas/data:/data
- openvas:/data
ports:
- "8088:9392"
environment:
- RELAYHOST=10.0.0.20
- QUIET=true
- PASSWORD=testpwd
volumes:
openvas:
Am I doing something wrong, maybe wrong volume configuration? Do I have to create the volume explicitly with specific options?
Log output:
openvas_1 | 8:C 29 Jun 2021 12:41:34.567 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
openvas_1 | 8:C 29 Jun 2021 12:41:34.567 # Redis version=5.0.7, bits=64, commit=00000000, modified=0, pid=8, just started
openvas_1 | 8:C 29 Jun 2021 12:41:34.567 # Configuration loaded
openvas_1 | Wait for redis socket to be created...
openvas_1 | Testing redis status...
openvas_1 | Redis ready.
openvas_1 | Creating Data and database folder...
openvas_1 | mv: cannot create hard link '/data/database/global/4060' to '/data/database/base': Operation not permitted
openvas_1 | mv: will not create hard link '/data/database/pg_logical/mappings' to directory '/data/database/pg_dynshmem'
openvas_1 | mv: will not create hard link '/data/database/pg_logical/snapshots' to directory '/data/database/pg_commit_ts'
openvas_1 | mv: cannot create hard link '/data/database/pg_notify/0000' to '/data/database/pg_multixact': Operation not permitted
openvas_1 | mv: cannot create hard link '/data/database/pg_subtrans/0000' to '/data/database/pg_stat_tmp': Operation not permitted
openvas_1 | mv: cannot create hard link '/data/database/pg_wal/000000010000000000000001' to '/data/database/pg_snapshots': Operation not permitted
openvas_1 | mv: will not create hard link '/data/database/pg_wal/archive_status' to directory '/data/database/pg_twophase'
openvas_1 | mv: cannot create hard link '/data/database/pg_xact/0000' to '/data/database/pg_serial': Operation not permitted
openvas_1 | 8:C 29 Jun 2021 12:41:38.673 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
openvas_1 | 8:C 29 Jun 2021 12:41:38.673 # Redis version=5.0.7, bits=64, commit=00000000, modified=0, pid=8, just started
openvas_1 | 8:C 29 Jun 2021 12:41:38.673 # Configuration loaded
openvas_1 | Wait for redis socket to be created...
openvas_1 | Testing redis status...
openvas_1 | Redis ready.
openvas_1 | Fixing Database folder...
openvas_1 | Fixing local/var/lib ...
openvas_1 | Fixing local/share ...
openvas_1 | Fixing log directory for persistent logs ....
openvas_1 | Creating postgresql.conf and pg_hba.conf
openvas_1 | Starting PostgreSQL...
openvas_1 | waiting for server to start....2021-06-29 12:41:51.320 GMT [34] FATAL: data directory "/data/database" has invalid permissions
openvas_1 | 2021-06-29 12:41:51.320 GMT [34] DETAIL: Permissions should be u=rwx (0700) or u=rwx,g=rx (0750).
openvas_1 | pg_ctl: could not start server
openvas_1 | Examine the log output.
openvas_1 | stopped waiting
openvas_1 | 8:C 29 Jun 2021 12:41:53.667 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
openvas_1 | 8:C 29 Jun 2021 12:41:53.667 # Redis version=5.0.7, bits=64, commit=00000000, modified=0, pid=8, just started
openvas_1 | 8:C 29 Jun 2021 12:41:53.667 # Configuration loaded
openvas_1 | Wait for redis socket to be created...
openvas_1 | Testing redis status...
openvas_1 | Redis ready.
openvas_1 | Creating postgresql.conf and pg_hba.conf
openvas_1 | Starting PostgreSQL...
openvas_1 | waiting for server to start....2021-06-29 12:41:54.793 GMT [20] FATAL: data directory "/data/database" has invalid permissions
openvas_1 | 2021-06-29 12:41:54.793 GMT [20] DETAIL: Permissions should be u=rwx (0700) or u=rwx,g=rx (0750).
openvas_1 | stopped waiting
openvas_1 | pg_ctl: could not start server
openvas_1 | Examine the log output.
openvas_openvas_1 exited with code 1
Can you try again and change the image line to: image: immauss/openvas:latest
Seems to be the same problem with the latest
image, sorry:
openvas_1 | 8:C 29 Jun 2021 18:18:14.555 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
openvas_1 | 8:C 29 Jun 2021 18:18:14.555 # Redis version=5.0.7, bits=64, commit=00000000, modified=0, pid=8, just sta rted
openvas_1 | 8:C 29 Jun 2021 18:18:14.555 # Configuration loaded
openvas_1 | Wait for redis socket to be created...
openvas_1 | Testing redis status...
openvas_1 | Redis ready.
openvas_1 | Creating Data and database folder...
openvas_1 | mv: cannot create hard link '/data/database/global/4060' to '/data/database/base': Operation not permitte d
openvas_1 | mv: will not create hard link '/data/database/pg_logical/mappings' to directory '/data/database/pg_dynshm em'
openvas_1 | mv: will not create hard link '/data/database/pg_logical/snapshots' to directory '/data/database/pg_commi t_ts'
openvas_1 | mv: cannot create hard link '/data/database/pg_notify/0000' to '/data/database/pg_multixact': Operation n ot permitted
openvas_1 | mv: cannot create hard link '/data/database/pg_subtrans/0000' to '/data/database/pg_stat_tmp': Operation not permitted
openvas_1 | mv: cannot create hard link '/data/database/pg_wal/000000010000000000000001' to '/data/database/pg_snapsh ots': Operation not permitted
openvas_1 | mv: will not create hard link '/data/database/pg_wal/archive_status' to directory '/data/database/pg_twop hase'
openvas_1 | mv: cannot create hard link '/data/database/pg_xact/0000' to '/data/database/pg_serial': Operation not pe rmitted
openvas_1 | 8:C 29 Jun 2021 18:18:18.256 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
openvas_1 | 8:C 29 Jun 2021 18:18:18.256 # Redis version=5.0.7, bits=64, commit=00000000, modified=0, pid=8, just sta rted
openvas_1 | 8:C 29 Jun 2021 18:18:18.256 # Configuration loaded
openvas_1 | Wait for redis socket to be created...
openvas_1 | Testing redis status...
openvas_1 | Redis ready.
openvas_1 | Fixing Database folder...
openvas_1 | Fixing local/var/lib ...
openvas_1 | Fixing local/share ...
openvas_1 | Fixing log directory for persistent logs ....
openvas_1 | Creating postgresql.conf and pg_hba.conf
openvas_1 | Starting PostgreSQL...
openvas_1 | waiting for server to start....2021-06-29 18:18:30.636 GMT [34] FATAL: data directory "/data/database" h as invalid permissions
openvas_1 | 2021-06-29 18:18:30.636 GMT [34] DETAIL: Permissions should be u=rwx (0700) or u=rwx,g=rx (0750).
openvas_1 | stopped waiting
openvas_1 | pg_ctl: could not start server
openvas_1 | Examine the log output.
Somewhat frustrating. OS specific issue?
Maybe it's necessary to modify access rights from within the Dockerfile according to https://github.com/docker-library/postgres/blob/master/Dockerfile-debian.template#L183? Whereby this seems to be already treated in the script.sh...
The permissions internal to the container image are already being set. I suspect this has something to do with your setup, but I'm not able to identify it.
The image is working on multiple Linux distros, Docker Desktop on Mac, and on Raspberry Pi running Kali. (I've personally tested them all. ) I don't have a windows computer with Docker Desktop Handy. I could clone a Win10 VM and install it, but that will take me some time.
The common thread in every error you have shown me is permissions. If this were a problem with the image, it would be the same on every platform.
I really want to figure this out, but I've no idea where to tell you to look for your platform.
If you just run a simple start, what do you get?
docker run -d --name openvas immauss/openvas:latest
I'll hijack this to say that both the latest (710275929ba4) and the previous one (9d489b116b42) that was working fine, don't start anymore. Processes are up and running but nothing binds to the network. If needed for debug I have an old and a new instances running.
@LoZio Hate to say, but it sounds like you may have a local problem. I have the latest running on 3 platforms including my prod and an RPi, and working fine.
The previous image was running for weeks on my Prod with no issues.
docker restart?
Two different hosts here, ubuntu 20.04 and Debian 10, each with several containers running. The Debian 10 was running fine until last week, yesterday I just restarted the stopped container and it does not start anymore. On the 20.04 host I just did what I do each time, since I have a script: it stop the container, deletes it, pulls new images, creates and starts a new container with the old data volume. It worked fine since now. The only curious thing is that we passed the month, is there something (a certificate or date somewhere) that was due on June 30? Each docker was updated and restarted, together with the host and other containers runs with no problems. Also this one runs (except it says it is not healthy after some time, but this is another issue and does not affect the openvas) but it just does not bind the local port 9392. I read all the logs and there is no error in it, just the port is not listening. And yes, I check it entering a bash on the container.
So .... there was a problem with the script I use to pull the latest versions. Apparently github "latest" is the most recent saved release, not the highest numbered release. So when Greenbone pushed a new release of the 20.08 branch, it showed up as the "latest" release and my 21.04 build tried to install the 20.08 branch of gsa. That is fixed now as of early yesterday. So the latest 21.04 has the most recent gsa. The worst part was my build was completing instead of failing when the gsa didn't build. It sounds like you pulled one of those images. Please give the current latest a shot. -Scott
latest: Pulling from immauss/openvas
Digest: sha256:7ce13203928684fe62c31fb2bb4638d92b6bc8f809e9c3d121297f7891410774
Status: Image is up to date for immauss/openvas:latest
docker.io/immauss/openvas:latest
I had the latest, yesterday I deleted/re-created a lot of containers so at some point I got the right one.
Mumble.. re-run the script (that gives up to date for the image) and the new container binds and runs with the latest data volume. This is the Deb 10 install. Will try to connect the VPN and test the Ubuntu 20 install...
Sorry about that... That change caught me off guard. I test the build one day, then the next it tanked. The one that tanked was the one that got pushed to docker hub because I thought it was all good. :/
No problem, s***t happens and I thank you for maintaining this. I always hope to help to make it better. Also I learned to never start an upgrade while writing docs and having the need for the reports :) As long as I test the other install I'll report the results.
@gschafra
Are you able to get the latest up and running? Please let me know.
-Scott
Still evaluating/testing, sorry... I'm currently a little bit busy. Running the given config with Docker Desktop (v3.5.1) on Windows works like a charm. I am currently trying to install a test environment (Ubuntu v18 VM) to isolate the problem and not bother you further with system specific (configuration) problems. Just give me a week or so.
O.k... on a fresh installed Ubuntu v18 it works without any problems. Seems to be a local configuration problem on the server. Sorry for the trouble caused!
Container seems not to start correctly (trying restarts and quitting after a bunch of restarts):
/data/local-share/gvm/gvmd/global_schema_formats/787a4a18-dabc-11df-9486-002264764cea/rnc.xsl
indeed exists in/on the volumedocker-compose.yml
: