search

Secure-Compliance-Solutions-LLC / GVM-Docker

Greenbone Vulnerability Management Docker Image with OpenVAS
https://securecompliance.gitbook.io/projects/
MIT License
248 stars 91 forks source link

The SCAP database is required #136

Closed coderkk closed 3 years ago

coderkk commented 3 years ago

Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

  1. I have install the Secure-Compliance-Solutions-LLC / GVM-Docker
  2. I login to GVM
  3. In SecInfo\cves, it show 
    The SCAP database is required
  4. See error 
    
==> /usr/local/var/log/gvm/gsad.log <==
gsad  gmp:MESSAGE:2021-03-01 09h00.48 utc:818: Authentication success for 'admin' from 103.29.198.25
2021-03-01 17:01:23.478 +08 [806] LOG:  could not send data to client: Broken pipe
2021-03-01 17:01:23.479 +08 [806] FATAL:  connection to client lost

==> /usr/local/var/log/gvm/gvmd.log <== md manage:WARNING:2021-03-01 09h01.29 utc:1099: update_scap: No SCAP db present, rebuilding SCAP db from scratch md manage: INFO:2021-03-01 09h01.29 utc:1099: update_scap: Updating data from feed md manage: INFO:2021-03-01 09h01.29 utc:1099: Updating CPEs 



**Expected behavior**
Show all the SecInfo information
markdesilva commented 3 years ago

I'm not seeing similar behavior on my side?

coderkk commented 3 years ago

I deployed it in digitalocean.

I use this command to deploy docker

docker run --detach --publish 8080:9392 --publish 5432:5432 --publish 2222:22 --env DB_PASSWORD="postgres DB password" --env PASSWORD="webUI password" --volume gvm-data:/data --name gvm securecompliance/gvm
markdesilva commented 3 years ago

I run it on my server with the same command but with HTTPS.

docker run --detach --publish 9392:9392 --publish 5432:5432 --publish 2020:22 --env SSHD="true" --env DB_PASSWORD="<mydbpass>" --env PASSWORD="<mywebuipass>" --env HTTPS="true" --volume gvm-data:/data --name gvm securecompliance/gvm

No issues.

This error "could not send data to client: Broken pipe" sounds like a postgres error. Have you tried deleting the container, deleting the image and running the docker command again to pull the image clean and this time follow the logs as it runs (using command: docker logs -f gvm) and see if there is some error in setting up the postgres?

I deployed it in digitalocean.

I use this command to deploy docker

docker run --detach --publish 8080:9392 --publish 5432:5432 --publish 2222:22 --env DB_PASSWORD="postgres DB password" --env PASSWORD="webUI password" --volume gvm-data:/data --name gvm securecompliance/gvm
coderkk commented 3 years ago

I got this error in my log now

==> /usr/local/var/log/gvm/ospd-openvas.log <==
OSPD[1232] 2021-03-02 17:05:53,589: ERROR: (ospd.server) Error sending data to the client. [Errno 32] Broken pipe

==> /usr/local/var/log/gvm/gvmd.log <==
md manage:WARNING:2021-03-02 09h05.59 utc:1491: update_scap: No SCAP db present, rebuilding SCAP db from scratch
md manage:   INFO:2021-03-02 09h05.59 utc:1495: OSP service has different VT status (version 202103011132) from database (version (null), 0 VTs). Starting update ...
md manage:   INFO:2021-03-02 09h06.00 utc:1491: update_scap: Updating data from feed
md manage:   INFO:2021-03-02 09h06.00 utc:1491: Updating CPEs

Is it the progres or ssh issue?

I have remove the container and images. Then, I re-run the docker run again.

markdesilva commented 3 years ago

I think it’s Postgres. The dbs in /data in the container might be bad.

The /data in the container is mapped to /var/lib/docker/volumes/gvm-data in the host machine

You might need to move that directory to gvmdata.old and delete the containers and images again and then rerun the docker command.

coderkk commented 3 years ago

I set --volume /mnt/sdb1/gvm_data:/data this parameter to store the data this this drive. Is it correct?

markdesilva commented 3 years ago

Seems correct to me.

I set --volume /mnt/sdb1/gvm_data:/data this parameter to store the data this this drive. Is it correct?

coderkk commented 3 years ago

I got another error again


2021-03-02 21:13:45.305 +08 [8895] FATAL:  password authentication failed for user "postgres"
2021-03-02 21:13:45.305 +08 [8895] DETAIL:  User "postgres" has no password assigned.
        Connection matched pg_hba.conf line 98: "host    all             all              0.0.0.0/0                 md5"
markdesilva commented 3 years ago

You ran the docker command without the --env DB_PASSWORD="<mydbpass>"?

I got another error again


2021-03-02 21:13:45.305 +08 [8895] FATAL:  password authentication failed for user "postgres"
2021-03-02 21:13:45.305 +08 [8895] DETAIL:  User "postgres" has no password assigned.
        Connection matched pg_hba.conf line 98: "host    all             all              0.0.0.0/0                 md5"
coderkk commented 3 years ago

No. I run the docker with --env DB_PASSWORD=":xxxxx"

coderkk commented 3 years ago

I check my log, it seen no problem in installation.

9:C 02 Mar 2021 16:07:42.033 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
9:C 02 Mar 2021 16:07:42.033 # Redis version=5.0.7, bits=64, commit=00000000, modified=0, pid=9, just started
9:C 02 Mar 2021 16:07:42.033 # Configuration loaded
Wait for redis socket to be created...
Testing redis status...
Redis ready.
Creating Database folder...
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "C.UTF-8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /data/database ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Asia/Kuching
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok

Success. You can now start the database server using:

    /usr/lib/postgresql/12/bin/pg_ctl -D /data/database -l logfile start

Starting PostgreSQL...
waiting for server to start....2021-03-02 16:07:44.041 +08 [37] LOG:  starting PostgreSQL 12.6 (Ubuntu 12.6-1.pgdg20.04+1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0, 64-bit
2021-03-02 16:07:44.043 +08 [37] LOG:  listening on IPv4 address "127.0.0.1", port 5432
2021-03-02 16:07:44.044 +08 [37] LOG:  could not bind IPv6 address "::1": Cannot assign requested address
2021-03-02 16:07:44.044 +08 [37] HINT:  Is another postmaster already running on port 5432? If not, wait a few seconds and retry.
2021-03-02 16:07:44.049 +08 [37] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-03-02 16:07:44.097 +08 [38] LOG:  database system was shut down at 2021-03-02 16:07:43 +08
2021-03-02 16:07:44.106 +08 [37] LOG:  database system is ready to accept connections
 done
server started
Creating SSH folder...
Running first start configuration...
Creating Greenbone Vulnerability system user...
Creating Greenbone Vulnerability Manager database
CREATE ROLE
GRANT ROLE
CREATE EXTENSION
CREATE EXTENSION
2021-03-02 16:07:46.056 +08 [37] LOG:  received fast shutdown request
waiting for server to shut down....2021-03-02 16:07:46.068 +08 [37] LOG:  aborting any active transactions
2021-03-02 16:07:46.074 +08 [37] LOG:  background worker "logical replication launcher" (PID 44) exited with exit code 1
2021-03-02 16:07:46.075 +08 [39] LOG:  shutting down
2021-03-02 16:07:46.119 +08 [37] LOG:  database system is shut down
 done
server stopped
waiting for server to start....2021-03-02 16:07:46.179 +08 [272] LOG:  starting PostgreSQL 12.6 (Ubuntu 12.6-1.pgdg20.04+1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0, 64-bit
2021-03-02 16:07:46.180 +08 [272] LOG:  listening on IPv4 address "0.0.0.0", port 5432
2021-03-02 16:07:46.180 +08 [272] LOG:  listening on IPv6 address "::", port 5432
2021-03-02 16:07:46.186 +08 [272] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2021-03-02 16:07:46.210 +08 [273] LOG:  database system was shut down at 2021-03-02 16:07:46 +08
2021-03-02 16:07:46.220 +08 [272] LOG:  database system is ready to accept connections
 done
server started
ALTER ROLE
Creating gvmd folder...
Fixing gvmd folder...
Creating certs folder...
Generating certs...
Generated private key in /tmp/tmp.B5eB1CnnzT/cakey.pem.
Generated self signed certificate in /tmp/tmp.B5eB1CnnzT/cacert.pem.
Installed private key to /usr/local/var/lib/gvm/private/CA/cakey.pem.
Installed certificate to /usr/local/var/lib/gvm/CA/cacert.pem.
Generated private key in /tmp/tmp.B5eB1CnnzT/serverkey.pem.
Generated certificate request in /tmp/tmp.B5eB1CnnzT/serverrequest.pem.
Signed certificate request in /tmp/tmp.B5eB1CnnzT/serverrequest.pem with CA certificate in /usr/local/var/lib/gvm/CA/cacert.pem to generate certificate in /tmp/tmp.B5eB1CnnzT/servercert.pem
Installed private key to /usr/local/var/lib/gvm/private/CA/serverkey.pem.
Installed certificate to /usr/local/var/lib/gvm/CA/servercert.pem.
Generated private key in /tmp/tmp.B5eB1CnnzT/clientkey.pem.
Generated certificate request in /tmp/tmp.B5eB1CnnzT/clientrequest.pem.
Signed certificate request in /tmp/tmp.B5eB1CnnzT/clientrequest.pem with CA certificate in /usr/local/var/lib/gvm/CA/cacert.pem to generate certificate in /tmp/tmp.B5eB1CnnzT/clientcert.pem
Installed private key to /usr/local/var/lib/gvm/private/CA/clientkey.pem.
Installed certificate to /usr/local/var/lib/gvm/CA/clientcert.pem.
Removing temporary directory /tmp/tmp.B5eB1CnnzT.
coderkk commented 3 years ago

I saw this error

Starting Open Scanner Protocol daemon for OpenVAS...
Creating OSPd socket link from old location...
Starting Greenbone Vulnerability Manager...
Waiting for Greenbone Vulnerability Manager to finish startup...
2021-03-02 17:00:51.942 +08 [1252] ERROR:  relation "public.meta" does not exist at character 19
2021-03-02 17:00:51.942 +08 [1252] STATEMENT:  SELECT value FROM public.meta WHERE name = 'database_version';

Do it will occur the problem?

markdesilva commented 3 years ago

Hmm this is beyond me then, sorry. I still think there is an issue with Postgres. Maybe Pix has some answers.

austinsonger commented 3 years ago

@coderkk can you access gsa?

maibold1 commented 3 years ago

Hey @austinsonger i got the same issue and i can access gsa but the db issue keep running in a loop

Update

I build it the the image on the master repository and it fixed the issue running a scan right now no issues at all

pulpo commented 3 years ago

Same issue here, I've compiled my own image to verify if was a problem with the one at the repo, but I'm in the loop.... 

md manage:WARNING:2021-06-03 17h38.52 utc:1682: update_scap: No SCAP db present, rebuilding SCAP db from scratch

from the UI I see a banner at the bottom with the following: The SCAP database is required