search

martinboller / gse

Bash script installing the latest version of Greenbone Community Edition the artist formerly known as OpenVAS. Greenbone Community Edition is the world’s most used open source vulnerability management solution.
https://www.infosecworrier.dk
13 stars 2 forks source link

readme

Greenbone Vulnerability Manager 22.4.x Community Edition Installation script

Bash script automating the installation of Greenbone Vulnerability Manager Community Version on Debian 12 (Bookworm) or Debian 11 (Bullseye)

Installation will be located in

Design principles:

Note: The primary server also serves as the Certificate Authority for itself and all secondaries. But you can create a Certificate SIgning Request and send that to your corporate PKI.

During installation a GVM user called 'admin' is created. The generated password for user admin is stored in the file /var/lib/gvm/adminuser. It is recommended that this password is changed and/or the file deleted. Do NOT delete the user admin unless you also change the feedowner to another user. This is described in the section Feed Owner.

To create a secondary see instructions later - but running the script add-secondary-2-primary.sh does the work required on the primary as well as on the secondary, hence this is the preferred method1.

1 You can install the complete installation on a secondary, that is include GVMD, GSAD etc, but there's no good reason to. The secondary with notus, ospd-openvas, and openvas only is a small efficient thing with a smaller attack surface.

Note: Due to the ugly hack made removing the Vagrant NIC uses (it gave me a lot of issues) use vboxmanage to control the VM afterwards, example: vboxmanage startvm manticore --type=headless

Note 2023-11-04: Used to be able to run the secondaries on 1 GB RAM and 1 shared CPU, but now need 2-4G B RAM and (preferably) 2 CPU's or better.

Versions used: Name URL Version
gvm-libs https://github.com/greenbone/gvm-libs/ 22.14.0
ospd-openvas https://github.com/greenbone/ospd-openvas/ 22.7.1
openvas-scanner https://github.com/greenbone/openvas-scanner/ 23.11.1
GVM Daemon https://github.com/greenbone/gvmd/ 24.0.0
GSA Daemon https://github.com/greenbone/gsad/ 24.0.0
GSA https://github.com/greenbone/gsa/ 24.0.1
openvas-smb https://github.com/greenbone/openvas-smb 22.5.6
python-gvm https://github.com/greenbone/python-gvm/ 24.8.0
gvm-tools https://github.com/greenbone/gvm-tools/ 24.8.0
pg-gvm https://github.com/greenbone/pg-gvm/ 22.6.5
notus-scanner https://github.com/greenbone/notus-scanner/ 22.6.4
greenbone-feed-sync https://github.com/greenbone/greenbone-feed-sync/ 24.9.0
Node.js for Debian 11 https://deb.nodesource.com/ 20.x
Node.js for Debian 12 Debian Repo 18.x

References

API Reference for Greenbone GMP 22.5

Install on Debian 11 or 12 directly (not needed when using Vagrant)

  1. Modify the .env file to match your environment (especially if you need to send mail), however you may want to personalize it even further.

  2. To install a primary, run: export DEBIAN_FRONTEND=noninteractive; apt update ; apt -y full-upgrade ; apt -y install git; git clone https://github.com/martinboller/gse.git; cd gse/installfiles/; chmod 755 *.sh; ./install-gse.sh

  3. Install secondaries by running: export DEBIAN_FRONTEND=noninteractive; apt update ; apt -y full-upgrade ; apt -y install git; git clone https://github.com/martinboller/gse.git; cd gse/installfiles/; chmod 755 *.sh; ./install-gse-secondary.sh¹ - When the secondary is finished installing the script returns the hostname and password needed to connect the secondary to the primary, however if you cleared the screen or forgot it, it can be found in /var/lib/gvm/greenboneuser on the secondary scanner in question.

  4. To connect the secondaries to the primary, on the primary, run: ./add-secondary-to-primary.sh and enter the hostname or ip and password for the secondary.

  5. Now use the different scripts (automatically cloned from GitHub on the primary) to create Credentials, Schedules, Targets, and Tasks (or do it manually from the Web UI). Details on the GMP Scripts can be found on GitHub.

¹ Use install-gse-secondary.sh for all the secondaries you want to deploy.

Latest changes

2024-11-12 - OpenVAS Scanner update

2024-10-30 - updates + changed prerequisites

2024-10-23 - gvm-libs updated

2024-10-17 - gsa, gsad, gvmd, and gvm-libs updated

2024-10-16 - gsa web frontend updated

2024-09-11 - Latest updates

2024-08-30 - Latest updates

2024-08-21 - Latest updates

2024-08-16 - Latest updates

2024-08-08 - Latest updates

2024-08-03 - Latest updates

2024-07-23 - Latest updates

2024-07-18 - Latest openvas-scanner v23.6.0

2024-07-15 - Latest openvas-scanner v23.5.2

2024-06-29 - Latest gsa v23.1.1

2024-06-25 - Latest openvas-scanner, gsad

2024-06-14 - Latest gvm-libs

2024-06-13 - Latest openvas-scanner

2024-05-23 - Latest openvas-scanner

2024-05-02 - Latest gvmd

2024-05-01 - Latest openvas-scanner

2024-04-26 - Latest GCE versions

2024-04-13 - Additional settings in .env

2024-03-28 - Updated component

² Turned out to be a badly configured alert that broke scanning. The alert triggered and an e-mail was sent, however GVMD never requested ospd-openvas to scan.

2024-03-17 - Updated component

2024-03-13 - gvmd updated

2024-03-05 - gvm-libs updated

2024-03-01 - GVMD updated

2024-02-14 - pg-gvm updated

2024-02-14 - Install optimizations and Maximum Rows Returned set to unlimited

2024-02-13 - Minor update GVMD

2024-02-11 - PAHO-MQTT 2.0.0 breaking changes

Traceback (most recent call last): File "/opt/gvm/gvmpy/lib/python3.11/site-packages/paho/mqtt/client.py", line 874, in del self._reset_sockets() File "/opt/gvm/gvmpy/lib/python3.11/site-packages/paho/mqtt/client.py", line 1133, in _reset_sockets self._sock_close() File "/opt/gvm/gvmpy/lib/python3.11/site-packages/paho/mqtt/client.py", line 1119, in _sock_close if not self._sock: ^^^^^^^^^^ AttributeError: 'MQTTClient' object has no attribute '_sock'

2024-02-02 - January Greenbone Releases

2024-01-16 - January Greenbone Releases

2023-12-01 - secondary_certs.sh now takes cert location from commandline

2023-12-01 - Tested on Debian 12 Bookworm + Environment variables from separate file (.env)

2023-11-30 - OpenVAS-Scanner updated

2023-11-28 - OpenVAS-Scanner updated

2023-11-24 - Updates to GSA, ospd-openvas, NOTUS-Scanner, and pg-gvm.

2023-11-18 - OpenVAS-SMB.

2023-11-15 - Latest versions.

2023-11-04 - Latest versions.

2023-10-29 - Latest versions.

2023-09-28 - GSA Update.

2023-09-16 - Latest versions.

2023-07-15 - GVMD updated to 22.5.5.

2023-07-12 - GVMD updated to working version.

2023-07-10 - Several modules updated.

2023-05-24 - gvm-libs version 22.6.1.

2023-05-21 - libxml2 from source.

2023-05-20 - OpenVAS Scanner upgrade.

2023-05-09 - OpenVAS Scanner upgrade, current versions

2023-05-06 - gvmpy Virtual Environment and latest versions of several components

2023-01-23 - GVMD version 22.4.1 and more + updated scripts (Python GMP)

2022-12-18 - GVMD version 22.4.1 and more + updated scripts (Python GMP)

2022-12-05 - Scripts and updated readme

2022-10-28 - Latest version 22.4.x

2022-03-18 - GSAD Couldn't create pid file after restart

2022-03-05 - Updated to 21.4.4 released February 24th 2022

2021-12-19 - Greenbone Security Assistant Daemon (GSAD) behind NGINX Proxy

2021-12-18 - Automated addition of secondary

2 I've successfully used 3389/TCP on networks that wouldn't allow port 9390 "for security reasons" but allowed RDP across all networks. (Yeah, those stupid rules do exist).

2021-12-12 - NodeJS 14 instead of 12.x with Buster and Bullseye

2021-11-14 - Vagrantfile and bootstrap for testing with vagrant

2021-10-25 - Correct ospd.sock patch. Without this NVTs, scan configs, and compliance policies do not sync

2021-10-23 - oct 13 bugfixes, moved install to /opt/gvm/ & /var/lib/gvm/ instead of /usr/local/ and use yarn from Deb repo

2021-09-14 - Debian 11 support

2021-05-08 - updated to 21.04.

2021-09-24 - August Greenbone releases

GSE Overview

The overall components are depicted in the figure below. All dotted lines are transitory, existing only during installation, that is initiated by the bash scripts.

Overview

Production Installation

1. Install a basic (net-install) Debian 12 (Bookworm) or 11 (Bullseye) server for the primary

Run install-gse.sh and wait for a (long) while.

Note: Several issues with TEX, currently resolved by installing texlive-full. Installing texlive-full takes a lot of time compared to everything else installed, but Debian has a quirk here that sometimes breaks apt when not installing texlive-full.

2. Install as many basic (net-install) Debian 12 (Bookworm) or 11 (Bullseye) servers needed for secondaries

Run install-gse-secondary.sh and wait for installation to finish.

3. Add secondaries

Run add-secondary-2-primary.sh on the primary.

The add-secondary-2-primary.sh does the following. a) Copies required certificates to the secondary. b) runs the helper script secondary-certs.sh on the secondary to ensure all certificates are in the right location. c) restarts ospd-openvas on the secondary. c) configures GVMD to use this scanner.

  1. You can now verify the secondary using either the UI or gvmd with the switch '--verify-scanner=' as discussed later in this README.
Verify Scanner

If this fails, just copy the .pem files from /var/lib/gvm/secondaries/hostname_of_secondary/ to the new secondary, run secondary-certs.sh and ospd-openvas.service should start and scanner can be verified. Follow the steps under Manual Installation below.

Vagrant installation

Provided you have Vagrant and VirtualBox installed, installation is "just".

  1. git clone https://github.com/martinboller/gse.git
  2. cd /gse/
  3. vagrant up

In reality you might have to do the following the first time to build the testlab:

Packages required:

Installation

VirtualBox

Both software titles can be downloaded from https://www.virtualbox.org/ They can also be added to your package manager, which help with keeping them up-to-date. This can also easily be changed to run with VMWare.

Vagrant

Testlab

This will install a primary called "manticore" and a secondary called "aboleth", which can be changed inside "Vagrantfile". Prerequisite: A DHCP server on the network, alternatively change the NIC to use a static or NAT within Vagrantfile.

You may have to select which NIC to use for this e.g. wl02p01. Logon to the website on the server https://manticore (if you have not changed the hostname and DNS works. If not, use the ip address).

The first install will take longer, as it needs to download the Vagrant box for Debian 11 or (preferred) Debian 12 (which this build is based on) first, however that’ll be reused in subsequent installations.

Other useful tips and tricks

Scanners

The first OpenVas scanner is always UUID: 08b69003-5fc2-4037-a479-93b440211c73. The script verifies bot the OpenVAS and the GVMD Scanner by running. For OpenVAS:

su gvm -c '/opt/gvm/sbin/gvmd --verify-scanner 08b69003-5fc2-4037-a479-93b440211c73'

Which should return this (Version Mar. 2022).

Scanner version: OpenVAS 21.4.4.

For GVM:

su gvm -c '/opt/gvm/sbin/gvmd --verify-scanner 6acd0832-df90-11e4-b9d5-28d24461215b'

Which should return this (Version Mar. 2022).

Scanner version: GVM/21.4.5.

Admin Account

During install an Admin user is created, and the initial password stored here: 

cat /opt/gvm/lib/adminuser.

It is good security practice to change this (do it now):

/opt/gvm/sbin/gvmd --user admin --new-password 'Your new password'

Feed Owner

The admin account is import feed owner: https://community.greenbone.net/t/gvm-20-08-missing-report-formats-and-scan-configs/6397/2 So do not delete this account, unless you reconfigure it to be another. Do remember to change its initial password as discussed here.

Without a feed owner there will be no feeds!! (ask me how I know)

If you want to change feedowner, the following commands can be used to create another account and make that the feedowner. You can also just change it in install-gse.sh before running it the first time.

su gvm -c '/opt/gvm/sbin/gvmd --create-user=MyOwnUser'

Get the UUIDs of all users.

su gvm -c '/opt/gvm/sbin/gvmd --get-users --verbose'

Or just for your newly created user.

su gvm -c '/opt/gvm/sbin/gvmd --get-users --verbose | grep MyOwnUser'

Pick the UUID for the one you just created in the list provided and replace UUID of new account below.

su gvm -c '/opt/gvm/sbin/gvmd --modify-setting 78eceaec-3385-11ea-b237-28d24461215b --value UUID of new account'

Useful logs

Manually adding a secondary

1. On the primary; Create the certificate and key needed (The primary is the CA for all secondaries as well as itself)

create a directory for the files needed, and:

Before doing the above, verify if the required certificates can be created by add-secondary-2-primary.sh, as that will still do most of the work even if not able to copy the required files to the secondary.

2. On the secondary, do as follows to get the certs and keys in place:

Copy the created secondary-cert.pem, secondary-key.pem, as well as the cacert.pem file to the secondary (the cacert.pem can be found in /var/lib/gvm/CA/ on the primary)

su gvm -c 'cp ./secondary-cert.pem /var/lib/gvm/CA/'
su gvm -c 'cp ./secondary-key.pem /var/lib/gvm/private/CA/'
su gvm -c 'cp ./cacert.pem /var/lib/gvm/CA/'

Restart ospd-openvas:

systemctl restart ospd-openvas.service

Update Openvas feed:

su gvm -c '/opt/gvm/sbin/openvas --update-vt-info'

3. On the primary, create the scanner in GVMD

Whereever the required files (secondary-cert.pem and secondary-key.pem) are:

chown gvm:gvm *.pem
su gvm -c '/opt/gvm/sbin/gvmd --create-scanner="OSP Scanner secondary hostname" --scanner-host=hostname --scanner-port=9390 --scanner-type="OpenVas" --scanner-ca-pub=/var/lib/gvm/CA/cacert.pem --scanner-key-pub=./secondary-cert.pem --scanner-key-priv=./secondary-key.pem'

Example:

su gvm -c '/opt/gvm/sbin/gvmd --create-scanner="OpenVAS Secondary host aboleth" --scanner-host=aboleth --scanner-port=9390 --scanner-type="OpenVas" --scanner-ca-pub=/var/lib/gvm/CA/cacert.pem --scanner-key-pub=./secondary-cert.pem --scanner-key-priv=./secondary-key.pem'

Which should output this: Scanner created.

4. Verification steps on the primary

su gvm -c '/opt/gvm/sbin/gvmd --get-scanners'

Outputting something like this (the UUID will be different for the scanner just created) 08b69003-5fc2-4037-a479-93b440211c73 OpenVAS /var/run/ospd/ospd-openvas.sock 0 OpenVAS Default 6acd0832-df90-11e4-b9d5-28d24461215b CVE 0 CVE 3e2232e3-b819-41bc-b5be-db52bfb06588 OpenVAS mysecondary 9390 OSP Scanner mysecondary

Verify the secondary just added:

su gvm -c '/opt/gvm/sbin/gvmd --verify-scanner=3e2232e3-b819-41bc-b5be-db52bfb06588'

Which, provided the scanner works, should return this:

Scanner version: OpenVAS 21.4.4.

Congrats, You have now added a secondary scanner manually

5. Delete a scanner

If you've created a scanner in error or have decommissioned it, remove it using --delete-scanner.

su gvm -c '/opt/gvm/sbin/gvmd --delete-scanner=f12cca78-c6b9-4fd1-ad4f-9a9eb2037d29'

6. Scripts

Added the following scripts: -create_targets.sh. This takes the admin password as input and creates some example networks -export-csv-report.gmp.py. Creates a csv formatted report. You need to specify user, password, connection type as well as the id of the report: Examples in the script. -export-pdf-report.gmp.py. Same as above but pdf format.

Other useful information

Just after installation, going from empty feeds to fully up-to-date, you'll notice that postgres is being hammered by gvmd and that redis are by ospd-openvas as openvas-scanner uses Redis (on the secondary only ospd-openvas, openvas, and redis is running). When feeds are updated this isn't as obvious, as the delta is significantly less than "everything". Use ps or top to follow along - the UI also show that the feeds are updating under Administration -> Feed Status.

Primary, Web Interface:

Update in progress

Primary, top:

Update in progress, top

Secondary, top:

Update in progress, top

Hang in there, depending on your server it will take quite a while.


Checking Certificates

If you want to check the certificates are correct and contain the desired information, openssl is useful;

For the webserver on the primary host
openssl s_client -showcerts -servername primary_host_name -connect primary_host_name:443

For the secondary with ospd-openvas listening on 9390
openssl s_client -showcerts -servername secondary_host_name -connect secondary_host_name:9390

When removing Vagrant specific configs, control the VMs using the Hypervisors specific tools, for VirtualBox:

Blog Post

There's a short companion blogpost on https://blog.infosecworrier.dk/2020/12/building-your-own-greenbone.html