greenbone / ospd-openvas

ospd-openvas is an OSP server implementation to allow GVM to remotely control an OpenVAS Scanner
GNU Affero General Public License v3.0
67 stars 58 forks source link

[22.4.2] - raise Exception("GPG verification of notus sha256sums failed") #765

Closed tgurr closed 2 years ago

tgurr commented 2 years ago

Running into an error with the latest version 22.4.2 (downgrading back to 22.4.0 resolves the problem).

Expected behavior

Starting up and running without any issues (like 22.4.0 did and still does for me).

Actual behavior

Running into the following error since upgrading from 22.4.0 to 22.4.2.

Sep 06 18:34:57 hostname ospd-openvas[4407]: Traceback (most recent call last):
Sep 06 18:34:57 hostname ospd-openvas[4407]:   File "/usr/host/bin/ospd-openvas", line 33, in <module>
Sep 06 18:34:57 hostname ospd-openvas[4407]:     sys.exit(load_entry_point('ospd-openvas==22.4.2', 'console_scripts', 'ospd-openvas')())
Sep 06 18:34:57 hostname ospd-openvas[4407]:   File "/usr/lib/python3.10/site-packages/ospd_openvas/daemon.py", line 1243, in main
Sep 06 18:34:57 hostname ospd-openvas[4407]:     daemon_main('OSPD - openvas', OSPDopenvas, NotusParser())
Sep 06 18:34:57 hostname ospd-openvas[4407]:   File "/usr/lib/python3.10/site-packages/ospd/main.py", line 164, in main
Sep 06 18:34:57 hostname ospd-openvas[4407]:     daemon.init(server)
Sep 06 18:34:57 hostname ospd-openvas[4407]:   File "/usr/lib/python3.10/site-packages/ospd_openvas/daemon.py", line 524, in init
Sep 06 18:34:57 hostname ospd-openvas[4407]:     self.update_vts()
Sep 06 18:34:57 hostname ospd-openvas[4407]:   File "/usr/lib/python3.10/site-packages/ospd_openvas/daemon.py", line 649, in update_vts
Sep 06 18:34:57 hostname ospd-openvas[4407]:     self.nvti.notus.reload_cache()
Sep 06 18:34:57 hostname ospd-openvas[4407]:   File "/usr/lib/python3.10/site-packages/ospd_openvas/notus.py", line 119, in reload_cache
Sep 06 18:34:57 hostname ospd-openvas[4407]:     if self._verifier(f):
Sep 06 18:34:57 hostname ospd-openvas[4407]:   File "/usr/lib/python3.10/site-packages/ospd_openvas/gpg_sha_verifier.py", line 121, in verify
Sep 06 18:34:57 hostname ospd-openvas[4407]:     assumed_name = sha256sums().get(hash_sum)
Sep 06 18:34:57 hostname ospd-openvas[4407]:   File "/usr/lib/python3.10/site-packages/ospd_openvas/gpg_sha_verifier.py", line 63, in internal_reload
Sep 06 18:34:57 hostname ospd-openvas[4407]:     return config.on_verification_failure(None)
Sep 06 18:34:57 hostname ospd-openvas[4407]:   File "/usr/lib/python3.10/site-packages/ospd_openvas/notus.py", line 50, in on_hash_sum_verification_failure
Sep 06 18:34:57 hostname ospd-openvas[4407]:     raise Exception("GPG verification of notus sha256sums failed")
Sep 06 18:34:57 hostname ospd-openvas[4407]: Exception: GPG verification of notus sha256sums failed
Sep 06 18:34:57 hostname ospd-openvas[4407]: Exception ignored in atexit callback: <function exit_cleanup at 0x7f5245740310>
Sep 06 18:34:57 hostname ospd-openvas[4407]: Traceback (most recent call last):
Sep 06 18:34:57 hostname ospd-openvas[4407]:   File "/usr/lib/python3.10/site-packages/ospd/main.py", line 86, in exit_cleanup
Sep 06 18:34:57 hostname ospd-openvas[4407]:     sys.exit()
Sep 06 18:34:57 hostname ospd-openvas[4407]: SystemExit:
Sep 06 18:34:57 hostname systemd[1]: ospd-openvas.service: Main process exited, code=exited, status=1/FAILURE
Sep 06 18:34:57 hostname systemd[1]: ospd-openvas.service: Failed with result 'exit-code'.

Steps to reproduce

  1. upgrade ospd-openvas from the previously working 22.4.0 to 22.4.2
  2. start service
  3. run into error

GVM versions

gsa: Greenbone Security Assistant 22.04.0

gvm: Greenbone Vulnerability Manager 22.4.0~dev1 (<- note: ~dev1 was somehow introduced between tag 22.4 and the actual release tag 22.4.0 with the change to PROJECT_DEV_VERSION 1 in CMakeLists.txt: https://github.com/greenbone/gvmd/compare/v22.4...v22.4.0) Manager DB revision 250

openvas-scanner: OpenVAS 22.4.0

gvm-libs: gvm-libs 22.4.0

Environment

Operating system: Exherbo Linux

Installation method / source: source-based packages

Logfiles

/var/log/gvm/ospd-openvas.log

OSPD[14136] 2022-09-06 16:52:33,999: INFO: (ospd.main) Starting OSPd OpenVAS version 22.4.2.
OSPD[14136] 2022-09-06 16:52:34,007: WARNING: (ospd_openvas.messaging.mqtt) Could not connect to MQTT broker, error was: [Errno 111] Connection refused. Trying again in 10s.
OSPD[14136] 2022-09-06 16:52:44,020: WARNING: (ospd_openvas.messaging.mqtt) Could not connect to MQTT broker, error was: [Errno 111] Connection refused. Trying again in 10s.
OSPD[14136] 2022-09-06 16:52:44,054: INFO: (ospd_openvas.daemon) Loading VTs. Scans will be [requested|queued] until VTs are loaded. This may take a few minutes, please wait...
OSPD[14136] 2022-09-06 16:52:44,242: WARNING: (gnupg) potential problem: ERROR: add_keyblock_resource 33587201
OSPD[14136] 2022-09-06 16:52:44,243: WARNING: (gnupg) potential problem: ERROR: keydb_search 33554445
OSPD[14136] 2022-09-06 16:52:44,243: WARNING: (gnupg) potential problem: ERROR: keydb_search 33554445
OSPD[14136] 2022-09-06 16:52:44,243: WARNING: (gnupg) gpg returned a non-zero error code: 2
OSPD[14136] 2022-09-06 16:52:44,252: INFO: (ospd.main) Shutting-down server ...

Note for the MQTT broker WARNING: I've not yet setup MQTT & packaged notus-scanner, so I already had that warning with 22.4.0 previously as well of course.

Additional information:

# ls -la /var/lib/notus/advisories
insgesamt 46828
drwxrwxr-x 2 gvm gvm     4096  6. Sep 12:42 .
drwxrwxr-x 4 gvm gvm     4096  6. Sep 12:42 ..
-rw-rw-r-- 1 gvm gvm 14294650  6. Sep 06:38 euleros.notus
-rw-rw-r-- 1 gvm gvm  9050712  6. Sep 06:38 mageia.notus
-rw-rw-r-- 1 gvm gvm      318  6. Sep 06:38 sha256sums
-rw-rw-r-- 1 gvm gvm      833  6. Sep 06:38 sha256sums.asc
-rw-rw-r-- 1 gvm gvm  2522789  6. Sep 06:38 slackware.notus
-rw-rw-r-- 1 gvm gvm 22062329  6. Sep 06:38 suse.notus
# ls -la /var/lib/gvm/gvmd/gnupg
insgesamt 32
drwx------ 4 gvm gvm 4096  6. Sep 18:56 .
drwxr-xr-x 4 gvm gvm 4096  6. Sep 17:35 ..
drwx------ 2 gvm gvm 4096 21. Okt 2019  openpgp-revocs.d
drwx------ 2 gvm gvm 4096 21. Okt 2019  private-keys-v1.d
-rw------- 1 gvm gvm  818 21. Okt 2019  pubring.kbx
-rw------- 1 gvm gvm   32 21. Okt 2019  pubring.kbx~
-rw------- 1 gvm gvm  600  6. Sep 18:56 random_seed
-rw------- 1 gvm gvm 1280 21. Okt 2019  trustdb.gpg
# cat /etc/gvm/ospd-openvas.conf 
[OSPD - openvas]
log_level = INFO
socket_mode = 0o770
unix_socket = /run/ospd/ospd-openvas.sock
pid_file = /run/ospd/ospd-openvas.pid
log_file = /var/log/gvm/ospd-openvas.log
lock_file_dir = /run/ospd

I also tried adding notus-feed-dir = /var/lib/notus/advisories to the ospd-openvas.conf as I've seen it's also passed in your systemd file suggestion at https://greenbone.github.io/docs/latest/22.4/source-build/index.html#setting-up-services-for-systemd but it didn't make any difference.

nichtsfrei commented 2 years ago

The exception is thrown when the cache could not be initialized due to a verification error; the line: OSPD[14136] 2022-09-06 16:52:44,243: WARNING: (gnupg) gpg returned a non-zero error code: 2 indicates that gpg verification failed.

However rather than printing a stacktrace and quitting ospd-openvas I think it should print a warning and ignore the notus advisories.

nichtsfrei commented 2 years ago

For your actual issue: I think you need to set a proper GNUPGHOME environment variable otherwise it is using: /etc/openvas/gnupg which on most machines isn't accurate.

So to start ospd-openvas as your user you could use:

GNUPGHOME=$HOME/.gnupg/ ospd-openvas --config ~/ospd.conf -f

In the meantime I am going to find a way to handle this default more gracefully for future versions.

tgurr commented 2 years ago

In the meantime I am going to find a way to handle this default more gracefully for future versions.

Thanks for implementing the steps to handle this gracefully for future versions.

For your actual issue: I think you need to set a proper GNUPGHOME environment variable otherwise it is using: /etc/openvas/gnupg which on most machines isn't accurate.

Thanks for the explanation, the documentation at https://greenbone.github.io/docs/latest/22.4/source-build/index.html#feed-validation is a little bit scattered since some required inital steps are found at the step before at https://greenbone.github.io/docs/latest/22.4/source-build/index.html#importing-the-greenbone-signing-key and it's a hassle to implement this packaging wise as running gnupg import will start gpg-agent and requires a socket which fails without workarounds for us as our package manager runs in a sandbox, see https://git.exherbo.org/net.git/commit/?id=2c1b948ed22f937215d65944863ff88a64e6de14 on the required steps to workaround this problem for us. It would be great if we could e.g. just dump the GBCommunitySigningKey.asc and a ownertrust.txt in /etc/openvas or /etc/openvas/gnupg and ospd-openvas would handle the import automatically on startup/runtime. After implementing the key import steps (as seen in the linked commit above) 22.4.2 runs fine here as well.

nichtsfrei commented 2 years ago

I will forward your suggestions regarding the documentation to the corresponding teams.

Although I understand that within a sandbox that it would be great to automatically import public keys and ownertrust on an initial start of ospd-openvas it would may interfere with our current packaging.

Since this is a feature request it would help me when you create an own ticket for that so that I can discuss it more easily.