ospd failed to load notus data

[deajan]

Hello,

Trying your setup, I noticed that notus scanner data cannot be loaded by ospd:

OSPD[640] 2023-04-18 04:28:10,673: WARNING: (gnupg) gpg returned a non-zero error code: 2
OSPD[640] 2023-04-18 04:28:10,674: WARNING: (ospd_openvas.notus) GPG verification of notus sha256sums failed. Notus advisories are not loaded.
OSPD[640] 2023-04-18 04:28:10,674: WARNING: (ospd_openvas.notus) ignoring /var/lib/notus/advisories/slackware.notus due to invalid signature
OSPD[640] 2023-04-18 04:28:10,702: WARNING: (gnupg) gpg returned a non-zero error code: 2
OSPD[640] 2023-04-18 04:28:10,702: WARNING: (ospd_openvas.notus) GPG verification of notus sha256sums failed. Notus advisories are not loaded.
OSPD[640] 2023-04-18 04:28:10,702: WARNING: (ospd_openvas.notus) ignoring /var/lib/notus/advisories/ubuntu.notus due to invalid signature
OSPD[640] 2023-04-18 04:28:10,749: WARNING: (gnupg) gpg returned a non-zero error code: 2
OSPD[640] 2023-04-18 04:28:10,749: WARNING: (ospd_openvas.notus) GPG verification of notus sha256sums failed. Notus advisories are not loaded.
OSPD[640] 2023-04-18 04:28:10,749: WARNING: (ospd_openvas.notus) ignoring /var/lib/notus/advisories/suse.notus due to invalid signature
OSPD[640] 2023-04-18 04:28:10,783: WARNING: (gnupg) gpg returned a non-zero error code: 2
OSPD[640] 2023-04-18 04:28:10,783: WARNING: (ospd_openvas.notus) GPG verification of notus sha256sums failed. Notus advisories are not loaded.
OSPD[640] 2023-04-18 04:28:10,783: WARNING: (ospd_openvas.notus) ignoring /var/lib/notus/advisories/euleros.notus due to invalid signature
OSPD[640] 2023-04-18 04:28:10,815: WARNING: (gnupg) gpg returned a non-zero error code: 2
OSPD[640] 2023-04-18 04:28:10,815: WARNING: (ospd_openvas.notus) GPG verification of notus sha256sums failed. Notus advisories are not loaded.
OSPD[640] 2023-04-18 04:28:10,815: WARNING: (ospd_openvas.notus) ignoring /var/lib/notus/advisories/debian.notus due to invalid signature
OSPD[640] 2023-04-18 04:28:10,835: WARNING: (gnupg) gpg returned a non-zero error code: 2
OSPD[640] 2023-04-18 04:28:10,836: WARNING: (ospd_openvas.notus) GPG verification of notus sha256sums failed. Notus advisories are not loaded.
OSPD[640] 2023-04-18 04:28:10,836: WARNING: (ospd_openvas.notus) ignoring /var/lib/notus/advisories/mageia.notus due to invalid signature

I've tried this with three different fresh images of yours (and of course fresh data mounts in docker):

• immauss/openvas:latest
• immauss/openvas:22.4.11
• immauss/openvas:gaby

I've gone into the shell of those, and cannot see what is the culprit. feed syncs have been done without problems.

sha256 sums of notus advisories are correct:

root@e234c1dde715:/var/lib/notus/advisories# ls -alh
total 79M
drwxrwxr-x 2 gvm gvm  168 Apr 17 10:45 .
drwxrwxr-x 4 gvm gvm   98 Apr 17 10:45 ..
-rw-rw-r-- 1 gvm gvm  16M Apr 17 09:34 debian.notus
-rw-rw-r-- 1 gvm gvm  16M Apr 17 09:34 euleros.notus
-rw-rw-r-- 1 gvm gvm 9.1M Apr 17 09:34 mageia.notus
-rw-rw-r-- 1 gvm gvm  476 Apr 17 09:34 sha256sums
-rw-rw-r-- 1 gvm gvm  833 Apr 17 09:34 sha256sums.asc
-rw-rw-r-- 1 gvm gvm 2.6M Apr 17 09:34 slackware.notus
-rw-rw-r-- 1 gvm gvm  24M Apr 17 09:34 suse.notus
-rw-rw-r-- 1 gvm gvm  14M Apr 17 09:34 ubuntu.notus
root@e234c1dde715:/var/lib/notus/advisories# cat sha256sums
078fbd9f93bcde0d6517f9f857cf406f2024b389f1326135aae1b8a7375dc530  debian.notus
128dc05adf947e925d6aa393491f57e6812b9df2235c6fb460f73a8d1e99d8c2  euleros.notus
421f4167e539c51f0d0894d02e0bd238f5089915df0d2c5de8ee8336946a72cf  mageia.notus
45d583c115ddd8e482a54bd7e5e9c7c29a98a55bdd993d89e20c68e534296bb9  slackware.notus
a93f0f8b620cf72e233ec9be081e6ba86da2e3efcd5a1dcb595caf02250c47ee  suse.notus
49ab22519154b4078aceb240087468205bb7cc9e2b153687ce95fcbe48119eab  ubuntu.notus
root@e234c1dde715:/var/lib/notus/advisories# sha256sum debian.notus
078fbd9f93bcde0d6517f9f857cf406f2024b389f1326135aae1b8a7375dc530  debian.notus
root@e234c1dde715:/var/lib/notus/advisories# sha256sum suse.notus
a93f0f8b620cf72e233ec9be081e6ba86da2e3efcd5a1dcb595caf02250c47ee  suse.notus

Can I provide any helpful data perhaps ?

[deajan]

So far I've modded the image by adding --disable-notus-hashsum-verification True into /scripts/openvas.sh and /scripts/single.sh Also added --disable-hashsum-verification True in /scripts/notus-scanner.sh

This permits me to test notus. Of course, this is far from an ideal solution.

[immauss]

If you are still seeing this with the latest, could you please provide details on how you are starting the container.

Thank you, -Scott

[immauss]

OK .. found the issue here .... Need to set the environment variable for the OPENVAS_GPG_HOME ... It was set during the creation of the gpg keys, but not at runtime ... 22.4.14 resolves this.

Thanks, Scott

[deajan]

Thanks, I'll try this shortly. Btw, even with notus running via my workaround, I got 80k instead of 120k NVTs shown in GUI. Do you run feed update with--notus too ? Didn't see it in sync.sh

[immauss]

To the best of my knowledge, there is no "--notus" option. ( You freaked me out, so I just double-checked.) But, my production, which is running the latest is showing 128k NVTs.

[deajan]

Lol, sorry for that. I meant greenbone-nvt-sync --type notus. I'll recheck with your newer build shortly.

[immauss]

greenbone-nvt-sync does not have a "--type" option. It just syncs all of the NVTs.

su -c "./greenbone-nvt-sync --help" gvm 
<28>May 11 03:36:09 greenbone-nvt-sync: The log facility is not working as expected. All messages will be written to the standard error stream.
./greenbone-nvt-sync: Sync NVT data
 --describe      display current feed info
 --feedcurrent   just check if feed is up-to-date
 --feedversion   display version of this feed
 --help          display this help
 --identify      display information
 --nvt-dir dir   set dir as NVT directory
 --notus-dir dir set dir as NOTUS directory
 --selftest      perform self-test and set exit code
 --verbose       makes the sync process print details
 --version       display version

The greenbone-feed-sync script has a "--type" option, but "NOTUS" is not one of those options. Unless GB slipped something in somewhere without documenting it

./greenbone-feed-sync --help 
Running as root
./greenbone-feed-sync: Sync feed data
No access key found: Using Community Feed
 --describe      display current feed info
 --feedversion   display version of this feed
 --help          display this help
 --identify      display information
 --selftest      perform self-test
 --type <TYPE>   choose type of data to sync (CERT, SCAP or GVMD_DATA)
 --version       display version

To the best of my knowledge, and after reviewing the installation docs for 22.4, the greenbone-nvt-sync and feed-sync scripts with the types listed above cover the data needed for the notus scanner scans.

If I missed something, please let me know as that would be a huge omission.

Thanks, Scott

[deajan]

Then there's obviously something very strange in my witness setup where I am comparing self build with your docker build openvas

[immauss]

You are building from the master branch. I'm building from the stable branch.

Your version is 23.4 22.4 does not have that option.

[deajan]

^^ Well, then, sorry for the noise. I'll still have to try your new build, but obviously I'll need to compare the comparable.

What puzzles me in my build is that I used the advertised versions, eg:

Anyway, that's out of the scope of this issue.

[immauss]

More curious ... the nvt-sync is packaged with the openvas-scanner. Your openvas_scanner is 22.4.2 mine is 22.6.2 Yet your script is 23.4.0

I automated a check for the latest stable relase via the github APIs, so that's where I'm getting the versions. currently: pg_gvm=v22.4.0 notus_scanner=v22.5.0 gvmd=v22.4.2 openvas=v22.6.2 openvas_smb=v22.5.0 gvm_libs=v22.5.2 openvas_scanner=v22.6.2 gsa=v22.4.1 ospd=v21.4.4 ospd_openvas=v22.5.1 python_gvm=v23.4.2 gvm_tools=v23.4.0

Something similar came up on the forums, so I asked one of the lead developers to clarify.

https://forum.greenbone.net/t/why-does-gvm-22-4-1-have-half-of-the-scan-capabiliities-gvm-20-08-had/14458/5

[deajan]

Yes, you actually posted on my thread in the forum ^^

[immauss]

lol ... I guess I did ...

It seems there is a new version of the script separate from the other repos.

But, the end result, is that what I have should be getting everything. GB is not very good about announcing things like this for the opensource bits. Which was what led me to writing the github API bit for the latest releases. Guess I have something else to add to my build now. Thanks, Scott