Incorrect progress calculation due to excluded_hosts results in Interrupted scan

khesterproton commented 10 months ago

Originally asked as a forum question: https://forum.greenbone.net/t/excluded-hosts-causing-interrupted-scans/16257

I apologize if opening a ticket here isn't the correct way to continue the discussion!

Expected behavior

My use case is scanning a single specific FQDN on a single specific IP.

I expect to be able to create a FQDN target and exclude a list of IPs.

Steps to reproduce

For example, a.com resolves to 1.1.1.1 and 2.2.2.2, and I want to scan this single hostname on 2.2.2.2 only. I create a target for a.com, set excluded_hosts=1.1.1.1 and expand_vhosts=0.

I expect this to do a DNS lookup for a.com and find 1.1.1.1 and 2.2.2.2, and to exclude 1.1.1.1 and scan 2.2.2.2. It should use the one vhost a.com and expand to no other domains on 2.2.2.2.

Actual behavior

This works.

However, when the scan completes it is set to Interrupted rather than Done. I believe there's no error in the scan, and that this is because of the OSPD progress calculation.

For example, with 1 included domain and 1 excluded IP, ospd-openvas.log:

Host scan finished.
Host scan got interrupted. Progress: 50, Status: RUNNING
Scan interrupted.
Scan process is dead and its progress is 50

With 1 included domain and 2 excluded IPs the result is a scan Interrupted at 33%.

My assumptions based on the code

The problem appears to be in the OSPD. Initially the host count is unknown. The progress is calculated ospd/scan.py calculate_target_progress().

This calls get_count_total(), which sees there's no cached count in the scans table. This then calls get_host_count() which finds the one host and calls update_count_total(count_total=1) to cache the count for subsequent calls.

We have the 1 host (a.com) that takes part in the progress calculation. The excluded host (1.1.1.1) is ignored, i.e.simplify_exclude_host_count() sees that the excluded IPs aren't part of the host list and treats them as invalid_exc_hosts rather than counting them.

All is good until the OSPD daemon later updates the cached total with a different value.

ospd_openvas/daemon.py report_results() gets called while the scan is in progress. I haven't investigated the circumstances of this.

Part of this updates the host count, calling ospd/ospd.py set_scan_total_hosts(count_total=2) which calls ospd/scan.py update_count_total(count_total=2). I.e. this is a count that includes the exclude hosts.

Now the progress calculation uses a host count of 2. Since the "invalid" excluded host (1.1.1.1) is still ignored it comes up with a total of 2.

There could well be a work-around/alternative way to scan a specific domain and IP pair that avoids this problem! I'm afraid I haven't found it though.

For a newcomer to the code the host count logic in the different parts of the Greenbone ecosystem is a bit confusing :) I couldn't see a good place to make a change that wouldn't break something else!

GVM versions

I’ve built from source:

OSPd OpenVAS version 22.6.1 Greenbone Security Assistant 22.07.0 Greenbone Vulnerability Manager 23.0.1 Manager DB revision 255 OpenVAS 22.7.6 gvm-libs 22.7.3

Environment

DISTRIB_ID=Ubuntu DISTRIB_RELEASE=22.04 DISTRIB_CODENAME=jammy DISTRIB_DESCRIPTION="Ubuntu 22.04.1 LTS"

ArnoStiefvater commented 10 months ago

Hey @khesterproton,

Thanks a lot for creation of this issue and your thorough investigation! We will also look into it.

jjnicola commented 9 months ago

Jira: SC-964

greenbone / ospd-openvas