Garbage collection issue with multiple nmap boefjes enabled (tcp + udp)

[stephanie0x00]

Tldr; The trigger for this bug is the nmap udp scan. If nmap scans already exists and the udp scans complete, the first data is discarded. During the standup it was mentioned that this was due to insufficient garbage collection.

Describe the bug There appears to be a bug in the parsing or visualisation of nmap data for TCP data. When scanning against mispo.es I observe the following data:

$ nmap mispo.es
Starting Nmap 7.80 ( https://nmap.org ) at 2024-04-23 11:18 CEST
Nmap scan report for mispo.es (134.209.85.72)
Host is up (0.0096s latency).
Not shown: 992 filtered ports
PORT     STATE  SERVICE
22/tcp   open   ssh
23/tcp   closed telnet
53/tcp   open   domain
80/tcp   open   http
110/tcp  closed pop3
143/tcp  closed imap
443/tcp  open   https
3306/tcp open   mysql

Nmap done: 1 IP address (1 host up) scanned in 4.80 seconds

When enabling the Nmap TCP boefje I observe that at first I get a finding for port 3306 as expected. However after waiting half an hour or so, this finding disappears from the finding list as shown in the screenshots below.

12 findings observed at: 16:45:

9 findings observed at 17:15:

To Reproduce Steps to reproduce the behavior:

1. Scan an object (mispo.es) with nmap tcp and DNS boefjes enabled
2. Observe that the open database port (mysql) was found and has a finding.
3. Enable nmap udp boefje
4. Wait for the scan to complete and observe that the open database port has disappeared.

Expected behavior A finding for the presence of port 3306.

OpenKAT version commit e511c482e0a4eae8842e459e87ea401498fc01b6 (HEAD -> main, origin/main, origin/HEAD)

Raw files Output of raw files are shown below:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE nmaprun>
<?xml-stylesheet href="file:///usr/bin/../share/nmap/nmap.xsl" type="text/xsl"?>
<!-- Nmap 7.94 scan initiated Mon Apr 22 14:44:36 2024 as: /usr/bin/nmap -&#45;open -T4 -Pn -r -v10 -sV -sS -&#45;top-ports 250 -oX - 134.209.85.72 -->
<nmaprun scanner="nmap" args="/usr/bin/nmap -&#45;open -T4 -Pn -r -v10 -sV -sS -&#45;top-ports 250 -oX - 134.209.85.72" start="1713797076" startstr="Mon Apr 22 14:44:36 2024" version="7.94" xmloutputversion="1.05">
<scaninfo type="syn" protocol="tcp" numservices="250" services="1,3,7,9,13,17,19-26,33,37,42,53,79-82,88,100,106,110-111,113,119,135,139,143-144,161,179,199,222,254-255,264,280,311,389,407,427,443-445,464-465,497,500,512-515,543-544,548,554,563,587,593,625,631,636,646,787,808,873,888,902,990,992-993,995,999-1000,1022-1044,1048-1050,1053-1054,1056,1058-1059,1064-1066,1068-1069,1071,1074,1080,1110-1111,1218,1234,1352,1433,1494,1521,1700,1717,1720,1723,1755,1761,1801,1900,1935,1998,2000-2010,2049,2065,2103,2105,2107,2121,2161,2301,2383,2401,2601-2602,2701,2717,2869,2967,3000-3001,3052,3128,3260,3268-3269,3306,3389,3689-3690,3703,3986,4000-4001,4045,4444,4662,4899,5000-5001,5003,5009,5050-5051,5060,5101,5120,5190,5357,5432,5550,5555,5631,5666,5800-5801,5900-5901,6000-6002,6004,6112,6543,6646,6666,7000-7001,7019,7070,7100,7937-7938,8000,8002,8008-8010,8031,8080-8082,8443,8888,9000-9001,9090,9100,9102,9999-10001,10010,15000,32768,32770-32772,42510,49152-49157,50000-50001"/>
<verbose level="10"/>
<debugging level="0"/>
<taskbegin task="Parallel DNS resolution of 1 host." time="1713797077"/>
<taskend task="Parallel DNS resolution of 1 host." time="1713797077"/>
<taskbegin task="SYN Stealth Scan" time="1713797077"/>
<taskend task="SYN Stealth Scan" time="1713797080" extrainfo="250 total ports"/>
<taskbegin task="Service scan" time="1713797080"/>
<taskend task="Service scan" time="1713797092" extrainfo="5 services on 1 host"/>
<taskbegin task="NSE" time="1713797092"/>
<taskend task="NSE" time="1713797092"/>
<taskbegin task="NSE" time="1713797092"/>
<taskend task="NSE" time="1713797092"/>
<host starttime="1713797077" endtime="1713797092"><status state="up" reason="user-set" reason_ttl="0"/>
<address addr="134.209.85.72" addrtype="ipv4"/>
<hostnames>
</hostnames>
<ports><extraports state="filtered" count="242">
<extrareasons reason="no-response" count="242" proto="tcp" ports="1,3,7,9,13,17,19-21,24-26,33,37,42,79,81-82,88,100,106,111,113,119,135,139,144,161,179,199,222,254-255,264,280,311,389,407,427,444-445,464-465,497,500,512-515,543-544,548,554,563,587,593,625,631,636,646,787,808,873,888,902,990,992-993,995,999-1000,1022-1044,1048-1050,1053-1054,1056,1058-1059,1064-1066,1068-1069,1071,1074,1080,1110-1111,1218,1234,1352,1433,1494,1521,1700,1717,1720,1723,1755,1761,1801,1900,1935,1998,2000-2010,2049,2065,2103,2105,2107,2121,2161,2301,2383,2401,2601-2602,2701,2717,2869,2967,3000-3001,3052,3128,3260,3268-3269,3389,3689-3690,3703,3986,4000-4001,4045,4444,4662,4899,5000-5001,5003,5009,5050-5051,5060,5101,5120,5190,5357,5432,5550,5555,5631,5666,5800-5801,5900-5901,6000-6002,6004,6112,6543,6646,6666,7000-7001,7019,7070,7100,7937-7938,8000,8002,8008-8010,8031,8080-8082,8443,8888,9000-9001,9090,9100,9102,9999-10001,10010,15000,32768,32770-32772,42510,49152-49157,50000-50001"/>
</extraports>
<extraports state="closed" count="3">
<extrareasons reason="reset" count="3" proto="tcp" ports="23,110,143"/>
</extraports>
<port protocol="tcp" portid="22"><state state="open" reason="syn-ack" reason_ttl="57"/><service name="ssh" product="OpenSSH" version="8.4p1 Debian 5+deb11u3" extrainfo="protocol 2.0" ostype="Linux" method="probed" conf="10"><cpe>cpe:/a:openbsd:openssh:8.4p1</cpe><cpe>cpe:/o:linux:linux_kernel</cpe></service></port>
<port protocol="tcp" portid="53"><state state="open" reason="syn-ack" reason_ttl="57"/><service name="domain" product="ISC BIND" version="9.16.48" extrainfo="Debian Linux" ostype="Linux" method="probed" conf="10"><cpe>cpe:/a:isc:bind:9.16.48</cpe><cpe>cpe:/o:linux:linux_kernel</cpe></service></port>
<port protocol="tcp" portid="80"><state state="open" reason="syn-ack" reason_ttl="57"/><service name="http" product="nginx" version="1.18.0" method="probed" conf="10"><cpe>cpe:/a:igor_sysoev:nginx:1.18.0</cpe></service></port>
<port protocol="tcp" portid="443"><state state="open" reason="syn-ack" reason_ttl="57"/><service name="http" product="nginx" version="1.18.0" tunnel="ssl" method="probed" conf="10"><cpe>cpe:/a:igor_sysoev:nginx:1.18.0</cpe></service></port>
<port protocol="tcp" portid="3306"><state state="open" reason="syn-ack" reason_ttl="57"/><service name="mysql" product="MySQL" extrainfo="unauthorized" method="probed" conf="10"><cpe>cpe:/a:mysql:mysql</cpe></service></port>
</ports>
<times srtt="11990" rttvar="4821" to="100000"/>
</host>
<runstats><finished time="1713797092" timestr="Mon Apr 22 14:44:52 2024" summary="Nmap done at Mon Apr 22 14:44:52 2024; 1 IP address (1 host up) scanned in 15.55 seconds" elapsed="15.55" exit="success"/><hosts up="1" down="0" total="1"/>
</runstats>
</nmaprun>

{"id": "4298824d-9ad2-4b93-8a11-e265a1770524", "boefje_meta": {"id": "acf928e3-b609-43fb-b8cd-1281119df6da", "started_at": "2024-04-22T14:44:36.268977Z", "ended_at": "2024-04-22T14:44:52.669408Z", "boefje": {"id": "nmap", "version": null}, "input_ooi": "IPAddressV4|internet|134.209.85.72", "arguments": {"input": {"object_type": "IPAddressV4", "scan_profile": "scan_profile_type='inherited' reference=Reference('IPAddressV4|internet|134.209.85.72') level=<ScanLevel.L2: 2>", "primary_key": "IPAddressV4|internet|134.209.85.72", "address": "134.209.85.72", "network": {"name": "internet"}, "netblock": "None"}}, "organization": "aa", "runnable_hash": "be65645cb2aabd0d6fb2bc39e218071efce81772ae35515f98c6dddd4aebe813", "environment": {}}, "mime_types": [{"value": "boefje/nmap"}], "secure_hash": "sha512:7f479e15c289458113214604a4a339b8860d98cd5394fa263b5aab115f3d49658ad0f41ce32fcf0176afcbad0729232db6ef291e875304b39cd27747e90e7213", "signing_provider_url": null, "hash_retrieval_link": "310f8b35-62e7-49b7-86e5-6bf6e75d4a71"}

Raw files for the Open-Database-Port finding:

{"KAT-NO-HSTS": {"description": "The website does not use HTTP Strict Transport Security (HSTS). HSTS ensures that browsers can only access the website using encryption (HTTPS).", "source": "https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Strict-Transport-Security", "risk": "medium", "impact": "Absence of the HSTS header allows clients to connect insecurely to the website. This may result in eavesdropping of (sensitive) data by an attacker. Enabling the HSTS header forces the web browser to choose HTTPS instead of HTTP", "recommendation": "Configure the Strict-Transport-Security HTTP header for all websites."}, "KAT-NO-CSP": {"description": "The website does not use a Content Security Policy (CSP) configuration. CSP is used to mitigate certain attacks, including loading malicious code (JavaScript) inside the users browser (XSS)", "source": "https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP", "risk": "medium", "impact": "The usage possibility of JavaScript is not limited by the website. If the website contains a cross-site scripting vulnerability, then JavaScript code can be injected into the web page. This code is then executed by the browser of the victim. If a well-established Content Security Policy is active, the attacker can inject JavaScript code into the browser of the victim, but then the code will not get executed by the browser. A good configured Content Security Policy is a strong protection against cross-site scripting vulnerabilities.", "recommendation": "1. Set the Content-Security-Policy HTTP header in all HTTP answers. 2. Make sure that when the Content Security Policy is violated by a browser, that this violation is logged and monitored. Point the content security violation variable report-uri to a server-side log script. 3. Implement a process that periodically analyses these logs for programming errors and hack attacks."}, "KAT-NO-X-PERMITTED-CROSS-DOMAIN-POLICIES": {"description": "The HTTP header X-Permitted-Cross-Domain- Policies is missing in HTTP responses. This header is not officially supported by Mozilla MDN.", "source": "https://owasp.org/www-project-secure-headers/#div-headers", "risk": "recommendation", "impact": "When the value of this header is not set to master- only, Adobe Flash or Adobe Acrobat (and possibly other software) can also look at cross-domain configuration files hosted at the web server.", "recommendation": "This header is not supported by default by Mozilla. If this header is required for your environment: Set the HTTP header X-Permitted-Cross- Domain-Policies: none in all HTTP responses. Use value master-only if a Flash or Acrobat cross- domain configuration file is used that is placed in the root of the web server"}, "KAT-NO-EXPLICIT-XSS-PROTECTION": {"description": "This is a deprecated header previously used to prevent against Cross-Site-Scripting attacks. Support in modern browsers could introduce XSS attacks again.", "source": "https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-XSS-Protection", "risk": "recommendation", "impact": "Reflected cross-site scripting attacks may not be blocked.", "recommendation": "This header is deprecated and should not be used."}, "KAT-NO-X-FRAME-OPTIONS": {"description": "HTTP header 'X-Frame-Options' is missing. It is possible that the website can be loaded via an <iframe>.", "source": "https://owasp.org/www-project-secure-headers/#div-headers", "risk": "recommendation", "impact": "There is a change that clickjacking is possible. This is an attack technique in which the website is invisibly loaded. On top of the original website, another malicious website is loaded that contains specially placed buttons or links. When the victim clicks on those buttons or links, the mouse click and thus its corresponding action is performed on the original website (which is made invisible). If the victim is logged in, then this click can perform an unauthorized action.", "recommendation": "1. Set the HTTP header <c>X-Frame- Options</c> with value deny (safest) or sameorigin in every HTTP answer for older browsers. 2. Set the frame-ancestors variable in the Content-Security-Policy header for modern browsers. 3. Add JavaScript code to all pages to ensure that these web pages may not be loaded within an <iframe>. In this manner also very old browsers are protected that do not support the HTTP header X-Frame-Options."}, "KAT-NO-X-DNS-PREFETCH-CONTROL": {"description": "This is a non-standard header. The HTTP header X-DNS-Prefetch-Control is missing. The X-DNS-Prefetch-Control HTTP response header controls DNS prefetching, a feature by which browsers proactively perform domain name resolution on both links that the user may choose to follow as well as URLs for items referenced by the document.", "source": "https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/X-DNS-Prefetch-Control", "risk": "recommendation", "impact": "This header not production ready and thus not officially supported by Mozilla MDN.", "recommendation": "If support is required: Set HTTP header X-DNS-Prefetch-Control: off in all HTTP answers."}, "KAT-NO-EXCPECT-CT": {"description": "HTTP header 'Expect-CT' is missing. The Expect-CT header allows sites to opt in to reporting and/or enforcement of Certificate Transparency requirements.", "source": "https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Expect-CT", "risk": "recommendation", "impact": "The Expect-CT header prevents the use of misissued certificates for the website from going unnoticed. This header is currently deprecated thus browsers support is limited.", "recommendation": "This header is deprecated and should not be used. Set HTTP header Expect-CT in all HTTP answers and configure the report-uri variable."}, "KAT-NO-PERMISSIONS-POLICY": {"description": "The HTTP header Permissions-Policy is missing. Via this header a website can set limits on what kind of capabilities a web pages is allowed to access in browsers that render them. For example, the header can prohibit the web page from addressing the microphone, camera, location or phone sensors.", "source": "https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Permissions-Policy", "risk": "recommendation", "impact": "When the website has a cross-site scripting vulnerability, then the attacker exploiting this vulnerability can use all the capabilities of the victim's browser.", "recommendation": "Set the Permissions-Policy HTTP header in all HTTP answers."}, "KAT-NO-REFERRER-POLICY": {"description": "The HTTP header Referrer-Policy is missing in HTTP responses.", "source": "https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referrer-Policy", "risk": "recommendation", "impact": "When a website visitor clicks on a link to another website, the browser sends the Referer HTTP header (a part of the URL) to the other website. This is a privacy leak for the website visitor. In some cases, sensitive information such as session tokens may leak to websites that are linked to.", "recommendation": "Set the header Referrer-Policy: no- referrer in every HTTP answer."}, "KAT-NO-X-CONTENT-TYPE-OPTIONS": {"description": "The HTTP header <c>X-Content-Type- Options</c> is not set. Internet Explorer and Chrome apply MIME type sniffing in order to guess the content type of a document served and ignore the file extension.", "source": "http://www.owasp.org/index.php/Cross-site_Scripting_(XSS)", "risk": "recommendation", "impact": "A malicious user of the system could upload a legitimate file containing HTML code to the website (if such functionality exists) with a file extension such as <c>.jpg</c> or <c>.png</c>. If the victim uses Internet Explorer or Chrome and downloads the malicious file, the uploaded HTML code will be executed, even though the file contains an image extension and the server would return an image header such as <c>Content-Type: image/jpeg</c>. This may include a <i>cross-site scripting</i> vulnerability.", "recommendation": "Set the HTTP header <c>X-Content-Type- Options: nosniff</c> in at least all web pages that contain user input (and uploads)."}, "KAT-SSL-2-SUPPORT": {"description": "The server supports SSL version 2. This is a protocol that encrypts data traffic by using a legacy protocol and encryption ciphers which contains various security vulnerabilities.", "source": "https://github.com/ssllabs/research/wiki/SSL-and-TLS-Deployment-Best-Practices#22-use-secure-protocols", "risk": "medium", "impact": "Attackers, who can perform a man-in-the-middle attack, can decrypt traffic. Thus no integrity or confidentiality is offered to between the client and server.", "recommendation": "Disable support for SSL version 2."}, "KAT-SSL-3-SUPPORT": {"description": "The server supports SSL version 3. This is a protocol that encrypts data traffic by using a legacy protocol and encryption ciphers which contains various security vulnerabilities.", "source": "https://github.com/ssllabs/research/wiki/SSL-and-TLS-Deployment-Best-Practices#22-use-secure-protocols", "risk": "medium", "impact": "Attackers, who can perform a man-in-the-middle attack, can decrypt traffic. Thus no integrity or confidentiality is offered to between the client and server.", "recommendation": "Disable support for SSL version 3."}, "KAT-TLS-1.0-SUPPORT": {"description": "The server supports TLS version 1.0. This is a protocol that encrypts data traffic by using a legacy protocol and encryption ciphers which contains various security vulnerabilities.", "source": "https://github.com/ssllabs/research/wiki/SSL-and-TLS-Deployment-Best-Practices#22-use-secure-protocols", "risk": "medium", "impact": "Attackers, who can perform a man-in-the-middle attack, can decrypt traffic. Thus no integrity or confidentiality is offered to between the client and server. The impact can be reduced by only using secure ciphers.", "recommendation": "Disable support for TLS version 1.1, unless it is required for backwards compatibility of devices. To reduce the attack surface ensure that only secure ciphers are used."}, "KAT-TLS-1.1-SUPPORT": {"description": "The server supports TLS version 1.1. This is a protocol that encrypts data traffic by using a legacy protocol and encryption ciphers which contains various security vulnerabilities.", "source": "https://github.com/ssllabs/research/wiki/SSL-and-TLS-Deployment-Best-Practices#22-use-secure-protocols", "risk": "medium", "impact": "Attackers, who can perform a man-in-the-middle attack, can decrypt traffic. Thus no integrity or confidentiality is offered to between the client and server. The impact can be reduced by only using secure ciphers.", "recommendation": "Disable support for TLS version 1.1, unless it is required for backwards compatibility of devices. To reduce the attack surface ensure that only secure ciphers are used."}, "KAT-TLS-1.0-AND-1.1-SUPPORT": {"description": "The server supports TLS version 1.0 and 1,1.This is a protocol that encrypts data traffic by using a legacy protocol and encryption ciphers which contains various security vulnerabilities.", "source": "https://github.com/ssllabs/research/wiki/SSL-and-TLS-Deployment-Best-Practices#22-use-secure-protocols", "risk": "medium", "impact": "Attackers, who can perform a man-in-the-middle attack, can decrypt traffic. Thus no integrity or confidentiality is offered to between the client and server.", "recommendation": "Disable support for TLS version 1.0 and 1.1, unless it is required for backwards compatibility of devices. To reduce the attack surface ensure that only secure ciphers are used."}, "KAT-NO-TLS-1.2": {"description": "TLS version 1.2 is not supported. This is a current and recommended protocol that securely encrypts data traffic.", "source": "https://github.com/ssllabs/research/wiki/SSL-and-TLS-Deployment-Best-Practices#22-use-secure-protocols", "risk": "low", "impact": "Lack of support for TLS 1.2 could impact backwards compatibility for older devices. Ensure that only safe ciphers are used for encryption.", "recommendation": "Enable support for TLS version 1.2."}, "KAT-NO-TLS-1.3": {"description": "TLS version 1.3 is not supported. This is a current and recommended protocol that securely encrypts data traffic.", "source": "https://github.com/ssllabs/research/wiki/SSL-and-TLS-Deployment-Best-Practices#22-use-secure-protocols", "risk": "medium", "impact": "TLS 1.3 must be enabled to support the newest devices and in order to offer Perfect Forward Secrecy.", "recommendation": "Enable support for TLS version 1.3."}, "KAT-NO-TLS-FALLBACK-SCSV": {"description": "The encrypted connection provides no protection against downgrade attacks.", "source": "https://www.rfc-editor.org/rfc/rfc7507", "risk": "low", "impact": "An attacker, who can perform a man-in-the-middle attack, can weaken the session between the client and server. This could result in loss of confidentiality and integrity of data. ", "recommendation": "Implement TLS_FALLBACK_SCSV."}, "KAT-OPEN-SYSADMIN-PORT": {"description": "A known system administration port is open.", "source": "https://en.wikipedia.org/wiki/List_of_TCP_and_UDP_port_numbers", "risk": "medium", "impact": "System administrator ports should only be reachable from safe and known locations to reduce attack surface.", "recommendation": "Determine if this port should be reachable from the identified location. Limit access to reduce the attack surface if necessary."}, "KAT-OPEN-DATABASE-PORT": {"description": "A database port is open.", "source": "https://en.wikipedia.org/wiki/List_of_TCP_and_UDP_port_numbers", "risk": "high", "impact": "Databases should never be reachable from the internet, but only from secured internal networks. This will reduce unauthorized access.", "recommendation": "Determine if this port should be reachable from the identified location. Limit access to reduce the attack surface if necessary. "}, "KAT-UNCOMMON-OPEN-PORT": {"description": "The firewall may be configured in a riskful manner.", "source": "https://en.wikipedia.org/wiki/List_of_TCP_and_UDP_port_numbers", "risk": "medium", "impact": "Uncommon ports are sometimes overlooked and may become unwanted entry points for attackers into an organisations network.", "recommendation": "Manually validate whether this port should be open."}, "KAT-OPEN-COMMON-PORT": {"description": "A port commonly used was found to be open.", "source": "https://en.wikipedia.org/wiki/List_of_TCP_and_UDP_port_numbers", "risk": "recommendation", "impact": "Depending on the port there may or may not be impact. ", "recommendation": "Manually validate whether this port should be open."}, "KAT-WEBSERVER-NO-IPV6": {"description": "For this website there is no web server with an IPv6 address available.", "source": "https://www.internetsociety.org/deploy360/ipv6/", "risk": "low", "impact": "Users that only have IPv6 support cannot access your server.", "recommendation": "Add an IPv6 address for at least one web server that has no IPv6 address yet."}, "KAT-NAMESERVER-IPV6-NOT-REACHABLE": {"description": "One or more name servers is not reachable on an IPv6 address.", "source": "https://www.internetsociety.org/deploy360/ipv6/", "risk": "low", "impact": "Users that only have IPv6 support cannot access your server.", "recommendation": "Check IPv6 addresses for all name servers."}, "KAT-WEBSERVER-IPV6-NOT-REACHABLE": {"description": "OpenKAT checks if all web server that have an AAAA record with IPv6 address are reachable over IPv6. In this case the web server(s) is/are not reachable via IPv6.", "source": "https://www.internetsociety.org/deploy360/ipv6/", "risk": "low", "impact": "Users that only have IPv6 support cannot access your server.", "recommendation": "Configure IPv6 addresses for the web servers"}, "KAT-NOT-ENOUGH-IPV6-NAMESERVERS": {"description": "OpenKAT tests all IPv6 addresses received from your name servers. For this website there are not enough name servers accessible via IPv6.", "source": "https://www.internetsociety.org/deploy360/ipv6/", "risk": "medium", "impact": "Some users may not be able to reach your website without IPv6 support.", "recommendation": "Add an IPv6 address for at least two name servers that have no IPv6 address yet."}, "KAT-NO-DNSSEC": {"description": "The provided domain does not have DNSSEC enabled.", "source": "https://www.dns-school.org/Documentation/dnssec_howto.pdf", "risk": "medium", "impact": "DNS requests are not authenticated, thus there is no protection against DNS poisoning or manipulation.", "recommendation": "Enable DNSSEC on your name servers."}, "KAT-INVALID-DNSSEC": {"description": "The provided domain is DNSSEC signed, but the DNSSEC config is invalid.", "source": "https://www.dns-school.org/Documentation/dnssec_howto.pdf", "risk": "medium", "impact": "Invalid DNS requests may not provide the wanted protection against DNS poisoning or manipulation attacks.", "recommendation": "Reconfigure DNSSEC on your name servers."}, "KAT-HSTS-VULNERABILITIES": {"description": "List of vulnerabilities found in the HTTP strict transport security (HSTS) settings of the http header.", "source": "https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Strict-Transport-Security", "risk": "medium", "impact": "Impact depends on the HSTS misconfiguration. Determine what is wrong.", "recommendation": "Adjust the HSTS header to make it as strict as possible in order to reduce the attack surface."}, "KAT-CSP-VULNERABILITIES": {"description": "List of vulnerabilities found in the content security policy (CSP) settings of the http header.", "source": "https://developer.mozilla.org/en-US/docs/Web/HTTP/CSP", "risk": "medium", "impact": "A too naive CSP policy allows attackers to perform attacks from/to external locations.", "recommendation": "Adjust the CSP header to make it as strict as possible in order to reduce the attack surface."}, "KAT-POTENTIAL-MALWARE": {"description": "This software is known to be used as malware.", "source": "https://learn.microsoft.com/en-us/microsoft-365/security/defender/criteria?view=o365-worldwide", "risk": "high", "impact": "Depending on the malware this could be anything from eavesdropping to a ransomware attack.", "recommendation": "Alert your CERT to verify if your systems are compromised."}, "KAT-EXPOSED-SOFTWARE": {"description": "This software is often used for administrative purposes and should not be exposed to the Internet. This could be an easy entry point into your organisations environment.", "source": "Check your OpenKAT install on what software was identified.", "risk": "critical", "impact": "Impact depends on the identified software.", "recommendation": "Move the software to a more secure location and/or make it only accessible through a VPN. "}, "KAT-VERIFIED-VULNERABILITY": {"description": "A verified vulnerability is found by BinaryEdge.", "source": "Check your OpenKAT install on what software was identified.", "risk": "critical", "impact": "Impact depends on the identified software.", "recommendation": "Inspect the software version, determine if additional measures need to be taken and install updates to reduce the attack surface."}, "KAT-DICOM-EXPOSED": {"description": "A Dicom server is exposed.", "source": "https://en.wikipedia.org/wiki/DICOM", "risk": "critical", "impact": "Impact depends on segmentation and where the server is reachable from.", "recommendation": "Validate whether this server should actually be exposed."}, "KAT-10-OR-MORE-NEW-PORTS-OPEN": {"description": "A lot of ports are open which were not open a week ago.", "source": "Check your OpenKAT install on which ports are identified.", "risk": "critical", "impact": "Impact depends on what was identified.", "recommendation": "Validate if the firewall config is correct."}, "KAT-LEAKIX-CRITICAL": {"description": "A leak with severity critical has been found.", "source": "https://leakix.net/", "risk": "critical", "impact": "Impact depends on what was identified.", "recommendation": "Validate if this service is configured correctly, up-to-date and exposed on the correct port."}, "KAT-LEAKIX-HIGH": {"description": "A leak with severity high has been found.", "source": "https://leakix.net/", "risk": "high", "impact": "Impact depends on what was identified.", "recommendation": "Validate if this service is configured correctly, up-to-date and exposed on the correct port."}, "KAT-LEAKIX-MEDIUM": {"description": "A leak with severity medium has been found.", "source": "https://leakix.net/", "risk": "medium", "impact": "Impact depends on what was identified.", "recommendation": "Validate if this service is configured correctly, up-to-date and exposed on the correct port."}, "KAT-LEAKIX-LOW": {"description": "A leak with severity low has been found.", "source": "https://leakix.net/", "risk": "low", "impact": "Impact depends on what was identified.", "recommendation": "Validate if this service is configured correctly, up-to-date and exposed on the correct port."}, "KAT-LEAKIX-RECOMMENDATION": {"description": "A leak with severity information has been found.", "source": "https://leakix.net/", "risk": "recommendation", "impact": "Impact depends on what was identified.", "recommendation": "Validate if this service is configured correctly, up-to-date and exposed on the correct port."}, "KAT-SOFTWARE-UPDATE-AVAILABLE": {"description": "There is a newer version for this software.", "source": "Check your OpenKAT install on what software was identified.", "risk": "recommendation", "impact": "Impact depends on what was identified.", "recommendation": "Install the updates for the software to reduce potential attack vectors."}, "KAT-NO-GREEN-HOSTING": {"description": "According to the Green Web Foundation, this website is not hosted in a 'green' way.", "source": "https://www.thegreenwebfoundation.org/", "risk": "recommendation", "impact": "No security impact, only environmental.", "recommendation": "Change hosting providers if you wish to host 'green'."}, "KAT-NXDOMAIN": {"description": "The domain name does not exist.", "source": "https://datatracker.ietf.org/doc/html/rfc8020", "risk": "medium", "impact": "If this is a critical service, it stopped working. Fix it now. :).", "recommendation": "Try again later, or verify if the host should be reachable."}, "KAT-NXDOMAIN-HEADER": {"description": "The hostname in this header does not exist.", "source": "https://datatracker.ietf.org/doc/html/rfc8020", "risk": "critical", "impact": "If this is a critical service, it stopped working. Fix it now. :).", "recommendation": "Check if the hostname in the header is correct and update accordingly."}, "KAT-NAMESERVER-NO-TWO-IPV6": {"description": "This webserver does not have at least two nameservers with an ipv6 address.", "source": "https://www.rfc-editor.org/rfc/rfc3901.txt", "risk": "low", "impact": "Some users may not be able to reach your website without IPv6 support.", "recommendation": "Ensure that both nameservers have an reachable ipv6 address for higher availability."}, "KAT-NAMESERVER-NO-IPV6": {"description": "This nameserver does not have an ipv6 address.", "source": "https://www.rfc-editor.org/rfc/rfc3901.txt", "risk": "recommendation", "impact": "Some users may not be able to reach your website without IPv6 support.", "recommendation": "Ensure that the nameserver has an ipv6 address that can be reached."}, "KAT-INTERNETNL": {"description": "This website does not comply to the internet.nl standards. Currently, we check the following standards: IPv6 on webservers and nameservers, DNSSEC, missing and mis-configured headers.", "source": "https://internet.nl/faqs/", "risk": "medium", "impact": "You may not be in sync with your organisations policy if this is a requirement.", "recommendation": "Make changes to your web server in order to comply with the Internet.nl standards."}, "KAT-NO-CERTIFICATE": {"description": "This website does not have an SSL certificate.", "source": "https://datatracker.ietf.org/doc/html/rfc5280", "risk": "medium", "impact": "Attackers, who can perform a man-in-the-middle attack, can read all your traffic.", "recommendation": "Generate an SSL certificate for this web server to offer confidentiality and integrity to users."}, "KAT-SSL-CERT-HOSTNAME-MISMATCH": {"description": "The alternative name of the certificate does not match with the hostname of the website", "source": "https://datatracker.ietf.org/doc/html/rfc5280#section-4.1.2.6", "risk": "high", "impact": "A properly configured client cannot connect to your server.", "recommendation": "Update your certificates such that the alternative names include this hostname."}, "KAT-NO-HTTPS-REDIRECT": {"description": "This HTTP URL may not redirect to HTTPS; 'Location' was not found in HTTPHeader.", "source": "https://datatracker.ietf.org/doc/html/rfc7231#section-6.4", "risk": "low", "impact": "Users may not connect over a secured connection to your server.", "recommendation": "Check if redirection is setup properly."}, "KAT-CERTIFICATE-EXPIRING-SOON": {"description": "TLS certificate is expiring soon", "source": "https://datatracker.ietf.org/doc/html/rfc5280#section-4.1.2.5", "risk": "medium", "impact": "Expired certificates could result in compromise of confidentiality and integrity of clients that connect to the service.", "recommendation": "Update the certificate to expire on a date further in the future."}, "KAT-CERTIFICATE-EXPIRED": {"description": "TLS certificate has expired", "source": "https://datatracker.ietf.org/doc/html/rfc5280#section-4.1.2.5", "risk": "critical", "impact": "Expired certificates could result in compromise of confidentiality and integrity of clients that connect to the service.", "recommendation": "Replace the certificate with a valid one."}, "KAT-NO-DMARC": {"description": "This hostname does not have a DMARC record.", "source": "https://www.cloudflare.com/en-gb/learning/dns/dns-records/dns-dmarc-record/", "risk": "medium", "impact": "E-mail from this domain can potentially be spoofed if DMARC is not (properly) implemented in combination with DKIM and SPF.", "recommendation": "Set a DMARC record to protect your domain."}, "KAT-NO-DKIM": {"description": "This hostname does not support a DKIM record.", "source": "https://www.cloudflare.com/en-gb/learning/dns/dns-records/dns-dkim-record/", "risk": "medium", "impact": "E-mail from this domain can potentially be spoofed if DMARC is not (properly) implemented in combination with DKIM and SPF.", "recommendation": "Set a DKIM record to protect your domain."}, "KAT-NO-SECURITY-TXT": {"description": "This hostname does not have a Security.txt file.", "source": "https://securitytxt.org/", "risk": "recommendation", "impact": "Security researchers and/or bounty hunter may not be able to properly disclose vulnerabilities for your website.", "recommendation": "Make sure there is a security.txt available."}, "KAT-INVALID-SECURITY-TXT": {"description": "Required elements of the security.txt are missing.", "source": "https://securitytxt.org/", "risk": "recommendation", "impact": "Security researchers and/or bounty hunter may not be able to properly disclose vulnerabilities for your website.", "recommendation": "Make sure the security.txt is in line with the requirements."}, "KAT-BAD-FORMAT-SECURITY-TXT": {"description": "There are flaws in the format of the security.txt.", "source": "https://www.rfc-editor.org/rfc/rfc9116.html#section-4", "risk": "recommendation", "impact": "Security researchers and/or bounty hunter may not be able to properly disclose vulnerabilities for your website.", "recommendation": "Make sure the security.txt is correctly formatted."}, "KAT-NO-SPF": {"description": "This hostname does not have an SPF record.", "source": "https://www.cloudflare.com/en-gb/learning/dns/dns-records/dns-spf-record/", "risk": "medium", "impact": "E-mail from this domain can potentially be spoofed if DMARC is not (properly) implemented in combination with DKIM and SPF.", "recommendation": "Set an SPF record to protect your domain."}, "KAT-INVALID-SPF": {"description": "This SPF record could not be parsed by the internet.nl SPF parser and is therefore deemed invalid.", "source": "https://www.cloudflare.com/en-gb/learning/dns/dns-records/dns-spf-record/", "risk": "low", "impact": "E-mail from this domain can potentially be spoofed if DMARC is not (properly) implemented in combination with DKIM and SPF.", "recommendation": "Fix the syntax of the SPF record."}, "SUB-DOMAIN-TAKEOVER": {"description": "Subdomain takeover is when an attacker takes control of an unused or improperly configured subdomain, potentially accessing sensitive information or conducting phishing attacks.", "source": "https://developer.mozilla.org/en-US/docs/Web/Security/Subdomain_takeovers", "risk": "high", "impact": "An attacker using your hosting provider may setup a virtual host for your domain and thus intercept and trick users.", "recommendation": "To prevent subdomain takeover, organizations should regularly monitor their DNS records to identify and remove any unused subdomains. Additionally, they should ensure that all subdomains are properly configured and point to valid services."}, "EXPOSED-PANELS": {"description": "Exposed login panels for services can pose security risks as they can be targeted by malicious actors for brute-force attacks, phishing attempts, and other forms of unauthorized access.", "source": "https://resources.infosecinstitute.com/topics/application-security/dangers-web-management/", "risk": "recommendation", "impact": "Administrative interfaces may be easy ways for an attacker to gain access to your network.", "recommendation": "Ideally to minimize the attack surface as much as possible these panels should not be directly exposed to the internet."}, "KAT-CRITICAL-BAD-CIPHER": {"description": "Ciphers are used that are labeled as bad. These should not be used anymore", "source": "https://wiki.mozilla.org/Security/Server_Side_TLS", "risk": "critical", "impact": "Weak or insecure ciphers may result in loss of confidentiality and integrity of data through decryption.", "recommendation": "It is recommended to only use ciphers labelled as 'good'. Check https://cipherlist.eu/ for safe ciphers."}, "KAT-MEDIUM-BAD-CIPHER": {"description": "Ciphers are used that are labeled as bad. These should not be used anymore", "source": "https://wiki.mozilla.org/Security/Server_Side_TLS", "risk": "medium", "impact": "Weak or insecure ciphers may result in loss of confidentiality and integrity of data through decryption.", "recommendation": "It is recommended to only use ciphers labelled as 'good'. Check https://cipherlist.eu/ for safe ciphers."}, "KAT-RECOMMENDATION-BAD-CIPHER": {"description": "Ciphers are used that are labeled as bad. These should not be used anymore", "source": "https://wiki.mozilla.org/Security/Server_Side_TLS", "risk": "recommendation", "impact": "Weak or insecure ciphers may result in loss of confidentiality and integrity of data through decryption.", "recommendation": "It is recommended to only use ciphers labelled as 'good'. Check https://cipherlist.eu/ for safe ciphers."}, "KAT-NO-RPKI": {"description": "The IP address does not have a route announcement that is matched by the published Route Policy and Authorization (RPKI)", "source": "https://blog.cloudflare.com/rpki/", "risk": "low", "impact": "Without RPKI validation, your servers might be more vulnerable to unintended or malicious routing configuration errors, potentially leading to inaccessibility of your servers or interception of internet traffic directed to them.", "recommendation": "Work on implementing RPKI for your IP addresses. This may involve creating Route Origin Authorizations (ROAs) that specify which Autonomous Systems (AS) are authorized to announce your IP addresses."}, "KAT-EXPIRED-RPKI": {"description": "The route announcement that is matched by the published Route Policy and Authorization (RPKI) is expired", "source": "https://blog.cloudflare.com/rpki/", "risk": "low", "impact": "Without RPKI validation, your servers might be more vulnerable to unintended or malicious routing configuration errors, potentially leading to inaccessibility of your servers or interception of internet traffic directed to them.", "recommendation": "Make sure that the Route Origin Authorizations (ROAs) that specify which Autonomous Systems (AS) are authorized to announce your IP addresses are valid and not expired."}, "KAT-NO-CAA": {"description": "This zone does not carry at least one CAA record.", "source": "https://letsencrypt.org/docs/caa/", "risk": "low", "impact": "All Certificate Authorities may issue certificates for you domain.", "recommendation": "Set a CAA record to limit which CA's are allowed to issue certs."}, "KAT-DISALLOWED-DOMAIN-IN-CSP": {"description": "This CSP header contains domains that are not allowed, If the website contains a cross-site scripting vulnerability, then JavaScript code can be injected into the web page hosted on these domains which can host files for anyone.", "risk": "medium", "impact": "Disallowed domains are domains that are for example 'world writable', this opens up the possibility for an atacker to host malicious files on a csp whitelisted domain.", "recommendation": "Remove the offending hostname from the CSP header."}}

{"id": "6d99ee5a-b014-4d5b-b3e8-40f44d462283", "boefje_meta": {"id": "38bffcf0-9721-4377-b14f-2ac55b19d8d9", "started_at": "2024-04-22T14:45:20.547900Z", "ended_at": "2024-04-22T14:45:20.548539Z", "boefje": {"id": "kat-finding-types", "version": null}, "input_ooi": "KATFindingType|KAT-OPEN-DATABASE-PORT", "arguments": {"input": {"object_type": "KATFindingType", "scan_profile": "scan_profile_type='empty' reference=Reference('KATFindingType|KAT-OPEN-DATABASE-PORT') level=<ScanLevel.L0: 0>", "primary_key": "KATFindingType|KAT-OPEN-DATABASE-PORT", "id": "KAT-OPEN-DATABASE-PORT", "description": "None", "source": "None", "impact": "None", "recommendation": "None", "risk_score": 0.0, "risk_severity": "pending"}}, "organization": "aa", "runnable_hash": "f7138cee9f723fef7ff7ad4d26cb0dbb66cf8b44dd1aa98be40f58434104b182", "environment": {}}, "mime_types": [{"value": "boefje/kat-finding-types"}], "secure_hash": "sha512:d16c4208e7f373445da5bbc248647cfadcab3877149e865bcf852c7c5909ab9f60e682284ca2c641c16e8d1e11c093c95821be1661b6a311bf95e8828d9c98fc", "signing_provider_url": null, "hash_retrieval_link": "2d6ba845-4c97-4780-afc7-eef151e89def"}

[Donnype]

See the branch fix/disappearing-ports for an integration test confirming the bug. It is due to the origin.id being the unique identifier on which we apply the "dereferencing oois that were not found again". But the origin.id only uses the source field (input_ooi) and the method field, but this is only the normalizer_id. As a consequence, the fact that both tcp and udp ports are found by kat_nmap_normalize, we delete all the tcp ports after running the udp scan. Hence we have to add the boefje_id to the origin_id and perform a proper migration.