search

wazuh / wazuh

Wazuh - The Open Source Security Platform. Unified XDR and SIEM protection for endpoints and cloud workloads.
https://wazuh.com/
Other
9.34k stars 1.48k forks source link

Vulnerability Detection is not working as expected in Demo environment #23322

Closed Rebits closed 1 week ago

Rebits commented 1 week ago

Test information
Main release stage issue https://github.com/wazuh/wazuh/issues/23246
Version 4.8.0
Release candidate RC 1
Tag https://github.com/wazuh/wazuh-qa/tree/v4.8.0-rc1

Description

This issue addresses concerns regarding vulnerability detection in the Demo environment. It has been observed that vulnerability detection is not functioning correctly for Debian agents, and there are unexpected behaviors when toggling the vulnerability detection feature. Further investigation is necessary to identify the root causes of these issues.

Problem Statement

Test Requirements

Environment

Access to the Demo environment is necessary. Credentials can be obtained from the devops team .

Configuration

The environment should have enabled the wazuh-modulesd debug mode. This will provide more detailed logs for analysis.

Tests Cases

T1: Basic Vulnerability Detection

Description: Verify if vulnerability detection triggers vulnerabilities for all agents of the Demo Environment, having the vulnerability detection module enabled when agents were registered. Initial conditions: Initially, vulnerability detection is enabled, and all agents are registered while this module remains active.

ID Description Status
T1.1 Basic vulnerability detection for Debian agent :green_circle:
T1.2 Basic vulnerability detection for RHEL agent :green_circle:
T1.3 Basic vulnerability detection for centOS agent :green_circle:
T1.4 Basic vulnerability detection for Windows Server agent :green_circle:

T2: Assess Vulnerability Detection Re-Scan Behavior

Description: Verify if re-scan mechanism works as expected when a vulnerability detection is enabled once agents have been already registered.

Initial conditions: At the outset, vulnerability detection remains disabled, and all agents are registered while this module remains in its deactivated state.

ID Description Status
T2.1 Vulnerability detection for Debian agent when re-scan is triggered :red_circle:
T2.2 Vulnerability detection for RHEL agent when re-scan is triggered :red_circle:

Conclusion :red_circle:

Multiple issues were detected in the demo environment

Rebits commented 1 week ago

T1: Basic Vulnerability Detection

ID Description Status
T1.1 Basic vulnerability detection for Debian agent :green_circle:
T1.2 Basic vulnerability detection for RHEL agent :green_circle:
T1.3 Basic vulnerability detection for centOS agent :green_circle:
T1.4 Basic vulnerability detection for Windows Server agent :green_circle:

Description

Initially, the Demo environment comprised a RHEL and a Debian agent. Vulnerability scans on these agents did not detect any vulnerabilities. However, the remaining agents exhibited triggered vulnerabilities.

image

[!NOTE] No vulnerabilities were detected for RHEL agent

Upon assessing the current status of the manager, it was observed that removing and re-registering the agents resolved the issue.

image

It's likely that the reported errors are specifically related to the re-scan mechanism. This will be further investigated in the T2 cases.

After a few hours, after restraint manager and worker several times for other tests cases, vulnerabilities were detected for RHEL but not for Debian agent:

image

Rebits commented 1 week ago

T2: Assess Vulnerability Detection Re-Scan Behavior :red_circle:

ID Description Status
T2.1 Vulnerability detection for Debian agent when re-scan is triggered :red_circle:
T2.2 Vulnerability detection for RHEL agent when re-scan is triggered :red_circle:

Description

In an environment where all agents are linked and Vulnerability Detection is actively running, our next steps involve disabling the Vulnerability Detection module. We'll initiate this process via the dashboard, starting with the worker nodes and then proceeding to the master node.

Subsequently, we'll undertake the crucial task of backing up agents's client keys and then removing them from both Debian and RHEL agents. Before removal, we'll ensure to rename them within the configuration settings as 'RHEL-4' and 'Debian-4' respectively.

Once the client keys are secured and renamed, we'll proceed to register the agents in the environment with the Vulnerability Detection disabled. Following this, as the agents reconnect, we'll re-enable the Vulnerability Detection module seamlessly

Results

After enabling the vulnerability detector module again, after 1:30 hours, vulnerabilities of new agents appear:

Debian

debian4

RHEL

rhel4

However several issues were detected:

Rebits commented 1 week ago

No existing agent's vulnerabilities found in index :red_circle:

Description

After registering new agents in the environment, we observed the following list of agents registered:

image

However, upon removing the default wazuh.cluster.name filter, vulnerabilities were detected for a non-existing Ubuntu agent with wazuh.cluster.name value set to 2:

image

The exact registration time of this old agent cannot be provided due to the previous use of a Demo instance. However, it has been more than 1 day since registration.

Steps to Reproduce

No steps to reproduce can be provided for now due this agent was already present in the environment

Expected Behavior

No vulnerabilities should be detected for non-existing agents.

Rebits commented 1 week ago

Vulnerabilities of some agents disappear suddely :red_circle:

[!NOTE] After researching this issue, it seems directly related to https://github.com/wazuh/wazuh/issues/23322#issuecomment-2101089209

Description

No vulnerabilities were detected in Debian and RHEL9 agents during testing. These agents were initially deployed in the environment and replicating the same conditions with new agents proved unsuccessful, suggesting a unique setup at the environment's inception.

Various scenarios were tested, including:

Despite these efforts, the absence of vulnerabilities remains consistent, hinting at specific conditions present at the environment's outset

After toggling the vulnerability detection module on both the master and worker nodes, please allow approximately 30 minutes for the expected vulnerabilities to manifest.

IMAGE

Few minutes later, vulnerabilities were automatically deleted image

Steps to Reproduce

No steps to reproduce can be provided for now due this agent was already present in the environment

Expected behaviour

Vulnerability scan should detect vulnerabilities for all agents without the need of disabling and enabling vulnerability detection module.

Rebits commented 1 week ago

Index unstable when vulnerability detection module is disabled and enabled again :red_circle:

Description

The vulnerabilities index seems to be unstable after enabling and disabling the vulnerability detection module in master and worker nodes. Vulnerabilities of the environment are being removed completely and generated again several times, leading into some agents without any vulnerability (Check https://github.com/wazuh/wazuh/issues/23322#issuecomment-2100941195 for more information). In this case the agent 013, created during the test end with no vulnerabilities detected:

image

Video

vuln_dissapear.webm

Steps to reproduce

Expected behavior

Vulnerability index should be stable in case of enabling and disabling the vulnerability detection module. It is expected the removal of the index, although this should be produced only one time, generating vulnerabilities for all the agents.

Rebits commented 1 week ago

Inconsistent wazuh.cluster.name filter :red_circle:

Description

The default setting for the vulnerability detection dashboard incorporates a filter labeled wazuh.cluster.name set to the value "wazuh1". Initially, it appears this filter is not intended for modification. However, it is possible to deactivate this filter and apply negation by accessing the "Change all filters" menu.

image

Video

filter_inconsistent.webm

Steps to reproduce

Rebits commented 1 week ago

Debian agent includes Windows agent syscheck events :red_circle:

Description

Difficult to replicate error appear in the Debian endpoint menu. In the FIM recent events menu appear some Windows agent events

debian_bad_dashboard

Steps to reproduce

No steps to reproduce can be provided because this issue was impossible to reproduce

santipadilla commented 1 week ago

LGTM

juliamagan commented 1 week ago

LGTM