Failed tests cause of missing vulnerabilities in Vulnerability Detection E2E tests

[joaquinsgi]

Target version	Related issue	Related PR/dev branch
4.9.0 RC 1	https://github.com/wazuh/wazuh/issues/25080	4.9.0 beta 1

Description

Analyzing the build: https://ci.wazuh.info/job/Test_e2e_system/342/ of the issue: https://github.com/wazuh/wazuh/issues/25080, an error has been found in test_install_vulnerable_package_when_agent_down[install_package], which appears to be too too many missing vulnerabilities, this behavior has been observed in: https://github.com/wazuh/wazuh/issues/24809 , but not with so many vulnerabilities. Some of them seem to be repeated.

We can se in the log of the report:

AssertionError: 
E         Test test_install_vulnerable_package_when_agent_down[install_package] failed
E         
E         Check no_errors succeeded
E         Check operation_successfull_for_all_agents succeeded
E         Check expected_vulnerabilities_found_in_index failed. Evidences (['missing_vulnerabilities']) can be found in the report.
E         Check no_unexpected_vulnerabilities_found_in_index succeeded
E         Check no_duplicated_vulnerabilities succeeded

In missing_vulnerabilities

{
    "agent1": [
        [
            "CVE-2022-23498",
            "grafana",
            "8.5.5-1",
            "x86_64"
        ],
        [
            "CVE-2022-23552",
            "grafana",
            "8.5.5-1",
            "x86_64"
        ],
        [
            "CVE-2022-31097",
            "grafana",
            "8.5.5-1",
            "x86_64"
        ],
        [
            "CVE-2022-31107",
            "grafana",
            "8.5.5-1",
            "x86_64"
        ],
        .
        .
        .

Proposed checks

• [ ] Investigate why those vulnerabilities are missing.
• [ ] Investigate why there are so many missing vulnerabilities.
• [ ] Investigate if some of them are repeated.

Configuration and considerations

Full report: Test_e2e_system_342_test_vulnerability_detector.zip

Expected results

Have no missed vulnerabilities.

Related

https://github.com/wazuh/wazuh/issues/25080 https://github.com/wazuh/wazuh/issues/24809

[jnasselle]

Update

We presume that this issue is related to the following analysis https://github.com/wazuh/wazuh-qa/issues/5655#issuecomment-2292544200

[MARCOSD4]

Update

An analysis of the report has been performed and it has been found that the CentOS 7 agents are generating the alerts correctly but the test is not finding the vulnerabilities in the index. More information about the analysis can be found in this comment. In addition, a test has been performed locally to check that the vulnerabilities are being stored correctly in the index. The test was successful and the vulnerabilities are indexed correctly for CentOS agents. Further research is required.

[MARCOSD4]

Conclusion

After analyzing the tests performed locally, the cause of this problem could not be identified. The queries made to the index to obtain the vulnerability information and the filtering of that information by the test have been reviewed, but all of that seems to work correctly. In addition, it has been verified that the vulnerability information for 4.8.1 is the same as for 4.9.0, so they should be detected and filtered in the same way.

The cause of this problem seems to be related to this other issue, which is also being investigated so that we can close this issue in favor of the other one.

[Rebits]

Closing. Research will be carried on in https://github.com/wazuh/wazuh-qa/issues/5665