wazuh / wazuh

Wazuh - The Open Source Security Platform. Unified XDR and SIEM protection for endpoints and cloud workloads.
https://wazuh.com/
Other
10.98k stars 1.67k forks source link

The new vulnerability detector does not produce any events, resulting in no historical data #24290

Open misje opened 4 months ago

misje commented 4 months ago
Wazuh version Component Install type Install method Platform
4.8.0 ? Manager docker

In the old vulnerability detector, the alert rules 23502–23507 were created when vulnerabilities were found, and later resolved. This provided important historical information. For instance, in an investigation, the presence of a vulnerability in a given time frame is very useful along with other events.

In the new module, vulnerability state information is kept in a new index, and as far as I can see, documents are removed when vulnerabilities are resolved. Once a vulnerability is resolved, there is no track of it ever being present in the system, unlike in earlier versions of Wazuh. There are no longer any alerts created in wazuh-alerts-* for 4.8.0 agents. Is this intentional? After all, the new Vulnerability Detection interface still has an "Events" tab, which will no longer have any use.

I hope to see vulnerability events back in the alert index. I would prefer them as they were, with all of the CVE context.

Dwordcito commented 4 months ago

Hello @misje , could you tell me if the alert.json file has the missing alerts? only vulnerability detector does not have alerts?

Keep in mind that if you force restarts of your agent so that the inventory information is refreshed, by design we will not alert you.

Wait the time after the fix of that update or lower the time in the configuration of your agents for syscollector.

@sebasfalcone take a look on that.

misje commented 4 months ago

There are alerts in alerts.json. As far as I know, only the vulnerability detector no longer has alerts. Alerts are produced for older clients, but not for newer clients. This is easy to reproduce by running wazuh-client 4.8.0 in a docker image (I tested ubuntu:22.04). Vulnerabilities are detected, but no events are created for when vulnerabilities are detected, nor when they are resolved. Should not this test (running a new agent) produce any events?

branchnetconsulting commented 4 months ago

In reference to:

Keep in mind that if you force restarts of your agent so that the inventory information is refreshed, by design we will not alert you.

Does this mean that any time a Wazuh agent restarts between the time that that system's software inventory has changed in a vulnerability-impacting way and the next scheduled syscollector self-inventory, that such changes to the vulnerability state will never be accounted for with a vulnerability event? Even if I:

Those would be very common scenarios we really can't afford to skip making events for if we are to trust the new Wazuh vulnerability detection service.

DokorWolf commented 4 months ago

Hello, I had exactly the same issue after upgrading 4.7.5 to 4.8. So I Reinstalled everything on other computers directly in 4.8 and I compared ossec.conf files. I saw that in the reinstalled version the lines below were added at the very end of the file. So I added them, restarted wazuh-manager and all agents, and after some minutes I started receiving vulnerabilities.

syslog /var/ossec/logs/active-responses.log syslog /var/log/dpkg.log
misje commented 4 months ago

Are you having a different issue, perhaps? dpkg.log is present in all of my agents, including 4.8.0. Vulnerability detection works for all of them, old and new, but alerts are not produced for 4.8.0 agents. Alerts are OpenSearch documents with rule.id 23502–23507. That is this issue I'm having.

DokorWolf commented 4 months ago

Oh sorry for my misunderstanding, unfortunately I don't have them either. My apologies.

ricokritzer commented 4 months ago

Are you having a different issue, perhaps? dpkg.log is present in all of my agents, including 4.8.0. Vulnerability detection works for all of them, old and new, but alerts are not produced for 4.8.0 agents. Alerts are OpenSearch documents with rule.id 23502–23507. That is this issue I'm having.

I have the same issue: Vulnerabilities are only listed in index files wazuh-states-vulnerabilities-* but not in wazuh-alerts-*. For that reason, the "Events" tab has "no results match the search criteria".

It does not matter which of my clients (on version 4.8) is used. None of them create alerts (as they should, as described in the documentation: https://documentation.wazuh.com/current/user-manual/capabilities/vulnerability-detection/how-it-works.html#alert-generation).

ricokritzer commented 4 months ago

could you tell me if the alert.json file has the missing alerts?

@Dwordcito and @sebasfalcone - in my case, the alerts are missing completely: In alert.json and alert.log.

haiau8520 commented 3 months ago

Hi, I'm facing the same issue, there is no events for vulnerability detection But the vulnerabilities are still visible in inventory Do we have any solutions for this?

nbrys commented 3 months ago

Is there any update / plan for this issue? We relied on vulnerabilities being reported in alerts.log / alerts.json for integration with Jira

mabelisle commented 3 months ago

I've just installed wazuh-manager from scratch with 4.8.1 and same issue. Everything works including the inventory but 0 Events. This issue should be classified as "type/bug/regression".

image

branchnetconsulting commented 3 months ago

I've been receiving Vulnerability Detection events for some time on my Wazuh dev SIEM with versions 4.8.0 and 4.8.1, though there are multiple categories of Vulnerability Detection events that are presently excluded from being alerted on, apparently by design. As I understand, unlike with 4.7 and previous, the findings from the first-time Vulnerability Detection scans of new agents are now treated as non-alerting "baseline" VD findings. So when you first set up Wazuh agent on a system, the initial VD findings on that system will not generate VD events. They will just silently appear in the VD inventory. After that, subsequent changes to the VD state for a given agent, in addition to showing up in VD inventory, will usually result in VD events as well, but with at least one exception. If a vulnerable software package is installed on an agent and the agent is promptly rebooted thereafter, or if a vulnerability on an agent is resolved by upgrade or uninstallation of the affected package followed promptly by a reboot, though the appropriate change will be reflected in that agent's VD inventory, you won't see any VD events about it.

mabelisle commented 2 months ago

There must be something wrong with my installation or my configuration then. I've had no luck with the vulnerability detection. Nothing is working, it hasn't detected or created events on the last patch Tuesday either. I've went through the logs and there's nothing indicating that it's not working other than the obvious.

nalakawula commented 2 months ago

Are you having a different issue, perhaps? dpkg.log is present in all of my agents, including 4.8.0. Vulnerability detection works for all of them, old and new, but alerts are not produced for 4.8.0 agents. Alerts are OpenSearch documents with rule.id 23502–23507. That is this issue I'm having.

I have the same issue: Vulnerabilities are only listed in index files wazuh-states-vulnerabilities-* but not in wazuh-alerts-*. For that reason, the "Events" tab has "no results match the search criteria".

It does not matter which of my clients (on version 4.8) is used. None of them create alerts (as they should, as described in the documentation: https://documentation.wazuh.com/current/user-manual/capabilities/vulnerability-detection/how-it-works.html#alert-generation).

I have same exact issue with wazuh 4.5.2, the manager is run docker single node.

sn0b4ll commented 2 months ago

Can confirm the same issue with a fresh docker single node installation of wazuh 4.9.0 as well as an "native" deployment done by a customer.

Since currently it is not possible to fully ignore CVEs in the inventory, this breaks the vuln-mgmt use case for us :/

iacob28 commented 1 month ago

Hello everyone!

Any updates about this issue ?

hmoreau commented 1 month ago

I had the same issue after upgrading to 4.9.0

Setup : Proxmox with Wazuh in CT

In /var/ossec/logs/ossec.log I had : wazuh-remoted: ERROR: Could not set resource limit for file descriptors to 458752: Operation not permitted (1)

Solved here : https://github.com/wazuh/wazuh/issues/11377

iacob28 commented 1 month ago

In my ossec.log I found this ERROR regarding the Vulnerabilit Scanner. Could this be the issue ?

ERROR: VulnerabilityScannerFacade::start: Failed to open RocksDB database. Reason: While opening a file for sequentially reading: queue/vd/event/MANIFEST-000005: No such file or directory.

0xr00tx commented 1 month ago

I've upgraded to 4.9.0 and it did not work, I've removed all wazuh, apt remove wazuh-* and installed all from scratch. Have the same issue.

Agents are connecting and I see them on list, SCA report in available but there is no events.

There are no WARN/ERROR in wazuh-agent log, same for dashboard, indexer, manager in /var/log/wazuh-indexer/wazuh-cluster.log there is cert exception which is coming from I dunno where (probably performance analyzer of opensearch)

[2024-09-26T15:29:40,203][WARN ][o.o.h.AbstractHttpServerTransport] [node-1] caught exception while handling client http traffic, closing connection Netty4HttpChannel{localAddress=/127.0.0.1:9200, remoteAddress=/127.0.0.1:49694} io.netty.handler.codec.DecoderException: javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate

ports are random in remoteAddress image

Agent is active and SCA has been submitted when restarting wazuh-agent Same situation is for agents v4.9.0 and v4.4.3

Agent log from connection attempt when restarting

2024/09/26 15:41:45 wazuh-agentd: INFO: (1410): Reading authentication keys file.
2024/09/26 15:41:45 wazuh-agentd: INFO: Using notify time: 10 and max time to reconnect: 60
2024/09/26 15:41:45 wazuh-agentd: INFO: Version detected -> Linux |redacted |5.10.0-21-amd64 |#1 SMP Debian 5.10.162-1 (2023-01-21) |x86_64 [Debian GNU/Linux|debian: 11 (bullseye)] - Wazuh v4.9.0
2024/09/26 15:41:45 wazuh-agentd: INFO: Started (pid: 302109).
2024/09/26 15:41:45 wazuh-agentd: INFO: Using AES as encryption method.
2024/09/26 15:41:45 wazuh-agentd: INFO: Trying to connect to server ([redacted]:1514/tcp).
2024/09/26 15:41:45 wazuh-agentd: INFO: (4102): Connected to the server ([redacted]:1514/tcp).
2024/09/26 15:41:46 wazuh-syscheckd: INFO: Started (pid: 302123).

Any ideas?

EDIT: This is the log from registering agent that was previously connected to wazuh. I've restarted agent by service restart and it re-registered with wazuh server

2024/09/27 08:17:19 wazuh-modulesd[2471666] wait_op.c:117 at os_wait_primitive(): DEBUG: Process locked due to agent is offline. Waiting for connection...
2024/09/27 08:17:19 wazuh-agentd[2471624] start_agent.c:54 at connect_server(): INFO: Closing connection to server ([redacted]:1514/tcp).
2024/09/27 08:17:19 wazuh-agentd[2471624] start_agent.c:86 at connect_server(): INFO: Trying to connect to server ([redacted]:1514/tcp).
2024/09/27 08:17:19 wazuh-agentd[2471624] start_agent.c:291 at receive_message(): DEBUG: Connection socket: Success (0)
2024/09/27 08:17:29 wazuh-agentd[2471624] start_agent.c:54 at connect_server(): INFO: Closing connection to server ([redacted]:1514/tcp).
2024/09/27 08:17:29 wazuh-agentd[2471624] start_agent.c:86 at connect_server(): INFO: Trying to connect to server ([redacted]:1514/tcp).
2024/09/27 08:17:29 wazuh-agentd[2471624] start_agent.c:291 at receive_message(): DEBUG: Connection socket: Success (0)
2024/09/27 08:17:39 wazuh-agentd[2471624] start_agent.c:54 at connect_server(): INFO: Closing connection to server ([redacted]:1514/tcp).
2024/09/27 08:17:39 wazuh-agentd[2471624] start_agent.c:86 at connect_server(): INFO: Trying to connect to server ([redacted]:1514/tcp).
2024/09/27 08:17:39 wazuh-agentd[2471624] start_agent.c:291 at receive_message(): DEBUG: Connection socket: Success (0)
2024/09/27 08:17:49 wazuh-agentd[2471624] start_agent.c:54 at connect_server(): INFO: Closing connection to server ([redacted]:1514/tcp).
2024/09/27 08:17:49 wazuh-agentd[2471624] start_agent.c:86 at connect_server(): INFO: Trying to connect to server ([redacted]:1514/tcp).
2024/09/27 08:17:50 wazuh-agentd[2471624] start_agent.c:291 at receive_message(): DEBUG: Connection socket: Success (0)
2024/09/27 08:17:50 wazuh-agentd[2471624] enrollment_op.c:122 at w_enrollment_request_key(): INFO: Requesting a key from server: redacted
2024/09/27 08:17:50 wazuh-agentd[2471624] ssl.c:83 at os_ssl_keys(): DEBUG: Returning CTX for client.
2024/09/27 08:17:50 wazuh-agentd[2471624] enrollment_op.c:244 at w_enrollment_connect(): DEBUG: (1209): Connected to enrollment service at '[redacted]:1515'
2024/09/27 08:17:50 wazuh-agentd[2471624] enrollment_op.c:482 at w_enrollment_verify_ca_certificate(): DEBUG: Registering agent to unverified manager
2024/09/27 08:17:50 wazuh-agentd[2471624] enrollment_op.c:601 at w_enrollment_load_pass(): INFO: No authentication password provided
2024/09/27 08:17:50 wazuh-agentd[2471624] enrollment_op.c:273 at w_enrollment_send_message(): INFO: Using agent name as: test
2024/09/27 08:17:50 wazuh-agentd[2471624] enrollment_op.c:313 at w_enrollment_send_message(): DEBUG: Request sent to manager
2024/09/27 08:17:50 wazuh-agentd[2471624] enrollment_op.c:338 at w_enrollment_process_response(): INFO: Waiting for server reply
2024/09/27 08:17:50 wazuh-agentd[2471624] enrollment_op.c:459 at w_enrollment_process_agent_key(): INFO: Valid key received
2024/09/27 08:17:50 wazuh-agentd[2471624] enrollment_op.c:364 at w_enrollment_process_response(): DEBUG: Connection closed.
2024/09/27 08:17:50 wazuh-agentd[2471624] start_agent.c:304 at try_enroll_to_server(): INFO: Waiting 20 seconds before server connection
2024/09/27 08:18:10 wazuh-agentd[2471624] keys.c:432 at OS_UpdateKeys(): DEBUG: Reloading keys
2024/09/27 08:18:10 wazuh-agentd[2471624] keys.c:442 at OS_UpdateKeys(): INFO: (1410): Reading authentication keys file.
2024/09/27 08:18:10 wazuh-agentd[2471624] msgs.c:83 at OS_StartCounter(): DEBUG: OS_StartCounter: keysize: 1
2024/09/27 08:18:10 wazuh-agentd[2471624] msgs.c:120 at OS_StartCounter(): DEBUG: Assigning sender counter: 257:6136
2024/09/27 08:18:10 wazuh-agentd[2471624] keys.c:454 at OS_UpdateKeys(): DEBUG: Key reloading completed
2024/09/27 08:18:10 wazuh-agentd[2471624] start_agent.c:54 at connect_server(): INFO: Closing connection to server ([redacted]:1514/tcp).
2024/09/27 08:18:10 wazuh-agentd[2471624] start_agent.c:86 at connect_server(): INFO: Trying to connect to server ([redacted]:1514/tcp).
2024/09/27 08:18:10 wazuh-agentd[2471624] start_agent.c:352 at agent_handshake_to_server(): INFO: (4102): Connected to the server ([redacted]:1514/tcp).
2024/09/27 08:18:10 wazuh-agentd[2471624] notify.c:135 at run_notify(): DEBUG: Sending agent notification.
nobe80 commented 3 weeks ago

Hi there,

we have the same issue with Wazuh 4.9.1. .- no new data or update data. The data under events tab are very old and there are also agents where no more exist since weeks!. New agents does not deliver fresh data in vulnerability tab.

So is there a possibility to clear the values or delete the agents in vulnerability tab?

In the ossec logfile i found this issue: 2024/10/21 15:26:38 wazuh-modulesd:vulnerability-scanner: ERROR: VulnerabilityScannerFacade::start: Failed to open RocksDB database. Reason: While opening a file for sequentially reading: queue/vd/event/MANIFEST-000005: No such file or directory.

Does this have somtehing to with the issue?

regards

iacob28 commented 2 days ago

@nobe80

Maybe you can take a look at this thread on slack.

https://wazuh.slack.com/archives/C0A933R8E/p1727347397196999

image