wazuh / wazuh

Wazuh - The Open Source Security Platform. Unified XDR and SIEM protection for endpoints and cloud workloads.
https://wazuh.com/
Other
10.56k stars 1.62k forks source link

Release 4.6.0 - Alpha 1 - Footprint Metrics - SYSCHECK (2.5d) #18915

Closed wazuhci closed 1 year ago

wazuhci commented 1 year ago

Footprint metrics information

Main release candidate issue # #18858
Main footprint metrics issue # #18862
Version 4.6.0
Release candidate # RC1
Tag https://github.com/wazuh/wazuh/tree/4.6.0-rc1

Stress test documentation

Packages used


Manager +
Plots ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_manager_centos/plots/monitor-manager-Test_stress_B4290_manager-pre-release_CPU.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_manager_centos/plots/monitor-manager-Test_stress_B4290_manager-pre-release_Disk.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_manager_centos/plots/monitor-manager-Test_stress_B4290_manager-pre-release_Disk_Read.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_manager_centos/plots/monitor-manager-Test_stress_B4290_manager-pre-release_Disk_Written.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_manager_centos/plots/monitor-manager-Test_stress_B4290_manager-pre-release_FD.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_manager_centos/plots/monitor-manager-Test_stress_B4290_manager-pre-release_PSS.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_manager_centos/plots/monitor-manager-Test_stress_B4290_manager-pre-release_Read_Ops.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_manager_centos/plots/monitor-manager-Test_stress_B4290_manager-pre-release_RSS_MAXMIN.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_manager_centos/plots/monitor-manager-Test_stress_B4290_manager-pre-release_RSS.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_manager_centos/plots/monitor-manager-Test_stress_B4290_manager-pre-release_SWAP.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_manager_centos/plots/monitor-manager-Test_stress_B4290_manager-pre-release_USS.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_manager_centos/plots/monitor-manager-Test_stress_B4290_manager-pre-release_VMS.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_manager_centos/plots/monitor-manager-Test_stress_B4290_manager-pre-release_Write_Ops.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_manager_centos/plots/Test_stress_B4290_manager_analysisd_events_Decoded_events.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_manager_centos/plots/Test_stress_B4290_manager_analysisd_events_Dropped_events.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_manager_centos/plots/Test_stress_B4290_manager_analysisd_events_EDPS.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_manager_centos/plots/Test_stress_B4290_manager_analysisd_events_Written_stats.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_manager_centos/plots/Test_stress_B4290_manager_analysisd_state_Number_Events.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_manager_centos/plots/Test_stress_B4290_manager_analysisd_state_Queues_state.png)
  • Logs and configuration [ossec_Test_stress_B4290_manager_2023-09-10.zip](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_manager_centos/logs/ossec_Test_stress_B4290_manager_2023-09-10.zip)
  • CSV [monitor-manager-Test_stress_B4290_manager-pre-release.csv](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_manager_centos/data/monitor-manager-Test_stress_B4290_manager-pre-release.csv) [Test_stress_B4290_manager_analysisd_events.csv](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_manager_centos/data/Test_stress_B4290_manager_analysisd_events.csv) [Test_stress_B4290_manager_analysisd_state.csv](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_manager_centos/data/Test_stress_B4290_manager_analysisd_state.csv) [Test_stress_B4290_manager_remoted_state.csv](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_manager_centos/data/Test_stress_B4290_manager_remoted_state.csv)

Centos agent +
Plots ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_centos/plots/monitor-agent-Test_stress_B4290_centos-pre-release_CPU.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_centos/plots/monitor-agent-Test_stress_B4290_centos-pre-release_Disk.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_centos/plots/monitor-agent-Test_stress_B4290_centos-pre-release_Disk_Read.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_centos/plots/monitor-agent-Test_stress_B4290_centos-pre-release_Disk_Written.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_centos/plots/monitor-agent-Test_stress_B4290_centos-pre-release_FD.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_centos/plots/monitor-agent-Test_stress_B4290_centos-pre-release_PSS.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_centos/plots/monitor-agent-Test_stress_B4290_centos-pre-release_Read_Ops.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_centos/plots/monitor-agent-Test_stress_B4290_centos-pre-release_RSS_MAXMIN.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_centos/plots/monitor-agent-Test_stress_B4290_centos-pre-release_RSS.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_centos/plots/monitor-agent-Test_stress_B4290_centos-pre-release_SWAP.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_centos/plots/monitor-agent-Test_stress_B4290_centos-pre-release_USS.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_centos/plots/monitor-agent-Test_stress_B4290_centos-pre-release_VMS.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_centos/plots/monitor-agent-Test_stress_B4290_centos-pre-release_Write_Ops.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_centos/plots/Test_stress_B4290_centos_agentd_state_AgentD_Number_of_events_buffered.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_centos/plots/Test_stress_B4290_centos_agentd_state_AgentD_Number_of_generated_events.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_centos/plots/Test_stress_B4290_centos_agentd_state_AgentD_Number_of_messages.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_centos/plots/Test_stress_B4290_centos_agentd_state_AgentD_Status.png)
  • Logs and configuration [ossec_Test_stress_B4290_centos_2023-09-10.zip](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_centos/logs/ossec_Test_stress_B4290_centos_2023-09-10.zip)
  • CSV [monitor-agent-Test_stress_B4290_centos-pre-release.csv](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_centos/data/monitor-agent-Test_stress_B4290_centos-pre-release.csv) [Test_stress_B4290_centos_agentd_state.csv](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_centos/data/Test_stress_B4290_centos_agentd_state.csv)

Ubuntu agent +
Plots ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_ubuntu/plots/monitor-agent-Test_stress_B4290_ubuntu-pre-release_CPU.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_ubuntu/plots/monitor-agent-Test_stress_B4290_ubuntu-pre-release_Disk.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_ubuntu/plots/monitor-agent-Test_stress_B4290_ubuntu-pre-release_Disk_Read.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_ubuntu/plots/monitor-agent-Test_stress_B4290_ubuntu-pre-release_Disk_Written.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_ubuntu/plots/monitor-agent-Test_stress_B4290_ubuntu-pre-release_FD.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_ubuntu/plots/monitor-agent-Test_stress_B4290_ubuntu-pre-release_PSS.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_ubuntu/plots/monitor-agent-Test_stress_B4290_ubuntu-pre-release_Read_Ops.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_ubuntu/plots/monitor-agent-Test_stress_B4290_ubuntu-pre-release_RSS_MAXMIN.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_ubuntu/plots/monitor-agent-Test_stress_B4290_ubuntu-pre-release_RSS.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_ubuntu/plots/monitor-agent-Test_stress_B4290_ubuntu-pre-release_SWAP.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_ubuntu/plots/monitor-agent-Test_stress_B4290_ubuntu-pre-release_USS.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_ubuntu/plots/monitor-agent-Test_stress_B4290_ubuntu-pre-release_VMS.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_ubuntu/plots/monitor-agent-Test_stress_B4290_ubuntu-pre-release_Write_Ops.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_ubuntu/plots/Test_stress_B4290_ubuntu_agentd_state_AgentD_Number_of_events_buffered.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_ubuntu/plots/Test_stress_B4290_ubuntu_agentd_state_AgentD_Number_of_generated_events.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_ubuntu/plots/Test_stress_B4290_ubuntu_agentd_state_AgentD_Number_of_messages.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_ubuntu/plots/Test_stress_B4290_ubuntu_agentd_state_AgentD_Status.png)
  • Logs and configuration [ossec_Test_stress_B4290_ubuntu_2023-09-10.zip](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_ubuntu/logs/ossec_Test_stress_B4290_ubuntu_2023-09-10.zip)
  • CSV [monitor-agent-Test_stress_B4290_ubuntu-pre-release.csv](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_ubuntu/data/monitor-agent-Test_stress_B4290_ubuntu-pre-release.csv) [Test_stress_B4290_ubuntu_agentd_state.csv](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_ubuntu/data/Test_stress_B4290_ubuntu_agentd_state.csv)

Windows agent +
Plots ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_windows/plots/monitor-winagent-Test_stress_B4290_windows-pre-release_CPU.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_windows/plots/monitor-winagent-Test_stress_B4290_windows-pre-release_Disk.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_windows/plots/monitor-winagent-Test_stress_B4290_windows-pre-release_Disk_Read.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_windows/plots/monitor-winagent-Test_stress_B4290_windows-pre-release_Disk_Written.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_windows/plots/monitor-winagent-Test_stress_B4290_windows-pre-release_Handles.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_windows/plots/monitor-winagent-Test_stress_B4290_windows-pre-release_Read_Ops.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_windows/plots/monitor-winagent-Test_stress_B4290_windows-pre-release_RSS_MAXMIN.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_windows/plots/monitor-winagent-Test_stress_B4290_windows-pre-release_RSS.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_windows/plots/monitor-winagent-Test_stress_B4290_windows-pre-release_USS.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_windows/plots/monitor-winagent-Test_stress_B4290_windows-pre-release_VMS.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_windows/plots/monitor-winagent-Test_stress_B4290_windows-pre-release_Write_Ops.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_windows/plots/Test_stress_B4290_windows_agentd_state_AgentD_Number_of_events_buffered.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_windows/plots/Test_stress_B4290_windows_agentd_state_AgentD_Number_of_generated_events.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_windows/plots/Test_stress_B4290_windows_agentd_state_AgentD_Number_of_messages.png) ![](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_windows/plots/Test_stress_B4290_windows_agentd_state_AgentD_Status.png)
  • Logs and configuration [ossec_Test_stress_B4290_windows_2023-09-10.zip](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_windows/logs/ossec_Test_stress_B4290_windows_2023-09-10.zip)
  • CSV [monitor-winagent-Test_stress_B4290_windows-pre-release.csv](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_windows/data/monitor-winagent-Test_stress_B4290_windows-pre-release.csv) [Test_stress_B4290_windows_agentd_state.csv](https://ci.wazuh.com/data/Test_stress/pre-release/4.6.0/B4290-3600m/B4290_agent_windows/data/Test_stress_B4290_windows_agentd_state.csv)

Conclusion :red_circle:

During testing no anomalies were found on the graphs, but the following Issues have been detected:

Deblintrake09 commented 1 year ago

Analysis report: Logs

Manager - Expected warnings in stress tests ``` wazuh-logcollector WARNING: Target 'agent' message queue is full (1024). Log lines may be lost. wazuh-syscheckd WARNING: Real-time inotify kernel queue is full. Some events may be lost. Next scheduled scan will recover lost data wazuh-remoted WARNING: Message queue is full (10). Events may be lost. wazuh-analysisd WARNING: Syscheck decoder queue is full. ``` - Known Issue https://github.com/wazuh/wazuh/issues/14710 ``` wazuh-authd ERROR: Invalid request for new agent from: 45.56.108.128 ``` - Known Issue https://github.com/wazuh/wazuh/issues/17596 ``` wazuh-remoted WARNING: Too big message size from socket [22]. wazuh-remoted WARNING: Too big message size from socket [24]. ``` - Known Issue https://github.com/wazuh/wazuh-jenkins/issues/5283 ``` wazuh-modulesd WARNING: Ubuntu Precise is no longer supported. ```
Ubuntu - Related to stress test configuration. Expected: ``` wazuh-agentd WARNING: Agent buffer is full: Events may be lost. wazuh-syscheckd WARNING: Real-time inotify kernel queue is full. Some events may be lost. Next scheduled scan will recover lost data ``` - Known Issue https://github.com/wazuh/wazuh-jenkins/issues/4867 ``` wazuh-agentd ERROR: (1137): Lost connection with manager. Setting lock. wazuh-agentd ERROR: (1216): Unable to connect to '[172.31.1.253]:1514/tcp': 'Connection refused'. ```
Centos - Related to stress test configuration. Expected: ``` wazuh-agentd WARNING: Agent buffer is full: Events may be lost. wazuh-syscheckd WARNING: Real-time inotify kernel queue is full. Some events may be lost. Next scheduled scan will recover lost data ``` - Known Issue: https://github.com/wazuh/wazuh-jenkins/issues/4867 ``` wazuh-agentd ERROR: (1137): Lost connection with manager. Setting lock. wazuh-agentd ERROR: (1216): Unable to connect to '[172.31.1.253]:1514/tcp': 'Connection refused'. ```
Windows - Expected warnings in stress tests ``` wazuh-agent WARNING: Agent buffer at 90 %. wazuh-agent WARNING: Agent buffer is full: Events may be lost. wazuh-agent WARNING: (6906): Real time process: no data. Probably buffer overflow. wazuh-agent WARNING: Agent buffer is flooded: Producing too many events. ``` - Known Issue https://github.com/wazuh/wazuh-jenkins/issues/5287 ``` wazuh-agent ERROR: (6716): Could not open handle for 'c:\\tmp\\syscheck_test\\files\\fimstress.2235855'. Error code: 2 [2023-09-07_19:42:12] [ERROR] (create_delete): files\\fimStress.513579 file cannot be deleted.\n ``` - Known Issues https://github.com/wazuh/wazuh/issues/11090 ``` wazuh-agent WARNING: At get_user(c:\\tmp\\syscheck_test\\files\\fimstress.11332140): CreateFile(): The system cannot find the file specified. (2 ``` - Known Issue https://github.com/wazuh/wazuh/issues/12074 ``` wazuh-agent ERROR: (6613): Real time Windows callback process: 'Access is denied.' (5). ``` - New Issue https://github.com/wazuh/wazuh/issues/18952 ``` wazuh-agent WARNING: The file 'C:\\Program Files (x86)\\ossec-agent\\libfimdb.dll' is not signed or its signature is invalid. ``` - Reported in https://github.com/wazuh/wazuh/issues/18967 ``` wazuh-agent WARNING: (6954): Entry 'c:\\tmp\\syscheck_test\\files\\fimstress.11999586' does not have any modified fields. No event will be generated. ```

Analysis reports: Graphs

No abnormal behaviors have been found with respect to 4.5.2 RC 1

juliamagan commented 1 year ago

Please add all reported issues to the main issue comment, separating the new ones from the known ones.

juliamagan commented 1 year ago

LGTM

mauromalara commented 1 year ago

LGTM