Closed rafabailon closed 6 months ago
I have used a mv with Ubuntu 22.04
to investigate in which I have installed the manager and the simulate-agents
. I have enabled the debug options to see if the logs are generated and transmitted correctly.
I have been able to find more error logs. They are generated at the same time that the simulate-agents
generates the information.
2024/04/11 10:06:37 wazuh-db: ERROR: DB(003) sqlite3_prepare_v2() stmt(13): no such table: sys_osinfo
2024/04/11 10:06:37 wazuh-modulesd:vulnerability-scanner: WARNING: Discarded event: DB query error: Cannot get sys_osinfo database table information; SQL err: no such table: sys_osinfo
2024/04/11 10:06:37 wazuh-db: ERROR: DB(007) sqlite3_prepare_v2() stmt(13): no such table: sys_osinfo
2024/04/11 10:06:37 wazuh-modulesd:vulnerability-scanner: WARNING: Discarded event: DB query error: Cannot get sys_osinfo database table information; SQL err: no such table: sys_osinfo
2024/04/11 10:06:37 wazuh-db: ERROR: DB(005) sqlite3_prepare_v2() stmt(13): no such table: sys_osinfo
2024/04/11 10:06:37 wazuh-modulesd:vulnerability-scanner: WARNING: Discarded event: DB query error: Cannot get sys_osinfo database table information; SQL err: no such table: sys_osinfo
2024/04/11 10:06:37 wazuh-modulesd:vulnerability-scanner: WARNING: Discarded event: Empty response from Wazuh-DB
2024/04/11 10:06:37 wazuh-modulesd:vulnerability-scanner: WARNING: Discarded event: Empty response from Wazuh-DB
2024/04/11 10:06:37 wazuh-modulesd:vulnerability-scanner: WARNING: Discarded event: Empty response from Wazuh-DB
I have tested different combinations of operating systems for managers and agents. The information of the sys_osinfo
table, which is mentioned in the logs, can be found in the documentation. The next step is to review the agent databases and check for errors.
In the test I have used 20 simulated agents. Therefore there are 21 .db
files. The first, 000.db
, is the one that belongs to the manager and the rest to the simulated agents.
root@ubuntu2204:/var/ossec/queue/db# ls
000.db 001.db 003.db 005.db 007.db 009.db 011.db 013.db 015.db 017.db 019.db global.db
000.db-journal 002.db 004.db 006.db 008.db 010.db 012.db 014.db 016.db 018.db 020.db wdb
I have verified that in sys_osinfo
there is information for 000.db
. The information that appears corresponds to the manager and is correct.
root@ubuntu2204:/var/ossec/queue/db# sqlite3 000.db
SQLite version 3.37.2 2022-01-06 13:25:41
Enter ".help" for usage hints.
sqlite> SELECT * FROM sys_osinfo;
0|2024/04/11 13:10:30|ubuntu2204.localdomain|x86_64|Ubuntu|22.04.4 LTS (Jammy Jellyfish)|jammy|22|04|4||ubuntu|Linux|5.15.0-91-generic|#101-Ubuntu SMP Tue Nov 14 13:30:08 UTC 2023||1712841029343235276||7ed3f869f36298214ca4b49f052d6a3086029825
The agents have been generated with the command simulate-agents -a 172.16.1.13 -n 20 -m syscollector -w 1 -s 100 -t 200 -o debian10 -v 4.8.0 --debug
. This is the result in all the simulated agent databases.
root@ubuntu2204:/var/ossec/queue/db# sqlite3 001.db
SQLite version 3.37.2 2022-01-06 13:25:41
Enter ".help" for usage hints.
sqlite> SELECT * FROM sys_osinfo;
sqlite>
It occurs in all cases, for all generated debian agents. The databases do not contain the required information. I have generated 10 more agents with `simulate-agents -a 172.16.1.13 -n 10 -m syscollector -w 1 -s 100 -t 200 -o ubuntu18.04 -v 4.8.0 --debug
and I have checked their databases . The result has been the same as with debian.
The use of simulate-agents
is defined here. Use the Agent class in agent_simulator. In agent_db are the operations in the database. It looks like the OS information should be inserted into the database but for some reason when checking the database it is empty.
On Hold
while I help with Disk read increased for syscollector test in 4.8.0-beta4
I have done a test and these are all the logs and results when simulating an agent
The problem seems to be that the pipeline is using some default simulate-agents
options. When generating packages
events, for the simulation to be correct, it is necessary to also send, before the packages, the OS information. There would be 2 possible solutions:
When generating the events, if only packages
has been selected, the OS information is sent in the first iteration. With this solution, the number of packages
events will be batch_size - 1
and the total number of events will be batch_size
.
When generating the events, if only packages
has been selected, the OS information and the first packages
event will be sent in the first iteration. With this solution, the number of packages
events will be batch_size
and the number of generated events is batch_size + 1
.
The simulator supports generating different events at the same time. When trying to send osinfo
and packages
events, the vulnerabilities are generated correctly.
I have added a change so that when selecting only packages
(or leaving the default option), osinfo
is added (and the batch_size is 1). In this way, the OS information is always sent and errors are avoided. In the tests I have been able to see that vulnerability logs are generated.
I have created a PR with all the changes to add a class to the agent simulator to generate vulnerability events. The tests have been done locally. The results are in the PR.
I have had little time to devote to this issue as I received 3 communities at once.
I have refactored the code according to the comments in the PR. I have removed all the code repetition and made some changes to remove all the unnecessary code.
I've tested the simulator with different OSs for the agent and the manager (debian and ubuntu). Everything seems to work correctly.
Details can be found in the PR. For the tests I have used the simulator and checked the logs manually to check that the necessary syscollector messages are sent and the vulnerability alerts appear.
Occasionally I have found errors in the simulated agents (not all). After several tests, I have related the errors to the value of -t
. It is necessary to give it an appropriate value (in seconds) (between 40 and 60 seconds is fine) to avoid problems with the agents.
_It has been detected a minor bug in the parse_packages_content
script_. Not development related
I have made the changes suggested in the PR. I have also reviewed the comments to fix some errors that I have detected. Finally, I repeated the tests to make sure everything still works.
Related to https://github.com/wazuh/wazuh/issues/20781
Description
During the issue Benchmark vulnerability detector performance after the refactor I have done a series of tests (both locally and on machines built with a pipeline ) of Vulnerability Detector. I have used
simulate-agents
for the tests. Each event thatsimulate-agents
has simulated has generated an entry in the logs with the following errorThe simulated agents have been both debian10 and ubuntu18.04. The full list of OS that can be simulated is
A similar error message was reported and fixed here Windows Server 2019 Scan Error. It is necessary to investigate what is the cause of this error and its possible solution.