search

wazuh / wazuh-qa

Wazuh - Quality Assurance
GNU General Public License v2.0
65 stars 32 forks source link

Functionality tests for the database comparison with state index approach #5303

Closed juliamagan closed 6 months ago

juliamagan commented 6 months ago

Description

This issue is intended to cover the testing process followed and obtained for https://github.com/wazuh/wazuh/issues/22867 development.

Test environment

Component Quantity Operating System CPU (cores) RAM (GB) Disk (GB)
Master 1 Ubuntu 22 4 8 50
Workers 2 Ubuntu 22 4 8 50
Agent 1 1 Ubuntu 22 2 4 30
Agent 2 1 Windows 11 2 4 30
Load Balancer 1 Ubuntu 22 4 8 50
Indexers 2 Ubuntu 22 2 4 30

[!NOTE] The load balancer is located on the master node.

Architecture Core development package URL
DEB 4.8.0-0.commitd31b277
RPM 4.8.0-0.commitd31b277

Test cases

ID Description Status
T1 Change of worker due to agent restart :green_circle:
T2 Change of worker due to connection loss :green_circle:
T3 Change of worker due to worker restart :green_circle:
T4 Loss and recovery of connection to a worker :green_circle:
T5 Change of worker due to kill of worker :green_circle:
T6 Restart of worker while vulnerability scanning is in progress :green_circle:
T7 Restart of indexer while states are being indexed :green_circle:
T8 Loss of connection of indexer while states are being indexed :green_circle:
T9 Restart of indexer while states are being indexed and agent changes worker :green_circle:
T10 Loss of connection of indexer while states are being indexed and agent changes worker :green_circle:
T11 Restart of both indexers while states are being indexed :green_circle:
T12 Loss of connection of both indexers while states are being indexed :green_circle:
T13 Removal of a vulnerable package and change worker with long syscollector interval :green_circle:
T14 Agent re-registered with same name but different ID :green_circle:
T15 Agent re-registered with same ID but different name :green_circle:
T16 Restart of the master node :green_circle:
T17 Loss of connection with the master node :green_circle:
T18 While the agent is switching workers, the target worker is unavailable :green_circle:
T19 While the agent is switching workers, the target worker is unavailable and the first worker is stopped :green_circle:

Status legend:

🟢 - Approved 🟡 - Approved with warnings or expected errors 🔴 - Rejected

MARCOSD4 commented 6 months ago

Before testing

Both agents connected to the worker 1
```console # /var/ossec/bin/cluster_control -a ID NAME IP STATUS VERSION NODE NAME 000 ip-XX.XX.XX.XX XX.XX.XX.XX active Wazuh v4.8.0 master 001 ip-XX.XX.XX.XX XX.XX.XX.XX active Wazuh v4.8.0 worker-1 002 DESKTOP-AQ2R8SM XX.XX.XX.XX active Wazuh v4.8.0 worker-1 ```
Total vulnerabilities
```console # curl -k -u USER:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 868, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Ubuntu agent vulnerabilities
```console # curl -k -u USER:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "001" } } }' { "count" : 118, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Windows agent vulnerabilities
```console # curl -k -u USER:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "002" } } }' { "count" : 651, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```

wazuh-states-vulnerabilities.json

MARCOSD4 commented 6 months ago

Case 1: Change of worker due to agent restart :green_circle:

Given a configured master And two workers configured And an agent connects to the worker1 node And the agent with state Active When the agent is restarted and changes worker Then No doubling of the vulnerability index

Ubuntu agent connected to the worker 2
```console # /var/ossec/bin/cluster_control -a ID NAME IP STATUS VERSION NODE NAME 000 ip-XX.XX.XX.XX XX.XX.XX.XX active Wazuh v4.8.0 master 001 ip-XX.XX.XX.XX XX.XX.XX.XX active Wazuh v4.8.0 worker-2 002 DESKTOP-AQ2R8SM XX.XX.XX.XX active Wazuh v4.8.0 worker-1 ```
Total vulnerabilities
```console # curl -k -u USER:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 868, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Ubuntu agent vulnerabilities
```console # curl -k -u USER:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "001" } } }' { "count" : 118, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Windows agent vulnerabilities
```console # curl -k -u USER:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "002" } } }' { "count" : 651, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```

wazuh-states-vulnerabilities.json

MARCOSD4 commented 6 months ago

Case 2: Change of worker due to connection loss :green_circle:

Given a configured master And two workers configured And an agent connects to the worker1 node with the two workers in his configuration And the agent with state Active When connection lost and worker changed to worker2 Then No doubling of the vulnerability index

Ubuntu agent configuration
```console
worker1-ip
1514 tcp
worker2-ip
1514 tcp ```
Ubuntu agent connected to the worker 2
```console # /var/ossec/bin/cluster_control -a ID NAME IP STATUS VERSION NODE NAME 000 ip-XX.XX.XX.XX XX.XX.XX.XX active Wazuh v4.8.0 master 001 ip-XX.XX.XX.XX XX.XX.XX.XX active Wazuh v4.8.0 worker-2 002 DESKTOP-AQ2R8SM XX.XX.XX.XX active Wazuh v4.8.0 worker-1 ```
Total vulnerabilities
```console # curl -k -u USER:PASS https://XX.XX.XX.XX9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 868, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Ubuntu agent vulnerabilities
```console # curl -k -u USER:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "001" } } }' { "count" : 118, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Windows agent vulnerabilities
```console # curl -k -u USER:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "002" } } }' { "count" : 651, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```

wazuh-states-vulnerabilities.json

MARCOSD4 commented 6 months ago

Case 3: Change of worker due to worker restart :green_circle:

Given a configured master And two workers configured And an agent connects to the worker2 node And the agent with state Active When Restart and change of worker Then No doubling of the vulnerability index

Ubuntu agent configuration
```console
worker1-ip
1514 tcp
worker2-ip
1514 tcp ```
Ubuntu agent connected to the worker 1
```console # /var/ossec/bin/cluster_control -a ID NAME IP STATUS VERSION NODE NAME 000 ip-XX.XX.XX.XX XX.XX.XX.XX active Wazuh v4.8.0 master 001 ip-XX.XX.XX.XX XX.XX.XX.XX active Wazuh v4.8.0 worker-1 002 DESKTOP-AQ2R8SM XX.XX.XX.XX active Wazuh v4.8.0 worker-1 ```
Total vulnerabilities
```console # curl -k -u USER:PASSC https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 868, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Ubuntu agent vulnerabilities
```console # curl -k -u USER:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "001" } } }' { "count" : 118, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Windows agent vulnerabilities
```console # curl -k -u USER:PASSC https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "002" } } }' { "count" : 651, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```

wazuh-states-vulnerabilities.json

MARCOSD4 commented 6 months ago

Case 4: Loss and recovery of connection to a worker :green_circle:

Given a configured master And two workers configured And an agent connects to the worker1 node And the agent with state Active When Worker loss and recovery connection without changing of worker Then No doubling of the vulnerability index

Loss and recovery connection
```console 2024/04/30 15:38:49 wazuh-agentd: ERROR: Connection socket: Connection reset by peer (104) 2024/04/30 15:38:49 wazuh-agentd: ERROR: (1137): Lost connection with manager. Setting lock. 2024/04/30 15:38:49 wazuh-agentd: INFO: Closing connection to server ([worker1-ip]:1514/tcp). 2024/04/30 15:38:49 wazuh-agentd: INFO: Trying to connect to server ([worker1-ip]:1514/tcp). 2024/04/30 15:38:57 wazuh-logcollector: WARNING: Process locked due to agent is offline. Waiting for connection... 2024/04/30 15:39:36 wazuh-modulesd:syscollector: INFO: Starting evaluation. 2024/04/30 15:39:37 wazuh-modulesd: WARNING: Process locked due to agent is offline. Waiting for connection... 2024/04/30 15:39:58 wazuh-agentd: INFO: (4102): Connected to the server ([worker1-ip]:1514/tcp). 2024/04/30 15:39:58 wazuh-agentd: INFO: Server responded. Releasing lock. 2024/04/30 15:40:02 wazuh-modulesd: INFO: Agent is now online. Process unlocked, continuing... 2024/04/30 15:40:02 wazuh-logcollector: INFO: Agent is now online. Process unlocked, continuing... ```
Ubuntu agent connected to the worker 1
```console # /var/ossec/bin/cluster_control -a ID NAME IP STATUS VERSION NODE NAME 000 ip-XX.XX.XX.XX XX.XX.XX.XX active Wazuh v4.8.0 master 001 ip-XX.XX.XX.XX XX.XX.XX.XX active Wazuh v4.8.0 worker-1 002 DESKTOP-AQ2R8SM XX.XX.XX.XX active Wazuh v4.8.0 worker-1 ```
Total vulnerabilities
```console # curl -k -u USER:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 868, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Ubuntu agent vulnerabilities
```console # curl -k -u USER:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "001" } } }' { "count" : 118, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Windows agent vulnerabilities
```console # curl -k -u USER:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "002" } } }' { "count" : 651, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```

wazuh-states-vulnerabilities.json

MARCOSD4 commented 6 months ago

Case 5: Change of worker due to kill of worker :green_circle:

Given a configured master And two workers configured And an agent connects to the worker1 node And the agent with state Active When kill worker1 and change to worker2 Then No doubling of the vulnerability index

Change of worker
```console 2024/05/02 07:21:55 wazuh-agentd: ERROR: (1137): Lost connection with manager. Setting lock. 2024/05/02 07:21:55 wazuh-agentd: INFO: Closing connection to server ([worker1-ip]:1514/tcp). 2024/05/02 07:21:55 wazuh-agentd: INFO: Trying to connect to server ([worker1-ip]:1514/tcp). 2024/05/02 07:21:55 wazuh-agentd: ERROR: (1216): Unable to connect to '[worker1-ip]:1514/tcp': 'Connection refused'. 2024/05/02 07:22:05 wazuh-agentd: INFO: Trying to connect to server ([worker1-ip]:1514/tcp). 2024/05/02 07:22:05 wazuh-agentd: ERROR: (1216): Unable to connect to '[worker1-ip]:1514/tcp': 'Connection refused'. 2024/05/02 07:22:15 wazuh-agentd: INFO: Trying to connect to server ([worker1-ip]:1514/tcp). 2024/05/02 07:22:15 wazuh-agentd: ERROR: (1216): Unable to connect to '[worker1-ip]:1514/tcp': 'Connection refused'. 2024/05/02 07:22:25 wazuh-agentd: INFO: Trying to connect to server ([worker1-ip]:1514/tcp). 2024/05/02 07:22:25 wazuh-agentd: ERROR: (1216): Unable to connect to '[worker1-ip]:1514/tcp': 'Connection refused'. 2024/05/02 07:22:35 wazuh-agentd: INFO: Trying to connect to server ([worker1-ip]:1514/tcp). 2024/05/02 07:22:35 wazuh-agentd: ERROR: (1216): Unable to connect to '[worker1-ip]:1514/tcp': 'Connection refused'. 2024/05/02 07:22:35 wazuh-agentd: INFO: Requesting a key from server: worker1-ip 2024/05/02 07:22:35 wazuh-agentd: ERROR: (1208): Unable to connect to enrollment service at '[worker1-ip]:1515' 2024/05/02 07:22:45 wazuh-agentd: WARNING: (4101): Waiting for server reply (not started). Tried: 'worker1-ip'. 2024/05/02 07:22:45 wazuh-agentd: INFO: Trying next server ip in the line: 'worker2-ip'. 2024/05/02 07:22:45 wazuh-agentd: INFO: Trying to connect to server ([worker2-ip]:1514/tcp). 2024/05/02 07:22:45 wazuh-agentd: INFO: (4102): Connected to the server ([worker2-ip]:1514/tcp). ```
Ubuntu agent connected to the worker 2
```console # /var/ossec/bin/cluster_control -a ID NAME IP STATUS VERSION NODE NAME 000 ip-XX.XX.XX.XX XX.XX.XX.XX active Wazuh v4.8.0 master 001 ip-XX.XX.XX.XX XX.XX.XX.XX active Wazuh v4.8.0 worker-2 002 DESKTOP-AQ2R8SM XX.XX.XX.XX active Wazuh v4.8.0 worker-1 ```
Total vulnerabilities
```console # curl -k -u USER:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 868, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Ubuntu agent vulnerabilities
```console # curl -k -u USER:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "001" } } }' { "count" : 118, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Windows agent vulnerabilities
```console # curl -k -u USER:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "002" } } }' { "count" : 651, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```

wazuh_states_vulnerabilities.json

santipadilla commented 6 months ago

Before testing in a different environment

Total vulnerabilities
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx.xx.xx.xx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 683, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Ubuntu agent vulnerabilities
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx.xx.xx.xx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "001" } } }' { "count" : 98, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Windows agent vulnerabilities
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx.xx.xx.xx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "002" } } }' { "count" : 486, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Total vulnerabilities json
```console curl -k -u user:pass https://xxx.xx.xx.xx:9200/wazuh-states-vulnerabilities/_search?pretty=true -H 'Content-Type: application/json' -d '{ "size": 1000}' > vulns.json ``` [vulns.json](https://github.com/wazuh/wazuh-qa/files/15185007/vulns.json)

Case 6: Restart of worker while vulnerability scanning is in progress :green_circle:

Given a configured master And two workers configured And an agent connects to the worker1 node And the agent with state Active When restart a worker while vulnerability scanning is in progress Then No doubling of the vulnerability index

Both agent connected to the worker 1
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# /var/ossec/bin/cluster_control -a ID NAME IP STATUS VERSION NODE NAME 000 ip-xxx-xx-xx-xxx xxx-xx-xx-xxx active Wazuh v4.8.0 node01 001 ip-xxx-xx-xx-xxx xxx-xx-xx-xxx active Wazuh v4.8.0 worker1-node 002 DESKTOP-AQ2R8SM xxx-xx-xx-xxx active Wazuh v4.8.0 worker1-node ```

Note: First we install vulnerable packages on Ubuntu and Windows agents

Total vulnerabilities
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx-xx-xx-xxx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 728, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Ubuntu agent vulnerabilities
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx-xx-xx-xxx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "001" } } }' { "count" : 118, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Windows agent vulnerabilities
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx-xx-xx-xxx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "002" } } }' { "count" : 511, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Restart worker
```console 2024/05/02 08:15:37 wazuh-modulesd:syscollector: INFO: Starting evaluation. 2024/05/02 08:15:39 wazuh-agentd: ERROR: (1137): Lost connection with manager. Setting lock. 2024/05/02 08:15:39 wazuh-agentd: INFO: Closing connection to server ([xxx-xx-xx-xxx]:1514/tcp). 2024/05/02 08:15:39 wazuh-agentd: INFO: Trying to connect to server ([xxx-xx-xx-xxx]:1514/tcp). 2024/05/02 08:15:39 wazuh-agentd: ERROR: (1216): Unable to connect to '[xxx-xx-xx-xxx]:1514/tcp': 'Connection refused'. 2024/05/02 08:15:39 wazuh-modulesd: WARNING: Process locked due to agent is offline. Waiting for connection... 2024/05/02 08:15:49 wazuh-agentd: INFO: Trying to connect to server ([xxx-xx-xx-xxx]:1514/tcp). 2024/05/02 08:15:49 wazuh-agentd: ERROR: (1216): Unable to connect to '[xxx-xx-xx-xxx]:1514/tcp': 'Connection refused'. 2024/05/02 08:15:59 wazuh-agentd: INFO: Trying to connect to server ([xxx-xx-xx-xxx]:1514/tcp). 2024/05/02 08:15:59 wazuh-agentd: INFO: (4102): Connected to the server ([xxx-xx-xx-xxx]:1514/tcp). 2024/05/02 08:15:59 wazuh-agentd: INFO: Server responded. Releasing lock. 2024/05/02 08:15:59 wazuh-modulesd: INFO: Agent is now online. Process unlocked, continuing... 2024/05/02 08:15:59 wazuh-modulesd:syscollector: INFO: Evaluation finished. ```

vulns.json

MARCOSD4 commented 6 months ago

Case 7: Restart of indexer while states are being indexed :green_circle:

Given a configured master And two workers configured And an agent connects to the worker2 and other agent to the worker1 And the agents with state Active When restart of indexer while states are being indexed Then no doubling of the vulnerability index

Install a vulnerable package on Windows agent
We install a vulnerable package to make changes to the database. We install node 18.1.1 (24 vulnerabilities). ```console PS C:\Users\Jenkins\Downloads> msiexec /package node-v18.1.0-x64.msi /passive ```
Restart wazuh-indexer1
We restart the Wazuh indexer right at the time when filebeat sends the alerts generated after installing the node package on Windows agent, as it has been found that vulnerabilities are indexed at that time. Cluster state: ```console # /var/ossec/bin/cluster_control -a ID NAME IP STATUS VERSION NODE NAME 000 ip-XX.XX.XX.XX XX.XX.XX.XX active Wazuh v4.8.0 master 001 ip-XX.XX.XX.XX XX.XX.XX.XX active Wazuh v4.8.0 worker-2 002 DESKTOP-AQ2R8SM XX.XX.XX.XX active Wazuh v4.8.0 worker-1 ```
Total vulnerabilities
```console # curl -k -u USER:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 893, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Ubuntu agent vulnerabilities
```console # curl -k -u USER:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "001" } } }' { "count" : 118, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Windows agent vulnerabilities
There is 25 more vulns, 24 from the node package and 1 from npm which is installed with node. ```console # curl -k -u adUSERmin:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "002" } } }' { "count" : 676, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```

wazuh-states-vulnerabilities.json

santipadilla commented 6 months ago

Case 8: Loss of connection of indexer while states are being indexed :green_circle:

Given a configured master And two workers configured And two indexers configured And both agents connects to the worker1 node And both agents with state Active When loss of connection of indexer while states are being indexed Then No doubling of the vulnerability index

Both agent connected to the worker 1
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# /var/ossec/bin/cluster_control -a ID NAME IP STATUS VERSION NODE NAME 000 ip-xxx-xx-xx-xxx xxx-xx-xx-xxx active Wazuh v4.8.0 node01 001 ip-xxx-xx-xx-xxx xxx-xx-xx-xxx active Wazuh v4.8.0 worker1-node 002 DESKTOP-AQ2R8SM xxx-xx-xx-xxx active Wazuh v4.8.0 worker1-node ```

Note: First we install vulnerable packages on Ubuntu and Windows agents

Total vulnerabilities
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx-xx-xx-xxx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 728, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Ubuntu agent vulnerabilities
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx-xx-xx-xxx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "001" } } }' { "count" : 118, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Windows agent vulnerabilities
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx-xx-xx-xxx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "002" } } }' { "count" : 511, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
States are being indexed
>Note: Delete agent, wait for connection, wait for VD logs to appear in worker, indexer connection lost. ```console root@ip-xxx-xx-xx-xx:/home/ubuntu# sudo iptables -A INPUT -p tcp --dport 9200 -j DROP root@ip-xxx-xx-xx-xx:/home/ubuntu# filebeat test output elasticsearch: https://xxx-xx-xx-xx:9200... parse url... OK connection... parse host... OK dns lookup... OK addresses: xxx-xx-xx-xx ^C root@ip-xxx-xx-xx-xx:/home/ubuntu# ```
Reactivate connection
```console root@ip-xxx-xx-xx-xx:/home/ubuntu# sudo iptables -D INPUT -p tcp --dport 9200 -j DROP root@ip-xxx-xx-xx-xx:/home/ubuntu# filebeat test output elasticsearch: https://xxx-xx-xx-xx:9200... parse url... OK connection... parse host... OK dns lookup... OK addresses: xxx-xx-xx-xx dial up... OK TLS... security: server's certificate chain verification is enabled handshake... OK TLS version: TLSv1.2 dial up... OK talk to server... OK version: 7.10.2 elasticsearch: https://xxx-xx-xx-xx:9200... parse url... OK connection... parse host... OK dns lookup... OK addresses: xxx-xx-xx-xx dial up... OK TLS... security: server's certificate chain verification is enabled handshake... OK TLS version: TLSv1.2 dial up... OK talk to server... OK version: 7.10.2 root@ip-xxx-xx-xx-xx:/home/ubuntu# ```

vulns.json

santipadilla commented 6 months ago

Case 9: Restart of indexer while states are being indexed and agent changes worker :green_circle:

Given a configured master And two workers configured And two indexers configured And both agents connects to the worker1 node And both agents with state Active When restart of indexer while states are being indexed and agent changes worker Then No doubling of the vulnerability index

Both agent connected to the worker 1
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# /var/ossec/bin/cluster_control -a ID NAME IP STATUS VERSION NODE NAME 000 ip-xxx-xx-xx-xxx xxx-xx-xx-xxx active Wazuh v4.8.0 node01 001 ip-xxx-xx-xx-xxx xxx-xx-xx-xxx active Wazuh v4.8.0 worker1-node 002 DESKTOP-AQ2R8SM xxx-xx-xx-xxx active Wazuh v4.8.0 worker1-node ```

Note: First we uninstall vulnerable packages on Ubuntu and Windows agents to update database

Total vulnerabilities
```console root@ip-172-31-40-132:/home/ubuntu# curl -k -u admin:I1APNa56feSy2I1GzET8?RMyWpRam7h6 https://172.31.42.75:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 683, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Ubuntu agent vulnerabilities
```console root@ip-172-31-40-132:/home/ubuntu# curl -k -u admin:I1APNa56feSy2I1GzET8?RMyWpRam7h6 https://172.31.46.237:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "004" } } }' { "count" : 98, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Windows agent vulnerabilities
```console root@ip-172-31-40-132:/home/ubuntu# curl -k -u admin:I1APNa56feSy2I1GzET8?RMyWpRam7h6 https://172.31.46.237:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "002" } } }' { "count" : 486, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
States are being indexed
>Note: Delete agent, wait for connection, wait for VD logs to appear in worker, indexer restart and agent changes worker. ```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# sudo systemctl restart wazuh-indexer root@ip-xxx-xx-xx-xxx:/home/ubuntu# root@ip-xxx-xx-xx-xxx:/home/ubuntu# nano /var/ossec/etc/ossec.conf root@ip-xxx-xx-xx-xxx:/home/ubuntu# sudo systemctl restart wazuh-agent root@ip-xxx-xx-xx-xxx:/home/ubuntu# root@ip-xxx-xx-xx-xxx:/home/ubuntu# /var/ossec/bin/cluster_control -a ID NAME IP STATUS VERSION NODE NAME 000 ip-xxx-xx-xx-xxx xxx-xx-xx-xxx active Wazuh v4.8.0 node01 002 DESKTOP-AQ2R8SM xxx-xx-xx-xxx active Wazuh v4.8.0 worker1-node 004 ip-xxx-xx-xx-xxx xxx-xx-xx-xxx active Wazuh v4.8.0 worker2-node ```

vulns.json

santipadilla commented 6 months ago

Case 10: Loss of connection of indexer while states are being indexed and agent changes worker :green_circle:

Given a configured master And two workers configured And two indexers configured And one agent connects to the worker1 node and the other agent to worker2 node. And both agents with state Active When loss of connection of indexer while states are being indexed and agent changes worker Then No doubling of the vulnerability index

Both agents connected to a different worker
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# /var/ossec/bin/cluster_control -a ID NAME IP STATUS VERSION NODE NAME 000 ip-xxx-xx-xx-xxx xxx-xx-xx-xxx active Wazuh v4.8.0 node01 002 DESKTOP-AQ2R8SM xxx-xx-xx-xxx active Wazuh v4.8.0 worker1-node 004 ip-xxx-xx-xx-xxx xxx-xx-xx-xxx active Wazuh v4.8.0 worker2-node ```

Note: First we install vulnerable packages on Ubuntu and Windows agents to update database

Total vulnerabilities
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx-xx-xx-xxx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 728, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Ubuntu agent vulnerabilities
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx-xx-xx-xxx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "004" } } }' { "count" : 118, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Windows agent vulnerabilities
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx-xx-xx-xxx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "002" } } }' { "count" : 511, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Agent changes worker
```console ```
States are being indexed
>Note: Delete agent, wait for connection, wait for VD logs to appear in worker, indexer loss connection and agent changes worker. ```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# sudo iptables -A INPUT -p tcp --dport 9200 -j DROP root@ip-xxx-xx-xx-xxx:/home/ubuntu# filebeat test output elasticsearch: https://xxx-xx-xx-xxx:9200... parse url... OK connection... parse host... OK dns lookup... OK addresses: xxx-xx-xx-xxx ^C root@ip-xxx-xx-xx-xxx:/home/ubuntu# root@ip-xxx-xx-xx-xxx:/home/ubuntu# nano /var/ossec/etc/ossec.conf root@ip-xxx-xx-xx-xxx:/home/ubuntu# sudo systemctl restart wazuh-agent root@ip-xxx-xx-xx-xxx:/home/ubuntu# root@ip-xxx-xx-xx-xxx:/home/ubuntu# /var/ossec/bin/cluster_control -a ID NAME IP STATUS VERSION NODE NAME 000 ip-xxx-xx-xx-xxx xxx-xx-xx-xxx active Wazuh v4.8.0 node01 002 DESKTOP-AQ2R8SM xxx-xx-xx-xxx active Wazuh v4.8.0 worker1-node 006 ip-xxx-xx-xx-xxx xxx-xx-xx-xxx active Wazuh v4.8.0 worker1-node ```
Reactivate connection
```console root@ip-xxx-xx-xx-xx:/home/ubuntu# sudo iptables -D INPUT -p tcp --dport 9200 -j DROP root@ip-xxx-xx-xx-xx:/home/ubuntu# filebeat test output elasticsearch: https://xxx-xx-xx-xx:9200... parse url... OK connection... parse host... OK dns lookup... OK addresses: xxx-xx-xx-xx dial up... OK TLS... security: server's certificate chain verification is enabled handshake... OK TLS version: TLSv1.2 dial up... OK talk to server... OK version: 7.10.2 elasticsearch: https://xxx-xx-xx-xx:9200... parse url... OK connection... parse host... OK dns lookup... OK addresses: xxx-xx-xx-xx dial up... OK TLS... security: server's certificate chain verification is enabled handshake... OK TLS version: TLSv1.2 dial up... OK talk to server... OK version: 7.10.2 root@ip-xxx-xx-xx-xx:/home/ubuntu# ```

vulns.json

MARCOSD4 commented 6 months ago

Case 11: Restart of both indexers while states are being indexed :green_circle:

Given a configured master And two workers configured And an agent connects to the worker2 and other agent to the worker1 And the agents with state Active When restart of both indexer while states are being indexed Then no doubling of the vulnerability index

Install a vulnerable package on Windows agent
We install a vulnerable package to make changes to the database. We install node 18.1.1 (24 vulnerabilities). ```console PS C:\Users\Jenkins\Downloads> msiexec /package node-v18.1.0-x64.msi /passive ```
Restart both indexers
We restart the Wazuh indexer right at the time when filebeat sends the alerts generated after installing the node package on the Windows agent, as it has been found that vulnerabilities are indexed at that time. Cluster state: ```console # /var/ossec/bin/cluster_control -a ID NAME IP STATUS VERSION NODE NAME 000 ip-XX.XX.XX.XX XX.XX.XX.XX active Wazuh v4.8.0 master 001 ip-XX.XX.XX.XX XX.XX.XX.XX active Wazuh v4.8.0 worker-2 002 DESKTOP-AQ2R8SM XX.XX.XX.XX active Wazuh v4.8.0 worker-1 ```
Total vulnerabilities
```console # curl -k -u USER:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 893, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Ubuntu agent vulnerabilities
```console # curl -k -u USER:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "001" } } }' { "count" : 118, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Windows agent vulnerabilities
There is 25 more vulns, 24 from the node package and 1 from npm which is installed with node. ```console # curl -k -u adUSERmin:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "002" } } }' { "count" : 676, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```

wazuh-states-vulnerabilities.json

MARCOSD4 commented 6 months ago

Case 12: Loss of connection of both indexers while states are being indexed :green_circle:

Given a configured master And two workers configured And an agent connects to the worker2 and other agent to the worker1 And the agents with state Active When loss of connection of both indexer while states are being indexed Then no doubling of the vulnerability index

Install a vulnerable package on Windows agent
We install a vulnerable package to make changes to the database. We install node 18.1.1 (24 vulnerabilities). ```console PS C:\Users\Jenkins\Downloads> msiexec /package node-v18.1.0-x64.msi /passive ```
Loss connection of both indexers
We loss the connection with the Wazuh indexer right at the time when filebeat sends the alerts generated after installing the node package on the Windows agent, as it has been found that vulnerabilities are indexed at that time. Cluster state: ```console # /var/ossec/bin/cluster_control -a ID NAME IP STATUS VERSION NODE NAME 000 ip-XX.XX.XX.XX XX.XX.XX.XX active Wazuh v4.8.0 master 001 ip-XX.XX.XX.XX XX.XX.XX.XX active Wazuh v4.8.0 worker-2 002 DESKTOP-AQ2R8SM XX.XX.XX.XX active Wazuh v4.8.0 worker-1 ```
Total vulnerabilities
```console # curl -k -u USER:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 893, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Ubuntu agent vulnerabilities
```console # curl -k -u USER:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "001" } } }' { "count" : 118, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Windows agent vulnerabilities
There is 25 more vulns, 24 from the node package and 1 from npm which is installed with node. ```console # curl -k -u adUSERmin:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "002" } } }' { "count" : 676, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```

wazuh-states-vulnerabilities.json

santipadilla commented 6 months ago

Case 13: Removal of a vulnerable package and change worker with long syscollector interval :green_circle:

Given a configured master And two workers configured And two indexers configured And both agents connected to the same worker And both agents with state Active When removal of a vulnerable package and change worker with long syscollector interval Then No doubling of the vulnerability index

Both agents connected to the same worker
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# /var/ossec/bin/cluster_control -a ID NAME IP STATUS VERSION NODE NAME 000 ip-xxx-xx-xx-xxx xxx-xx-xx-xxx active Wazuh v4.8.0 node01 002 DESKTOP-AQ2R8SM xxx-xx-xx-xxx active Wazuh v4.8.0 worker1-node 006 ip-xxx-xx-xx-xxx xxx-xx-xx-xxx active Wazuh v4.8.0 worker1-node ```
Total vulnerabilities
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx-xx-xx-xxx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 728, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Ubuntu agent vulnerabilities
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx-xx-xx-xxx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "006" } } }' { "count" : 118, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Windows agent vulnerabilities
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx-xx-xx-xxx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "002" } } }' { "count" : 511, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Long syscollector interval
```console 10m no ```
Removal of a vulnerable package
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# sudo dpkg -r grafana (Reading database ... 102757 files and directories currently installed.) Removing grafana (8.5.5) ... ```
Same vulnerabilities before syscollector update
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx-xx-xx-xxx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "006" } } }' { "count" : 118, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Change worker and syscollector update
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# nano /var/ossec/etc/ossec.conf 2024/05/02 13:00:42 wazuh-modulesd:syscollector: INFO: Module started. 2024/05/02 13:10:42 wazuh-modulesd:syscollector: INFO: Starting evaluation. 2024/05/02 13:10:42 wazuh-modulesd:syscollector: INFO: Evaluation finished. root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx-xx-xx-xxx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "006" } } }' { "count" : 98, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```

vulns.json

santipadilla commented 6 months ago

Case 14: Agent re-registered with same name but different ID :green_circle:

Given a configured master And two workers configured And two indexers configured And both agents connected to the same worker And both agents with state Active When agent re-registered with same name but different ID Then No doubling of the vulnerability index

Both agents connected to the same worker
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# /var/ossec/bin/cluster_control -a ID NAME IP STATUS VERSION NODE NAME 000 ip-xxx-xx-xx-xxx xxx-xx-xx-xxx active Wazuh v4.8.0 node01 002 DESKTOP-AQ2R8SM xxx-xx-xx-xxx active Wazuh v4.8.0 worker1-node 006 ip-xxx-xx-xx-xxx xxx-xx-xx-xxx active Wazuh v4.8.0 worker1-node ```

Note: First we install vulnerable packages on Ubuntu and Windows agents to update database

Total vulnerabilities
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx-xx-xx-xxx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 728, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Ubuntu agent vulnerabilities
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx-xx-xx-xxx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "006" } } }' { "count" : 118, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Windows agent vulnerabilities
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx-xx-xx-xxx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "002" } } }' { "count" : 511, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Re-registred agent with different ID
```console root@ip-xxx-xx-xx-xxx :/home/ubuntu# /var/ossec/bin/agent_control -l Wazuh agent_control. List of available agents: ID: 000, Name: ip-xxx-xx-xx-xxx (server), IP: 127.0.0.1, Active/Local ID: 002, Name: DESKTOP-AQ2R8SM, IP: any, Active List of agentless devices: root@ip-xxx-xx-xx-xxx :/home/ubuntu# /var/ossec/bin/agent_control -l Wazuh agent_control. List of available agents: ID: 000, Name: ip-xxx-xx-xx-xxx (server), IP: 127.0.0.1, Active/Local ID: 002, Name: DESKTOP-AQ2R8SM, IP: any, Active ID: 007, Name: ip-xxx-xx-xx-xxx , IP: any, Never connected List of agentless devices: root@ip-xxx-xx-xx-xxx:/home/ubuntu# /var/ossec/bin/agent_control -l Wazuh agent_control. List of available agents: ID: 000, Name: ip-xxx-xx-xx-xxx (server), IP: 127.0.0.1, Active/Local ID: 002, Name: DESKTOP-AQ2R8SM, IP: any, Active ID: 007, Name: ip-xxx-xx-xx-xxx , IP: any, Active List of agentless devices: root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx-xx-xx-xxx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 728, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx-xx-xx-xxx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "007" } } }' { "count" : 118, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```

vulns.json

MARCOSD4 commented 6 months ago

Case 15: Agent re-registered with same ID but different name :green_circle:

Given a configured master And two workers configured And an agent connects to the worker2 and other agent to the worker1 And the agents with state Active When re-register ubuntu agent with the same ID but different name Then no doubling of the vulnerability index

Change agent name, remove client.keys and restart it
Configuration: ```console ubuntu ``` Cluster state before restart agent: ```console # /var/ossec/bin/cluster_control -a ID NAME IP STATUS VERSION NODE NAME 000 ip-XX.XX.XX.XX XX.XX.XX.XX active Wazuh v4.8.0 master 001 ip-XX.XX.XX.XX XX.XX.XX.XX active Wazuh v4.8.0 worker-2 002 DESKTOP-AQ2R8SM XX.XX.XX.XX active Wazuh v4.8.0 worker-1 ``` ```console # systemctl restart wazuh-agent ``` Cluster state: ```console # /var/ossec/bin/cluster_control -a ID NAME IP STATUS VERSION NODE NAME 001 ubuntu 172.31.47.83 active Wazuh v4.8.0 worker-1 002 DESKTOP-AQ2R8SM 172.31.86.228 active Wazuh v4.8.0 worker-1 ```
Total vulnerabilities
```console # curl -k -u USER:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 893, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Ubuntu agent vulnerabilities
```console # curl -k -u USER:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "001" } } }' { "count" : 118, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Windows agent vulnerabilities
There is 25 more vulns, 24 from the node package and 1 from npm which is installed with node. ```console # curl -k -u adUSERmin:PASS https://XX.XX.XX.XX:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "002" } } }' { "count" : 676, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```

wazuh-states-vulnerabilities.json

Rebits commented 6 months ago

No duplicated vulnerabilities :green_circle:

The vulnerabilities in the tested cases (Case 1-13) are all unique; none are duplicated. This has been confirmed through testing utilizing the QA framework tools integrated into the following script.

check_dupliacted.py ```python import json from wazuh_testing.end_to_end.check_validators import get_duplicated_vulnerabilities from wazuh_testing.end_to_end.vulnerability_detector import get_vulnerabilities_from_states vulnerabilities = [] with open("vulnerabilities.json", "r") as f: vulnerabilities = json.load(f)['hits']['hits'] vuln_agent = {} parsed_vulns = {} for vuln in vulnerabilities: agent_id = vuln['_source']['agent']['id'] if agent_id not in vuln_agent: vuln_agent[agent_id] = [] vuln_agent[agent_id].append(vuln) for agent, vulns in vuln_agent.items(): parsed_vulns[agent] = get_vulnerabilities_from_states(vulns) duplicated = get_duplicated_vulnerabilities(parsed_vulns) print(duplicated) ```
santipadilla commented 6 months ago

Case 16: Restart of the master node :green_circle:

Given a configured master And two workers configured And two indexers configured And both agents connected to the same worker And both agents with state Active When restart of the master node Then No doubling of the vulnerability index

Both agents connected to the same worker
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# /var/ossec/bin/cluster_control -a ID NAME IP STATUS VERSION NODE NAME 000 ip-xxx-xx-xx-xxx 127.0.0.1 active Wazuh v4.8.0 node01 002 DESKTOP-AQ2R8SM xxx-xx-xx-xxx active Wazuh v4.8.0 worker1-node 007 ip-xxx-xx-xx-xxx xxx-xx-xx-xxx active Wazuh v4.8.0 worker1-node ```

Vulnerabilities before master node restart

Total vulnerabilities
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx-xx-xx-xxx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 728, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Ubuntu agent vulnerabilities
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx-xx-xx-xxx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "007" } } }' { "count" : 118, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Windows agent vulnerabilities
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx-xx-xx-xxx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "002" } } }' { "count" : 511, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Master node restart
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# sudo systemctl restart wazuh-manager ```

Vulnerabilities after master node restart

Total vulnerabilities
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx.xx.xx.xx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 733, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Ubuntu agent vulnerabilities
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx-xx-xx-xxx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "007" } } }' { "count" : 118, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Windows agent vulnerabilities
```console root@ip-xxx-xx-xx-xxx:/home/ubuntu# curl -k -u user:pass https://xxx-xx-xx-xxx:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "002" } } }' { "count" : 511, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```

vulns.json

juliamagan commented 6 months ago

Case 17: Loss of connection with the master node :green_circle:

Given a configured master And two workers configured And both agents connected to the same worker And both agents with state Active When the connection with the master node is lost Then vulnerabilities are not duplicated

Configured cluster
```console root@ip-X-X-X-x:/home/ubuntu# /var/ossec/bin/cluster_control -a ID NAME IP STATUS VERSION NODE NAME 000 ip-X-X-X-X 127.0.0.1 active Wazuh v4.8.0 node01 001 DESKTOP-AQ2R8SM X-X-X-X active Wazuh v4.8.0 node02 002 ip-X-X-X-X X-X-X-X active Wazuh v4.8.0 node02 ```
Total vulnerabilities before
```console root@ip-X-X-X-x:/home/ubuntu# curl -k -u admin:admin https://X-X-X-x:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 683, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Ubuntu agent vulnerabilities before
```console root@ip-X-X-X-x:/home/ubuntu# curl -k -u admin:admin https://172.31.35.40:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "002" } } }' { "count" : 98, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Windows agent vulnerabilities before
```console root@ip-X-X-X-x:/home/ubuntu# curl -k -u admin:admin https://X-X-X-x:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "001" } } }' { "count" : 486, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```

index.json

Total vulnerabilities during reconnect
```console root@ip-X-X-X-x:/home/ubuntu# curl -k -u admin:admin https://X-X-X-x:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 683, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```

index_while_deny.json

Total vulnerabilities after
```console root@ip-X-X-X-x:/home/ubuntu# curl -k -u admin:admin https://X-X-X-x:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 683, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```

index_after.json

juliamagan commented 6 months ago

Case 18: While the agent is switching workers, the target worker is unavailable :green_circle:

Given a configured master And two workers configured And an each agent connects to one worker And the agents with state Active When the agent changes worker And the target worker is unavailable Then the agent connects to the worker when its available And vulnerabilities are updated and not duplicated

Configured cluster
```console root@ip-X-X-X-x:/home/ubuntu# /var/ossec/bin/cluster_control -a ID NAME IP STATUS VERSION NODE NAME 000 ip-X-X-X-x 127.0.0.1 active Wazuh v4.8.0 node01 001 DESKTOP-AQ2R8SM X-X-X-x active Wazuh v4.8.0 node03 002 ip-X-X-X-X X-X-X-X active Wazuh v4.8.0 node02 ```
Total vulnerabilities before
```console root@ip-X-X-X-x:/home/ubuntu# curl -k -u admin:admin https://X-X-X-x:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 683, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Ubuntu agent vulnerabilities before
```console root@ip-X-X-X-x:/home/ubuntu# curl -k -u admin:admin https://X-X-X-x:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "002" } } }' { "count" : 98, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Windows agent vulnerabilities before
```console root@ip-X-X-X-x:/home/ubuntu# curl -k -u admin:admin https://X-X-X-x:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "001" } } }' { "count" : 486, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```

index.json

Total vulnerabilities during reconnect
```console root@ip-X-X-X-x:/home/ubuntu# curl -k -u admin:admin https://X-X-X-x:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 683, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```

index_while_deny.json

Total vulnerabilities after
```console root@ip-X-X-X-x:/home/ubuntu# curl -k -u admin:admin https://X-X-X-x:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 683, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```

index_after.json

juliamagan commented 6 months ago

Case 19: While the agent is switching workers, the target worker is unavailable and the first worker is stopped :green_circle:

Given a configured master And two workers configured And both agents connected to the same worker And both agents with state Active When the agent changes worker And the target worker is unavailable Then the agent connects to the worker when its available And vulnerabilities are updated and not duplicated

Configured cluster
```console root@ip-X-X-X-x:/home/ubuntu# /var/ossec/bin/cluster_control -a ID NAME IP STATUS VERSION NODE NAME 000 ip-X-X-X-X 127.0.0.1 active Wazuh v4.8.0 node01 001 DESKTOP-AQ2R8SM X-X-X-X active Wazuh v4.8.0 node03 002 ip-X-X-X-X X-X-X-X active Wazuh v4.8.0 node03 ```
Total vulnerabilities before
```console root@ip-X-X-X-x:/home/ubuntu# curl -k -u admin:admin https://X-X-X-x:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 683, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Ubuntu agent vulnerabilities before
```console root@ip-X-X-X-x:/home/ubuntu# curl -k -u admin:admin https://172.31.35.40:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "002" } } }' { "count" : 98, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```
Windows agent vulnerabilities before
```console root@ip-X-X-X-x:/home/ubuntu# curl -k -u admin:admin https://X-X-X-x:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' -d '{ "query": { "term": { "agent.id": "001" } } }' { "count" : 486, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```

index.json

Total vulnerabilities during reconnect
```console root@ip-X-X-X-x:/home/ubuntu# curl -k -u admin:admin https://X-X-X-x:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 683, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```

index_while_deny.json

Total vulnerabilities after worker started
```console root@ip-X-X-X-x:/home/ubuntu# curl -k -u admin:admin https://X-X-X-x:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 679, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```

index_after.json

Total vulnerabilities after worker started
```console root@ip-X-X-X-x:/home/ubuntu# curl -k -u admin:admin https://X-X-X-x:9200/wazuh-states-vulnerabilities/_count?pretty=true -H 'Content-Type: application/json' { "count" : 683, "_shards" : { "total" : 1, "successful" : 1, "skipped" : 0, "failed" : 0 } } ```

index_after_worker_started.json

Rebits commented 6 months ago

Vulnerability discrepancies :yellow_circle:

Between case1 and case 5, there are different vulnerabilities:

Case 1

       [
            "CVE-2022-28948",
            "snapd",
            "2.62",
            ""
        ],

Case 5

        [
            "CVE-2022-28948",
            "snapd",
            "2.61.2",
            ""
        ],

It seems that snapd was upgraded during the testing.

The vulnerability occurrence in the remaining cases aligns with expectations. There are differences between test cases due to variations in the installed vulnerable packages.

MARCOSD4 commented 6 months ago

Moved ETA to allow final review

davidjiglesias commented 6 months ago

LGTM