E2E tests: Local debugging and testing

[juliamagan]

Description

After the development of the tests in #2872 and the improvements made in later tasks, the tests are ready to be merged into the numbered branch, but first, we must make sure that they are fully functional and there are no errors. To do this, all the people involved in this development must launch several runs of these tests and that the result of all of them is successful.

Previous requirements

Environment

• CentOS manager (all-in-one)
• CentOS agent
• Ubuntu agent
• Windows agent (with Office installed)

Inventory

inventory.yaml

``` managers: hosts: centos-manager: ansible_host: ansible_connection: ssh ansible_user: ansible_ssh_private_key_file: ansible_python_interpreter: /usr/bin/python3 dashboard_user: dashboard_password: agents: children: linux: hosts: ubuntu-agent: ansible_host: ansible_connection: ssh ansible_user: ansible_ssh_private_key_file: ansible_python_interpreter: /usr/bin/python3 centos-agent: ansible_host: ansible_connection: ssh ansible_user: ansible_ssh_private_key_file: ansible_python_interpreter: /usr/bin/python3 windows: hosts: windows-agent: ansible_host: ansible_user: ansible_password: ansible_connection: winrm ansible_winrm_server_cert_validation: ignore ansible_winrm_transport: basic ansible_winrm_port: 5985 (Only in local executions) ansible_python_interpreter: C:\Users\vagrant\AppData\Local\Programs\Pyhton\Python39\python.exe all: vars: virustotal_key: bucket_name: aws_region: aws_access_key_id: aws_secret_access_key: web_hook_url: slack_token: slack_channel: s3_url: (Should be always present) ```

Packages

Type	OS	Link
Manager	CentOS	https://packages-dev.wazuh.com/warehouse/test/4.4/rpm/var/wazuh-manager-4.4.0-3190.e2e.x86_64.rpm
Indexer	CentOS	https://packages-dev.wazuh.com/staging/yum/wazuh-indexer-4.4.0-0.40400.20220808.x86_64.rpm
Dashboard	CentOS	https://packages-dev.wazuh.com/staging/yum/wazuh-dashboard-4.4.0-0.40400.20220808.x86_64.rpm
Agent	CentOS	https://packages-dev.wazuh.com/warehouse/test/4.4/rpm/var/wazuh-agent-4.4.0-3190.e2e.x86_64.rpm
Agent	Ubuntu	https://packages-dev.wazuh.com/warehouse/test/4.4/deb/var/wazuh-agent_4.4.0-3190.e2e_amd64.deb
Agent	Windows	https://packages-dev.wazuh.com/warehouse/test/4.4/windows/wazuh-agent-4.4.0-3190.e2e.msi

Execution

python -m pytest tests/end_to_end/ --inventory_path=<INVENTORY_PATH> --html=R${i}-3166-e2e-<TESTER>.html --self-contained-html

Color	Status
🟢	All tests passed successfully
🟡	All tests passed but there are some warnings
🔴	Some tests have failures or errors
🔵	Test execution in progress
⚫	To Do

Proposed use cases

• [x] (T1): Run a test using an invalid hostname in the inventory
• [x] (T2): Run a test using an invalid IP in the inventory
• [x] (T3): Run a test using an invalid indexer password in the inventory
• [x] (T4): Run 2 tests with different environments (E.g: 1 test requires a Ubuntu agent, 1 test requires a CentOS agent)
• [x] (T5): Run 2 tests with the same environment (E.g: Both tests require a CentOS agent)
• [x] (T6): Run 1 test (with more than 1 test case) selecting 1 test case (-k)
• [x] (T7): Run all tests but select only 1 test (-k)
• [x] (T8): Run a test with an unsupported Linux distribution installed in a target host (E.g: Prepare a CentOS agent and in the inventory indicate that this agent is Ubuntu when the validation is executed it should fail.)
• [x] (T9): Run a test in a target host with an incorrect OS (E.g.: In the inventory define a Windows agent but prepare a CentOS machine.)
• [x] (T10): Run a test in a target host with the manager service stopped
• [x] (T11): Run all test cases from the E2E folder
• [x] (T12): Run all tests that require Linux
• [x] (T13): Run all tests that require Windows
• [x] (T14): Run a test without specifying an inventory
• [x] (T15): Run all tests but unselect one (-k not)

Open issues

• 3199

• 3203

• 3204

• 3205

• 3207

• 3208

• 3209

• 3210

• 3211

• 3212

• 3213

• 3219

[mauromalara]

T1: Run a test using an invalid hostname in the inventory 🔴

Conclusion

Tester	Test	OS	Results
@mauromalara	test_suricata_integration	CentOS and Ubuntu	🔴 🔴 🔴
@BelenValdivia	test_sql_injection	CentOS	🔴 🔴 🔴
@juliamagan	test_audit	CentOS	🔴 🔴 🔴

---

Tester: @BelenValdivia :red_circle:

1. Force the error by modifying the inventory: ``` . . . agent-centos: <---- FORCING THE ERROR (expected by E2E tests: centos-agent) ansible_host: XX.XX.XX.XX ansible_connection: ssh ansible_user: ansible_ssh_private_key_file: . . . ``` 2. One of the tests that use this agent is `test_sql_injection`, so I will run this test as follows: ``` python -m pytest --tb=short tests/end_to_end/test_basic_cases/test_sql_injection/ --html=T1-3166-e2e-belenvaldivia.html --inventory_path /home/belen/inventory.yaml ``` *** ### Result 🔴 The test fails but the validation stage does not check that the name of the hosts follows the E2E standard: `-`, e.g.: `centos-agent` ***

Tester: @juliamagan 🔴

1. Force the error by modifying the inventory: ``` testing: <---- FORCING THE ERROR (expected by E2E tests: managers) hosts: centos-manager: . . . ``` 2. One test that uses `managers` is `test_audit`: ``` python -m pytest tests/end_to_end/test_basic_cases/test_audit/ --inventory_path=/home/juliamagan/Desktop/QA/2893/inventory.yml --html=3166-T1-R${i}-e2e-juliamagan.html --self-contained-html ``` *** ### Result 🔴 The test fails because the group name is invalid, but it wasn't checked.

Tester: @mauromalara 🔴

1. Force the error by modifying the inventory: ``` . . . agent-ubuntu: <---- FORCING THE ERROR (expected by E2E tests: ubuntu-agent) ansible_host: 172.31.5.95 ansible_connection: ssh ansible_user: qa ansible_ssh_private_key_file: /home/mauro/ephemeral.pem . . manager: <---- FORCING THE ERROR (expected: "managers") hosts: centos-manager: ``` 2. One of the tests that use this agent is `test_suricata_integration`, so I will run this test as follows: ``` python -m pytest --tb=short tests/end_to_end/test_basic_cases/test_suricata_integration/ --inventory_path /home/mauro/inventory.yaml --html /home/mauro/T1-3166-e2e-mauromalara.html --self-contained-html ``` *** ### Result 🔴 The test fails but the validation stage does not check that the name of the hosts follows the E2E standard: - inventory hostname: `-`, e.g.: `ubuntu-agent` - groups: [managers,agents] ***

[BelenValdivia]

T2: Run a test using an invalid IP in the inventory 🟡

Conclusion

Tester	Test	OS	Results
@mauromalara	test_brute_force_rdp	CentOS and Windows	🟡 🟡 🟡
@BelenValdivia	test_sql_injection	CentOS	🟡 🟡 🟡
@juliamagan	test_audit	CentOS	🟡 🟡 🟡

---

Tester: @BelenValdivia 🟡

1. Force the error by modifying the inventory: ``` . . . centos-agent: ansible_host: XX.XX.XX.XX <---- FORCING THE ERROR (invalid IP) ansible_connection: ssh ansible_user: ansible_ssh_private_key_file: . . . ``` 2. One of the tests that use this agent is `test_sql_injection`, so I will run this test as follows: ``` python -m pytest --tb=short tests/end_to_end/test_basic_cases/test_sql_injection/ --html=T2-3166-e2e-belenvaldivia.html --inventory_path /home/belen/inventory.yaml ``` *** ### Result 🟢 The validation stage checks that the IP of the host is invalid

Tester: @juliamagan 🟡

1. Force the error by modifying the inventory: ``` managers: hosts: centos-manager: ansible_host: 1.1.1.1 <---- FORCING THE ERROR (invalid IP) . . . ``` 2. One of the tests that use this agent is `test_audit`, so I will run this test as follows: ``` python -m pytest tests/end_to_end/test_basic_cases/test_audit/ --inventory_path=/home/juliamagan/Desktop/QA/2893/inventory.yml --html=3166-T2-R${i}-e2e-juliamagan.html --self-contained-html ``` *** ### Result 🟡 The validation stage checks that the IP of the host is invalid, but the error is not correctly displayed.

Tester: @mauromalara 🟡

1. Force the error by modifying the inventory: ``` agents: hosts: windows-agent: ansible_host: 172.31.10.1 <--- FORCING THE ERROR (non-existent IP) . . . managers: hosts: centos-manager: ansible_host: 172.31.3.228 <--- FORCING THE ERROR (non-existent IP) ``` 2. One of the tests that uses `test_brute_force_rdp` uses that agent and manager, so I will run that test as follows: ``` for i in {1..3}; do python -m pytest tests/end_to_end/test_basic_cases/test_brute_force/test_brute_force_rdp/ --inventory_path=/home/mauro/inventory.yaml --html=3166-T2-R${i}-e2e-mauromalara.html --self-contained-html; done ``` *** ### Result 🟡 The validation stage checks that the IP of the host is invalid, but the error is not correctly displayed. ***

[BelenValdivia]

T3: Run a test using an invalid indexer password in the inventory 🟡

Conclusion

Tester	Test	OS	Results
@mauromalara	test_docker_monitoring	CentOS	🟡 🟡 🟡
@BelenValdivia	test_audit	CentOS	🟡 🟡 🟡
@juliamagan	test_detecting_suspicious_binaries	CentOS	🟡 🟡 🟡

---

Tester: @BelenValdivia 🟡

1. Force the error by modifying the indexer password in the inventory: ``` . . . centos-manager: ansible_host: XX.XX.XX.XX ansible_connection: ssh ansible_user: ansible_ssh_private_key_file: /home/belen/ZZZZ.pem dashboard_user: dashboard_password: <---- FORCING THE ERROR (invalid indexer password) . . . ``` 2. Test executed `test_audit`, I will run this test as follows: ``` python -m pytest --tb=short tests/end_to_end/test_basic_cases/test_audit/ --html=T3-3166-e2e-belenvaldivia.html --inventory_path /home/belen/inventory.yaml ``` *** ### Result 🟡 401 Unauthorized response from indexer. We could check this in the validation phase.

Tester: @juliamagan 🟡

1. Force the error by modifying the indexer password in the inventory: ``` . . . centos-manager: dashboard_password: <---- FORCING THE ERROR (invalid indexer password) . . . ``` 2. Test executed `test_detecting_supsicious_binaries`, I will run this test as follows: ``` python -m pytest tests/end_to_end/test_basic_cases/test_detecting_suspicious_binaries/ --inventory_path=/home/juliamagan/Desktop/QA/2893/inventory.yml --html=3166-T3-R${i}-e2e-juliamagan.html --self-contained-html ``` *** ### Result 🟡 The API request fails as expected, but maybe, we could check this in the validation phase.

Tester: @mauromalara 🟡

1. Forcing the error by modifying the inventory: ``` managers: hosts: centos-manager: dashboard_user: admin dashboard_password: bad_pass <--- FORCING THE ERROR ``` 2. Run `test_docker_monitoring` as follows: ``` for i in {1..3}; do python -m pytest tests/end_to_end/test_basic_cases/test_docker_monitoring/ --inventory_path=/home/mauro/inventory.yaml --html=3166-T3-R${i}-e2e-mauromalara.html --self-contained-html; done ``` *** ### Result 🟡 The API request fails as expected, but maybe, we could check this in the validation phase. ***

[BelenValdivia]

T4: Run 2 tests with different environments 🟢

Conclusion

Tester	Test	OS	Results
@mauromalara	test_unauthorized_processes_detection and test_fim_windows	CentOS and Windows	🟢 🟢 🟢
@BelenValdivia	test_audit and test_windows_defender	CentOS and Windows	🟢 🟢 🟢
@juliamagan	test_detecting_suspicious_binaries and test_ip_reputation	CentOS and Windows	🟢 🟢 🟢

---

Tester: @BelenValdivia 🟢

1. Execute 1 test that needs centos-agent and 1 tests that needs windows-agent - Tests to run: - Test Audit (centos-agent) - Test Windows Defender (windows agent) - I will run these tests as follows: ``` python -m pytest --tb=short tests/end_to_end/test_basic_cases/test_audit/ tests/end_to_end/test_basic_cases/test_windows_defender/ --html=T4-3166-e2e-belenvaldivia.html --inventory_path /home/belen/inventory.yaml ``` *** ### Result 🟢 Both tests passed

Tester: @juliamagan 🟢

1. Execute 1 test that needs centos-manager and 1 tests that needs windows-agent - Tests to run: - `test_detectiong_suspicious_binaries` (centos-manager) - `test_ip_reputation` (centos-manager and windows-agent) - I will run these tests as follows: ``` python -m pytest tests/end_to_end/test_basic_cases/test_detecting_suspicious_binaries/ tests/end_to_end/test_basic_cases/test_ip_reputation/ --inventory_path=/home/juliamagan/Desktop/QA/2893/inventory.yml --html=3166-T4-R${i}-e2e-juliamagan.html --self-contained-html ``` *** ### Result 🟢 Both tests passed

Tester: @mauromalara 🟢

1. Run the tests as follows: ``` for i in {1..3}; do python -m pytest tests/end_to_end/test_basic_cases/test_fim/test_fim_windows/ tests/end_to_end/test_basic_cases/test_unauthorized_processes_detection/ --inventory_path=/home/mauro/inventory.yaml --html=3166-T3-R${i}-e2e-mauromalara.html --self-contained-html; done ``` *** ### Result :green_circle: All tests have passed successfully. ***

[BelenValdivia]

T5: Run 2 tests with the same environment 🟢

Conclusion

Tester	Test	OS	Results
@mauromalara	test_brute_force_rdp and test_fim_windows	CentOS and Windows	🟢 🟢 🟢
@BelenValdivia	test_emotet and test_windows_defender	CentOS and Windows	🟢 🟢 🟢
@juliamagan	test_ip_reputation and test_windows_defender	CentOS and Windows	🟢 🟢 🟢

---

Tester: @BelenValdivia 🟢

1. **Execute 2 tests that need Windows agent and CentOS manager** - Tests to run: - Test Emotet - Test Windows defender - I will run these tests as follows: ``` python -m pytest --tb=short tests/end_to_end/test_basic_cases/test_audit/ tests/end_to_end/test_basic_cases/test_sql_injection/ --html=T5-3166-e2e-belenvaldivia.html --inventory_path /home/belen/inventory.yaml ``` *** ### Result 🟢 Both tests passed. Report: [T5-3166-e2e-belenvaldivia.zip](https://github.com/wazuh/wazuh-qa/files/9427623/T5-3166-e2e-belenvaldivia.zip)

Tester: @juliamagan 🟢

1. Run two tests with the same environment: - `test_ip_reputation` (centos-manager and windows-agent) - `test_windows_defender` (centos-manager and windows-agent) 2. I will run the tests as follows: ``` python -m pytest tests/end_to_end/test_basic_cases/test_windows_defender/ tests/end_to_end/test_basic_cases/test_ip_reputation/ --inventory_path=/home/juliamagan/Desktop/QA/2893/inventory.yml --html=3166-T5-R${i}-e2e-juliamagan.html --self-contained-html ``` *** ### Result 🟢 Both tests passed.

Tester: @mauromalara 🟢

1. Run the tests as follow: ``` for i in {1..3}; do python -m pytest tests/end_to_end/test_basic_cases/test_fim/test_fim_windows/ tests/end_to_end/test_basic_cases/test_brute_force/test_brute_force_rdp/ --inventory_path=/home/mauro/inventory.yaml --html=3166-T3-R${i}-e2e-mauromalara.html --self-contained-html; done ``` *** ### Result 🟢 Both tests have passed. ***

[mauromalara]

T6: Run 1 test (with more than 1 test case) selecting 1 test case (-k) 🔴

Conclusion

Tester	Test	OS	Results
@mauromalara	test_fim -k delete_file_linux	CentOS	🔴 🔴 🔴
@BelenValdivia	test_virustotal_integration	CentOS	🟢 🟢 🟢
@juliamagan	test_docker_monitoring	CentOS	🟢 🟢 🟢

Tester: @BelenValdivia 🟢

1. Test with multiple cases to run `test_virustotal_integration` 2. Run one test case: ``` python -m pytest tests/end_to_end/test_basic_cases/test_virustotal_integration/ -k remove_malicious_file --inventory_path=/home/belen/inventory.yml --html=3166-T6-R${i}-e2e-belenvaldivia.html --self-contained-html ``` ### Result 🟢 The specific test case passed

Tester: @juliamagan 🟢

1. Choose a test with multiple test cases: `test_docker_monitoring` 2. Run one test case: ``` python -m pytest tests/end_to_end/test_basic_cases/test_docker_monitoring/ -k docker_pull --inventory_path=/home/juliamagan/Desktop/QA/2893/inventory.yml --html=3166-T6-R${i}-e2e-juliamagan.html --self-contained-html ``` *** ### Result 🟢 The specific test case passed

Tester: @mauromalara 🔴

1. Choose a test with multiple test cases: `test_fim` 2. Run one test case: ``` for i in {1..3}; do python -m pytest tests/end_to_end/test_basic_cases/test_fim/ -k delete_file_linux --inventory_path=/home/mauro/inventory.yaml --html=3166-T3-R${i}-e2e-mauromalara.html --self-contained-html; done ``` ### Result 🔴 The test has failed, the failure must be researched.

[mauromalara]

T7: Run all tests but select only 1 test (-k) 🟢

Conclusion

Tester	Test	OS	Results
@mauromalara	test_sql_injection	CentOS	🟢 🟢 🟢
@BelenValdivia	test_suricata_integration	CentOS	🟢 🟢 🟢
@juliamagan	test_audit	CentOS	🟢 🟢 🟢

Tester: @BelenValdivia 🟢

1. Launch all tests but select just one: `test_suricata_integration` 2. Command: ``` python -m pytest tests/end_to_end/test_basic_cases/ -k suricata_integration --inventory_path=/home/belen/inventory.yml --html=3166-T7-R${i}-e2e-belenvaldivia.html --self-contained-html ``` *** ### Result 🟢 The specific test passed

Tester: @juliamagan 🟢

1. Launch all tests but select just one: `test_audit` 2. Command: ``` python -m pytest tests/end_to_end/test_basic_cases/ -k audit --inventory_path=/home/juliamagan/Desktop/QA/2893/inventory.yml --html=3166-T7-R${i}-e2e-juliamagan.html --self-contained-html ``` *** ### Result 🟢 The specific test passed

Tester: @mauromalara 🟢

1. Launch all tests but select just one: `test_sql_injection`: ``` for i in {1..3}; do python -m pytest tests/end_to_end/test_basic_cases/ -k sql_injection --inventory_path=/home/mauro/inventory.yaml --html=3166-T7-R${i}-e2e-mauromalara.html --self-contained-html; done ``` ### Result 🟢 The specific test passed

[mauromalara]

T8: Run a test with an unsupported Linux distribution installed in a target host 🔴

Conclusion

Tester	Test	OS	Results
@mauromalara	test_brute_force_ssh	CentOS	🟢 🟢 🟢
@BelenValdivia	test_sql_injection	CentOS	🟢 🟢 🟢
@juliamagan	test_suricata_integration	CentOS and Ubuntu	🔴 🔴 🔴

Tester: @BelenValdivia 🟢

1. Force the error. Define a Centos agent in the Inventory but the host is a Ubuntu agent. ``` agents: children: linux: hosts: centos-agent: ---> FORCE ERROR ansible_host: ansible_user: ansible_connection: ssh ansible_ssh_private_key_file: /home/belen/zzzzz.pem ``` 2. Test executed `test_sql_injection`, I will run this test as follows: ``` python -m pytest tests/end_to_end/test_basic_cases/test_sql_injection/ --inventory_path=/home/belen/inventory.yml --html=3166-T9-R${i}-e2e-belenvaldivia.html --self-contained-html ``` *** ### Result 🟢 The validation stage checks the incorrect Linux distribution installed in a target host.

Tester: @juliamagan 🔴

1. The `test_suricata_integration` test needs an Ubuntu agent, we have configured a CentOS agent instead: ``` agents: children: linux: hosts: ubuntu-agent: ---> FORCE ERROR ansible_host: ``` 2. Run the test as follows: ``` python -m pytest tests/end_to_end/test_basic_cases/test_suricata_integration/ --inventory_path=/home/juliamagan/Desktop/QA/2893/inventory.yml --html=3166-T8-R${i}-e2e-juliamagan.html --self-contained-html ``` *** ### Result 🔴 The distribution wasn't checked in the validation phase, so the test was launched and failed.

Tester: @mauromalara 🟢

1. The `test_brute_force_ssh` test needs a CentOS agent, we have configured an Ubuntu agent instead: ``` centos-agent: ansible_host: 172.31.13.31 <--- FORCING THE ERROR (ubuntu agent IP) ``` 2. Run the test ### Result 🟢 The validation works as expected.

[mauromalara]

T9: Run a test in a target host with an incorrect OS 🟡

Conclusion

Tester	Test	OS	Results
@mauromalara	test_emotet	Centos and Windows	🟡 🟡 🟡
@BelenValdivia	test_sql_injection	CentOS	🟢 🟢 🟢
@juliamagan	test_windows_defender	CentOS and Windows	🟢 🟢 🟢

Tester: @BelenValdivia 🟢

1. Force the error in the Inventory. Define a CentOS agent but the host is a Windows agent ``` agents: hosts: centos-agent: ---> FORCE ERROR ansible_host: XX.XX.XX.XX ---> windows agent ansible_user: ansible_connection: ssh ansible_ssh_private_key_file: /home/belen/zzzzz.pem ``` 2. Test executed `test_sql_injection`, I will run this test as follows: ``` python -m pytest tests/end_to_end/test_basic_cases/test_sql_injection/ --inventory_path=/home/belen/inventory.yml --html=3166-T9-R${i}-e2e-belenvaldivia.html --self-contained-html ``` *** ### Result 🟢 The test fails but the validation stage does not check the incorrect OS defined in the inventory

Tester: @juliamagan 🟢

1. The `test_windows_defender` test needs a Windows agent, we have configured an Ubuntu agent instead: ``` agents: children: linux: hosts: windows-agent: ---> FORCE ERROR ansible_host: ``` 2. Run the test as follows: ``` python -m pytest tests/end_to_end/test_basic_cases/test_windows_defender/ --inventory_path=/home/juliamagan/Desktop/QA/2893/inventory.yml --html=3166-T9-R${i}-e2e-juliamagan.html --self-contained-html ``` *** ### Result 🟢 The OS was checked in the validation phase and failed, so the test wasn't launched.

Tester: @mauromalara 🟡

1. The `test_emotet` test needs a CentOS manager, so we force the error by modifying the inventory as follows: ``` managers: hosts: centos-manager: ansible_host: 172.31.11.210 <--- Force the error (Window agent IP) ``` ### Result 🟡 The validation phase fails, but it does not check the OS, instead, it fails while gathering Ansible facts.

[mauromalara]

T10: Run a test in a target host with the manager service stopped 🟡

Conclusion

🟡: All executions were run successfully, but @mauromalara detect that the validation stage does not include the validation for a Windows agent service.

---

Tester	Test	OS	Results
@mauromalara	test_yara_integration	CentOS	🟢 🟢 🟢
@BelenValdivia	test_detecting_suspicious_binaries	CentOS	🟢 🟢 🟢
@juliamagan	test_audit	CentOS	🟢 🟢 🟢

Tester: @BelenValdivia 🟢

1. Force the error by stopping the Wazuh manager. 2. Run `test_detecting_suspicious_binaries`, so I will run this test as follows: ``` python -m pytest tests/end_to_end/test_basic_cases/test_detecting_suspicious_binaries/ --html=T10-3166-e2e-belenvaldivia.html --inventory_path /home/belen/inventory.yaml ``` *** ### Result 🟢 The test fails, the validation stage checks that the Wazuh manager is stopped

Tester: @juliamagan 🟢

1. Force the error by stopping the manager. 2. Run the test as follows: ``` python -m pytest tests/end_to_end/test_basic_cases/test_audit/ --inventory_path=/home/juliamagan/Desktop/QA/2893/inventory.yml --html=3166-T10-R${i}-e2e-juliamagan.html --self-contained-html ``` *** ### Result 🟢 It is checked that the manager should be running correctly, so the test isn't launched.

Tester: @mauromalara 🟢

1. Stop the `wazuh-manager` service: `systemctl stop wazuh-manager` 2. Run the `test_yara_integration`, which requires 1 manager to run. ### Result 🟢

[juliamagan]

T11: Run all test cases from the E2E folder 🔴

Conclusion

Tester	Test	OS	Results
@mauromalara	end_to_end	CentOS, Ubuntu and Windows	🔴 🔴 🔴
@BelenValdivia	end_to_end	CentOS, Ubuntu and Windows	🔴 🔴 🔴
@juliamagan	test_basic_cases	CentOS, Ubuntu and Windows	🔴🔴🔴

Tester: @BelenValdivia 🔴

1. Launched every tests: ``` python -m pytest tests/end_to_end/test_basic_cases/ --inventory_path=/home/belen/inventory.yml --html=3166-T11-R${i}-e2e-juliamagan.html --self-contained-html ``` *** ### Result 🔴 The following tests should not have failed: - `test_brute_force_rdp` -> R1, R2, R3. Error: The alert has not ocurred - `test_virustotal_integration[remove_malicious_file]` -> R1. Error: Alert triggered but not indexed - `test_vulnerability_detector_windows` -> R1, R2, R3. Error: The alert has not ocurred. Bug opened - `test_vulnerability_detector_linux` -> R1, R2, R3. Error: The alert has not ocurred -` test_docker_monitoring[docker_pull]` -> R3. Error: The alert has not ocurred

Tester: @juliamagan 🔴

1. Launch every test ``` python -m pytest tests/end_to_end/test_basic_cases/ --inventory_path=/home/juliamagan/Desktop/QA/2893/inventory.yml --html=3166-T11-R${i}-e2e-juliamagan.html --self-contained-html ``` *** ### Result 🔴 The following tests should not have failed: - `test_aws_infrastructure_monitoring` -> R1 - `test_virustotal_integration[remove_malicious_file]` -> R1, R2, R3 - `test_vulnerability_detector`-> R1, R2, R3

Tester @mauromalara 🔴

1. In the root directory of the QA repo, go to `tests/end_to_end` 8. Run all tests as follows: ``` python -m pytest --tb=short ./ --inventory_path /home/mauro/inventory.yaml --html /home/mauro/T11-3166-e2e-mauromalara.html --self-contained-html ``` *** ### Result 🔴 The following tests have failed: - `test_vulnerability_detector_linux` - `test_vulnerability_detector_windows` - `test_brute_force_ssh[ssh_brute_force]` - `test_emotet[emotet_attack]` - `test_virustotal_integration[remove_malicious_file]` - `test_aws_infrastructure_monitoring[CloudTrail service]` ***

[juliamagan]

T12: Run all tests that require Linux 🔴

Conclusion

Tester	Test	OS	Results
@mauromalara	test_basic_cases -m linux	CentOS and Ubuntu	🔴 🔴 🔴
@BelenValdivia	test_basic_cases -m linux	CentOS and Ubuntu	🔴 🔴 🔴
@juliamagan	test_basic_cases -m linux	CentOS and Ubuntu	🔴 🔴 🔴

Tester: @BelenValdivia 🔴

1. Run all tests that require Linux as follows: ``` python -m pytest tests/end_to_end/test_basic_cases/ -m linux --inventory_path=/home/belen/inventory.yml --html=3166-T12-R${i}-e2e-belenvaldivia.html --self-contained-html ``` *** ### Result 🔴 We can't run our tests using markers because this feature isn't implemented yet.

Tester: @juliamagan 🔴

1. Run all tests that require Linux as follows: ``` python -m pytest tests/end_to_end/test_basic_cases/ -m linux --inventory_path=/home/juliamagan/Desktop/QA/2893/inventory.yml --html=3166-T12-R${i}-e2e-juliamagan.html --self-contained-html ``` *** ### Result 🔴 We can't run our tests using markers because this feature isn't implemented yet.

Tester: @mauromalara 🔴

1. Run all tests that require Linux as follows: ``` python -m pytest tests/end_to_end/test_basic_cases/ -m linux --inventory_path=/home/mauro/inventory.yaml ``` *** ### Result 🔴 We can't run our tests using markers because this feature isn't implemented yet.

[juliamagan]

T13: Run all tests that require Windows 🔴

Conclusion

Tester	Test	OS	Results
@mauromalara	test_basic_cases -m windows	CentOS and Windows	🔴 🔴 🔴
@BelenValdivia	test_basic_cases -m windows	CentOS and Windows	🔴 🔴 🔴
@juliamagan	test_basic_cases -m windows	CentOS and Windows	🔴🔴🔴

Tester: @BelenValdivia 🔴

1. Run all tests that require Windows as follows: ``` python -m pytest tests/end_to_end/test_basic_cases/ -m windows --inventory_path=/home/belen/inventory.yml --html=3166-T13-R${i}-e2e-belenvaldivia.html --self-contained-html ``` *** ### Result 🔴 We can't run our tests using markers because this feature isn't implemented yet.

Tester: @juliamagan 🔴

1. Run all tests that require Windows as follows: ``` python -m pytest tests/end_to_end/test_basic_cases/ -m windows --inventory_path=/home/juliamagan/Desktop/QA/2893/inventory.yml --html=3166-T13-R${i}-e2e-juliamagan.html --self-contained-html ``` *** ### Result 🔴 We can't run our tests using markers because this feature isn't implemented yet.

Tester: @mauromalara 🔴

1. Run all tests that require Windows as follows: ``` python -m pytest tests/end_to_end/test_basic_cases/ -m windows --inventory_path=/home/mauro/inventory.yaml ``` *** ### Result 🔴 We can't run our tests using markers because this feature isn't implemented yet.

[juliamagan]

T14: Run a test without specifying the inventory 🟢

Conclusion

Tester	Test	OS	Results
@mauromalara	all	CentOS, Ubuntu and Windows	🟢 🟢 🟢
@BelenValdivia	test_windows_defender	Windows	🟢 🟢 🟢
@juliamagan	test_audit	CentOS	🟢 🟢 🟢

Tester: @BelenValdivia 🟢

1. Run the test without specifying the inventory: ``` python -m pytest tests/end_to_end/test_basic_cases/test_windows_defender --html=3166-T14-R${i}-e2e-belenvaldivia.html --self-contained-html ``` *** ### Result 🟢 Nothing is launched because there is no inventory

Tester: @juliamagan 🟢

1. Run the test without specifying the inventory: ``` python -m pytest tests/end_to_end/test_basic_cases/test_audit/ --html=3166-T14-R${i}-e2e-juliamagan.html --self-contained-html ``` *** ### Result 🟢 Nothing is launched because there is no inventory

Tester: @mauromalara 🟢

1. Run the test without specifying the inventory: ``` python -m pytest tests/end_to_end/test_basic_cases/ ``` *** ### Result 🟢 No test was executed because there is no inventory.

[juliamagan]

T15: Run all tests but unselect tests (-k not) 🔴

Conclusion

Tester	Test	OS	Results
@mauromalara	test_basic_cases -k "not aws and not docker"	CentOS, Ubuntu and Windows	🔴 🔴 🔴
@BelenValdivia	test_basic_cases -k "not yara"	CentOS, Ubuntu and Windows	🔴 🔴 🔴
@juliamagan	test_basic_cases -k "not vulnerability"	CentOS, Ubuntu and Windows	🔴🔴🔴

Tester: @BelenValdivia 🔴

1. Launched every test except `test_yara_integration`: ``` python -m pytest tests/end_to_end/test_basic_cases/ -k "not yara" --inventory_path=/home/belen/inventory.yml --html=3166-T15-R${i}-e2e-belenvaldivia.html --self-contained-html ``` *** ### Result 🔴 The following tests should not have failed: - `test_brute_force_rdp` -> R1, R2, R3. Error: The alert has not ocurred - `test_virustotal_integration[remove_malicious_file]` -> R1. Error: The alert has not ocurred - `test_vulnerability_detector_windows` -> R1, R2, R3. Error: The alert has not ocurred. Bug opened - `test_vulnerability_detector_linux` -> R1, R2, R3. Error: The alert has not ocurred - `test_suricata_integration` -> R2. Error: The alert has not occurred

Tester: @juliamagan 🔴

1. Launched every test except `test_vulnerability_detector`: ``` python -m pytest tests/end_to_end/test_basic_cases/ -k "not vulnerability" --inventory_path=/home/juliamagan/Desktop/QA/2893/inventory.yml --html=3166-T15-R${i}-e2e-juliamagan.html --self-contained-html ``` *** ### Result 🔴 The following tests should not have failed: - `test_brute_force_ssh` -> R1 - `test_virustotal_integration[remove_malicious_file]` -> R1, R2 and R3 - `test_osquery_integration` -> R2 - `test_aws_infrastructure_monitoring` -> R3

Tester: @mauromalara 🔴

1. Run the tests as follows: `python -m pytest tests/end_to_end/test_basic_cases/ -k "not aws and not docker" --html=3166-T15-R${i}-e2e-mauromalara.html --inventory_path=/home/mauro/inventory.yaml --self-contained-html` ### Result 🔴 - test_brute_force_ssh - test_emotet - test_virustotal_integration (in R3 the test passed, maybe is the timeout) - test_vulnerability_detector_linux - test_vulnerability_detector_windows