search

wazuh / wazuh-qa

Wazuh - Quality Assurance
GNU General Public License v2.0
61 stars 30 forks source link

AWS integration tests #3911

Closed nico-stefani closed 1 year ago

nico-stefani commented 1 year ago
Related issue
#3333

Description

Added

Testing performed

Service Tier 0 Tier 1 Related Issues
CloudTrail :green_circle: :green_circle:
VPC :green_circle: :green_circle: wazuh/wazuh#16138 wazuh/wazuh#15763
Config :green_circle: :green_circle: wazuh/wazuh#16138
ALB :yellow_circle: :green_circle: wazuh/wazuh#15804
CLB :yellow_circle: :green_circle: wazuh/wazuh#15804
NLB :yellow_circle: :green_circle: wazuh/wazuh#15804
KMS :green_circle: :green_circle:
Macie :green_circle: :green_circle:
Trusted Advisor :green_circle: :green_circle:
GuardDuty :green_circle: :green_circle: wazuh/wazuh#16246
WAF :green_circle: :green_circle: wazuh/wazuh#16290 wazuh/wazuh#16246
Server Access :green_circle: :green_circle: wazuh/wazuh#16290 wazuh/wazuh#16246
Inspector :green_circle: :yellow_circle: wazuh/wazuh#16301 wazuh/wazuh#16303
CloudWatch :green_circle: :green_circle: wazuh/wazuh#16301 wazuh/wazuh#16375
Umbrella :green_circle: :green_circle: wazuh/wazuh#16246
Parser :green_circle: :green_circle: wazuh/wazuh#16383

Tier 0

root@ip-172-31-82-0:/home/ubuntu/wazuh-qa/tests/integration/test_aws# pytest --disable-warnings --tb=line --tier 0
================================================================================================ test session starts ================================================================================================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/ubuntu/wazuh-qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.4, html-3.1.1, testinfra-5.0.0
collected 195 items / 16 deselected / 179 selected

test_basic.py ................                                                                                                                                                                                [  8%]
test_discard_regex.py ..............                                                                                                                                                                          [ 16%]
test_log_groups.py ..                                                                                                                                                                                         [ 17%]
test_only_logs_after.py ...............................                                                                                                                                                       [ 35%]
test_parser.py ..........................                                                                                                                                                                     [ 49%]
test_path.py ..........................................                                                                                                                                                       [ 73%]
test_path_suffix.py .........                                                                                                                                                                                 [ 78%]
test_regions.py ........................                                                                                                                                                                      [ 91%]
test_remove_from_bucket.py ...sss.........                                                                                                                                                                    [100%]

============================================================================ 176 passed, 3 skipped, 16 deselected in 3645.39s (1:00:45) =============================================================================

Tier 1

root@ip-172-31-82-0:/home/ubuntu/wazuh-qa/tests/integration/test_aws# pytest --disable-warnings --tb=line --tier 1
================================================================================================ test session starts ================================================================================================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/ubuntu/wazuh-qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.4, html-3.1.1, testinfra-5.0.0
collected 195 items / 179 deselected / 16 selected

test_only_logs_after.py ..............x.                                                                                                                                                                      [100%]

============================================================================= 15 passed, 179 deselected, 1 xfailed in 362.17s (0:06:02) =============================================================================
nico-stefani commented 1 year ago

CloudTrail Results

Tier 0

============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 99 items / 85 deselected / 14 selected

test_basic.py .                                                          [  7%]
test_discard_regex.py .                                                  [ 14%]
test_only_logs_after.py ..                                               [ 28%]
test_path.py ...                                                         [ 50%]
test_path_suffix.py ...                                                  [ 71%]
test_regions.py ...                                                      [ 92%]
test_remove_from_bucket.py .                                             [100%]

=============================== warnings summary ===============================
../../../../../../usr/local/lib/python3.10/dist-packages/_pytest/cacheprovider.py:433
  /usr/local/lib/python3.10/dist-packages/_pytest/cacheprovider.py:433: PytestCacheWarning: cache could not write path /home/vagrant/qa/tests/integration/.pytest_cache/v/cache/nodeids
    config.cache.set("cache/nodeids", sorted(self.cached_nodeids))

../../../../../../usr/local/lib/python3.10/dist-packages/_pytest/stepwise.py:52
  /usr/local/lib/python3.10/dist-packages/_pytest/stepwise.py:52: PytestCacheWarning: cache could not write path /home/vagrant/qa/tests/integration/.pytest_cache/v/cache/stepwise
    session.config.cache.set(STEPWISE_CACHE_DIR, [])

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========== 14 passed, 85 deselected, 2 warnings in 309.55s (0:05:09) ===========

Tier 1

============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 81 items / 80 deselected / 1 selected

test_only_logs_after.py .                                                [100%]

=============================== warnings summary ===============================
================ 1 passed, 80 deselected, 2 warnings in 23.78s =================
nico-stefani commented 1 year ago

VPC Results

Tier 0

============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 99 items / 85 deselected / 14 selected

test_basic.py .                                                          [  7%]
test_discard_regex.py F                                                  [ 14%]
test_only_logs_after.py .F                                               [ 28%]
test_path.py F..                                                         [ 50%]
test_path_suffix.py F..                                                  [ 71%]
test_regions.py FFF                                                      [ 92%]
test_remove_from_bucket.py .                                             [100%]

=================================== FAILURES ===================================
____________________ test_discard_regex[vpc_discard_regex] _____________________
=========================== short test summary info ============================
FAILED test_discard_regex.py::test_discard_regex[vpc_discard_regex] - Timeout...
FAILED test_only_logs_after.py::test_with_only_logs_after[vpc_with_only_logs_after]
FAILED test_path.py::test_path[vpc_path_with_data] - TimeoutError: The AWS mo...
FAILED test_path_suffix.py::test_path_suffix[vpc_path_suffix_with_data] - Tim...
FAILED test_regions.py::test_regions[vpc_region_with_data] - TimeoutError: Th...
FAILED test_regions.py::test_regions[vpc_regions_with_data] - TimeoutError: T...
FAILED test_regions.py::test_regions[vpc_inexistent_region] - TimeoutError: T...
====== 7 failed, 7 passed, 85 deselected, 3 warnings in 433.79s (0:07:13) ======

Tier 1

============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 99 items / 98 deselected / 1 selected

test_only_logs_after.py F                                                [100%]

=================================== FAILURES ===================================
=========================== short test summary info ============================
FAILED test_only_logs_after.py::test_multiple_calls[vpc_only_logs_after_multiple_calls]
=========== 1 failed, 98 deselected, 3 warnings in 69.97s (0:01:09) ============
nico-stefani commented 1 year ago

Config Results

Tier 0

============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 99 items / 85 deselected / 14 selected

test_basic.py .                                                          [  7%]
test_discard_regex.py F                                                  [ 14%]
test_only_logs_after.py .F                                               [ 28%]
test_path.py F..                                                         [ 50%]
test_path_suffix.py ...                                                  [ 71%]
test_regions.py FFF                                                      [ 92%]
test_remove_from_bucket.py .                                             [100%]

=================================== FAILURES ===================================
___________________ test_discard_regex[config_discard_regex] ___________________
=========================== short test summary info ============================
FAILED test_discard_regex.py::test_discard_regex[config_discard_regex] - Time...
FAILED test_only_logs_after.py::test_with_only_logs_after[config_with_only_logs_after]
FAILED test_path.py::test_path[config_path_with_data] - TimeoutError: The AWS...
FAILED test_regions.py::test_regions[config_region_with_data] - TimeoutError:...
FAILED test_regions.py::test_regions[config_regions_with_data] - TimeoutError...
FAILED test_regions.py::test_regions[config_inexistent_region] - TimeoutError...
====== 6 failed, 8 passed, 85 deselected, 3 warnings in 431.28s (0:07:11) ======

Tier 1

============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 99 items / 98 deselected / 1 selected

test_only_logs_after.py F                                                [100%]

=================================== FAILURES ===================================
FAILED test_only_logs_after.py::test_multiple_calls[config_only_logs_after_multiple_calls]
=========== 1 failed, 98 deselected, 3 warnings in 60.59s (0:01:00) ============
nico-stefani commented 1 year ago

ALB Results

Tier 0

============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 99 items / 88 deselected / 11 selected

test_basic.py .                                                          [  9%]
test_discard_regex.py .                                                  [ 18%]
test_only_logs_after.py ..                                               [ 36%]
test_path.py ...                                                         [ 63%]
test_regions.py ...                                                      [ 90%]
test_remove_from_bucket.py s                                             [100%]

===== 10 passed, 1 skipped, 88 deselected, 2 warnings in 212.11s (0:03:32) =====

Tier 1

============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 99 items / 98 deselected / 1 selected

test_only_logs_after.py .                                                [100%]

================ 1 passed, 98 deselected, 2 warnings in 35.40s =================
nico-stefani commented 1 year ago

CLB Results

Tier 0

============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 99 items / 88 deselected / 11 selected

test_basic.py .                                                          [  9%]
test_discard_regex.py .                                                  [ 18%]
test_only_logs_after.py ..                                               [ 36%]
test_path.py ...                                                         [ 63%]
test_regions.py ...                                                      [ 90%]
test_remove_from_bucket.py s                                             [100%]

=============================== warnings summary ===============================
../../../../../../usr/local/lib/python3.10/dist-packages/_pytest/cacheprovider.py:433
  /usr/local/lib/python3.10/dist-packages/_pytest/cacheprovider.py:433: PytestCacheWarning: cache could not write path /home/vagrant/qa/tests/integration/.pytest_cache/v/cache/nodeids
    config.cache.set("cache/nodeids", sorted(self.cached_nodeids))

../../../../../../usr/local/lib/python3.10/dist-packages/_pytest/stepwise.py:52
  /usr/local/lib/python3.10/dist-packages/_pytest/stepwise.py:52: PytestCacheWarning: cache could not write path /home/vagrant/qa/tests/integration/.pytest_cache/v/cache/stepwise
    session.config.cache.set(STEPWISE_CACHE_DIR, [])

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
===== 10 passed, 1 skipped, 88 deselected, 2 warnings in 212.99s (0:03:32) =====

Tier 1

============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 99 items / 98 deselected / 1 selected

test_only_logs_after.py .                                                [100%]

================ 1 passed, 98 deselected, 2 warnings in 36.28s =================
nico-stefani commented 1 year ago

NLB Results

Tier 0

============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 99 items / 88 deselected / 11 selected

test_basic.py .                                                          [  9%]
test_discard_regex.py .                                                  [ 18%]
test_only_logs_after.py ..                                               [ 36%]
test_path.py ...                                                         [ 63%]
test_regions.py ...                                                      [ 90%]
test_remove_from_bucket.py s                                             [100%]

===== 10 passed, 1 skipped, 88 deselected, 2 warnings in 212.58s (0:03:32) =====

Tier 1

============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 99 items / 98 deselected / 1 selected

test_only_logs_after.py .                                                [100%]

=============================== warnings summary ===============================
================ 1 passed, 98 deselected, 2 warnings in 33.54s =================
nico-stefani commented 1 year ago

KMS Results

Tier 0

============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 99 items / 91 deselected / 8 selected

test_basic.py .                                                          [ 12%]
test_discard_regex.py .                                                  [ 25%]
test_only_logs_after.py ..                                               [ 50%]
test_path.py ...                                                         [ 87%]
test_remove_from_bucket.py .                                             [100%]

=============================== warnings summary ===============================
=========== 8 passed, 91 deselected, 2 warnings in 176.49s (0:02:56) ===========

Tier 1

============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 99 items / 98 deselected / 1 selected

test_only_logs_after.py .                                                [100%]

=============================== warnings summary ===============================
================ 1 passed, 98 deselected, 2 warnings in 38.15s =================
nico-stefani commented 1 year ago

Macie Results

Tier 0

root@ubuntu-jammy:/home/vagrant/qa/tests/integration/test_aws# pytest -k macie --tier 0
================================================================================================ test session starts ================================================================================================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 24 items / 16 deselected / 8 selected

test_basic.py .                                                                                                                                                                                               [ 12%]
test_discard_regex.py .                                                                                                                                                                                       [ 25%]
test_only_logs_after.py ..                                                                                                                                                                                    [ 50%]
test_path.py ...                                                                                                                                                                                              [ 87%]
test_remove_from_bucket.py .                                                                                                                                                                                  [100%]

================================================================================================= warnings summary ==================================================================================================
../../../../../../usr/local/lib/python3.10/dist-packages/_pytest/cacheprovider.py:433
  /usr/local/lib/python3.10/dist-packages/_pytest/cacheprovider.py:433: PytestCacheWarning: cache could not write path /home/vagrant/qa/tests/integration/.pytest_cache/v/cache/nodeids
    config.cache.set("cache/nodeids", sorted(self.cached_nodeids))

../../../../../../usr/local/lib/python3.10/dist-packages/_pytest/stepwise.py:52
  /usr/local/lib/python3.10/dist-packages/_pytest/stepwise.py:52: PytestCacheWarning: cache could not write path /home/vagrant/qa/tests/integration/.pytest_cache/v/cache/stepwise
    session.config.cache.set(STEPWISE_CACHE_DIR, [])

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================================================================= 8 passed, 16 deselected, 2 warnings in 164.31s (0:02:44) ==============================================================================

Tier 1

================================================================================================ test session starts ================================================================================================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 126 items / 125 deselected / 1 selected

test_only_logs_after.py .                                                                                                                                                                                     [100%]

================================================================================================= warnings summary ==================================================================================================
../../../../../../usr/local/lib/python3.10/dist-packages/_pytest/cacheprovider.py:433
  /usr/local/lib/python3.10/dist-packages/_pytest/cacheprovider.py:433: PytestCacheWarning: cache could not write path /home/vagrant/qa/tests/integration/.pytest_cache/v/cache/nodeids
    config.cache.set("cache/nodeids", sorted(self.cached_nodeids))

../../../../../../usr/local/lib/python3.10/dist-packages/_pytest/stepwise.py:52
  /usr/local/lib/python3.10/dist-packages/_pytest/stepwise.py:52: PytestCacheWarning: cache could not write path /home/vagrant/qa/tests/integration/.pytest_cache/v/cache/stepwise
    session.config.cache.set(STEPWISE_CACHE_DIR, [])

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================================================== 1 passed, 125 deselected, 2 warnings in 36.87s ===================================================================================
nico-stefani commented 1 year ago

Trusted Advisor Results

Tier 0

root@ubuntu-jammy:/home/vagrant/qa/tests/integration/test_aws# pytest -k trusted --tier 0
================================================================================================ test session starts ================================================================================================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 24 items / 16 deselected / 8 selected

test_basic.py .                                                                                                                                                                                               [ 12%]
test_discard_regex.py .                                                                                                                                                                                       [ 25%]
test_only_logs_after.py ..                                                                                                                                                                                    [ 50%]
test_path.py ...                                                                                                                                                                                              [ 87%]
test_remove_from_bucket.py .                                                                                                                                                                                  [100%]

================================================================================================= warnings summary ==================================================================================================
../../../../../../usr/local/lib/python3.10/dist-packages/_pytest/cacheprovider.py:433
  /usr/local/lib/python3.10/dist-packages/_pytest/cacheprovider.py:433: PytestCacheWarning: cache could not write path /home/vagrant/qa/tests/integration/.pytest_cache/v/cache/nodeids
    config.cache.set("cache/nodeids", sorted(self.cached_nodeids))

../../../../../../usr/local/lib/python3.10/dist-packages/_pytest/stepwise.py:52
  /usr/local/lib/python3.10/dist-packages/_pytest/stepwise.py:52: PytestCacheWarning: cache could not write path /home/vagrant/qa/tests/integration/.pytest_cache/v/cache/stepwise
    session.config.cache.set(STEPWISE_CACHE_DIR, [])

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================================================================= 8 passed, 16 deselected, 2 warnings in 171.01s (0:02:51) ==============================================================================

Tier 1

root@ubuntu-jammy:/home/vagrant/qa/tests/integration/test_aws# pytest --tier 1 -k trusted
================================================================================================ test session starts ================================================================================================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 126 items / 125 deselected / 1 selected

test_only_logs_after.py .                                                                                                                                                                                     [100%]

================================================================================================= warnings summary ==================================================================================================
../../../../../../usr/local/lib/python3.10/dist-packages/_pytest/cacheprovider.py:433
  /usr/local/lib/python3.10/dist-packages/_pytest/cacheprovider.py:433: PytestCacheWarning: cache could not write path /home/vagrant/qa/tests/integration/.pytest_cache/v/cache/nodeids
    config.cache.set("cache/nodeids", sorted(self.cached_nodeids))

../../../../../../usr/local/lib/python3.10/dist-packages/_pytest/stepwise.py:52
  /usr/local/lib/python3.10/dist-packages/_pytest/stepwise.py:52: PytestCacheWarning: cache could not write path /home/vagrant/qa/tests/integration/.pytest_cache/v/cache/stepwise
    session.config.cache.set(STEPWISE_CACHE_DIR, [])

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================================================== 1 passed, 125 deselected, 2 warnings in 34.30s ===================================================================================
nico-stefani commented 1 year ago

Guard Duty Results

Tier 0

root@ubuntu-jammy:/home/vagrant/qa/tests/integration/test_aws# pytest -k 'guardduty' --tier 0
============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 33 items / 17 deselected / 16 selected

test_basic.py ..                                                         [ 12%]
test_discard_regex.py ..                                                 [ 25%]
test_only_logs_after.py ....                                             [ 50%]
test_path.py ......                                                      [ 87%]
test_remove_from_bucket.py ..                                            [100%]

=============================== warnings summary ===============================
../../../../../../usr/local/lib/python3.10/dist-packages/_pytest/cacheprovider.py:433
  /usr/local/lib/python3.10/dist-packages/_pytest/cacheprovider.py:433: PytestCacheWarning: cache could not write path /home/vagrant/qa/tests/integration/.pytest_cache/v/cache/nodeids
    config.cache.set("cache/nodeids", sorted(self.cached_nodeids))

../../../../../../usr/local/lib/python3.10/dist-packages/_pytest/stepwise.py:52
  /usr/local/lib/python3.10/dist-packages/_pytest/stepwise.py:52: PytestCacheWarning: cache could not write path /home/vagrant/qa/tests/integration/.pytest_cache/v/cache/stepwise
    session.config.cache.set(STEPWISE_CACHE_DIR, [])

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
========== 16 passed, 17 deselected, 2 warnings in 333.61s (0:05:33) ===========

Tier 1

============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 33 items / 31 deselected / 2 selected

test_only_logs_after.py FF                                               [100%]

=================================== FAILURES ===================================
________ test_multiple_calls[guardduty_only_logs_after_multiple_calls] _________
------------------------------ Captured log call -------------------------------
DEBUG    wazuh_testing:cli_utils.py:23 Calling AWS module with: '[PosixPath('/var/ossec/wodles/aws/aws-s3'), '--bucket', 'wazuh-guardduty-integration-tests', '--type', 'guardduty', '--regions', 'us-east-1', '--aws_profile', 'qa', '--debug', '2']'
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Debug mode on - Level: 2
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: Generating default configuration for retries: mode standard - max_attempts 10
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Table does not exist; create
DEBUG    wazuh_testing:cli_utils.py:52 The functionality to process GuardDuty logs stored in S3 via Kinesis was deprecated in 4.5. Consider configuring GuardDuty to store its findings directly in an S3 bucket instead. Check https://documentation.wazuh.com/current/amazon/services/supported-services/guardduty.html for more information. 
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Marker: 2023/02/02
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ DB Maintenance
ERROR    wazuh_testing:cli_utils.py:62 Some logs may were processed
ERROR    wazuh_testing:cli_utils.py:63 Results found: 0
ERROR    wazuh_testing:cli_utils.py:64 Results expected: 1
_____ test_multiple_calls[native_guardduty_only_logs_after_multiple_calls] _____
------------------------------ Captured log call -------------------------------
DEBUG    wazuh_testing:cli_utils.py:23 Calling AWS module with: '[PosixPath('/var/ossec/wodles/aws/aws-s3'), '--bucket', 'wazuh-native-guardduty-integration-tests', '--type', 'guardduty', '--regions', 'us-east-1', '--aws_profile', 'qa', '--debug', '2']'
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Debug mode on - Level: 2
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: Generating default configuration for retries: mode standard - max_attempts 10
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Table does not exist; create
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Working on 819751203818 - us-east-1
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Marker: AWSLogs/819751203818/GuardDuty/us-east-1/2023/02/02
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ No logs to process in bucket: 819751203818/us-east-1
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ DB Maintenance
DEBUG    wazuh_testing:cli_utils.py:23 Calling AWS module with: '[PosixPath('/var/ossec/wodles/aws/aws-s3'), '--bucket', 'wazuh-native-guardduty-integration-tests', '--type', 'guardduty', '--regions', 'us-east-1', '--aws_profile', 'qa', '--debug', '2', '--only_logs_after', '2022-NOV-20']'
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Debug mode on - Level: 2
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: Generating default configuration for retries: mode standard - max_attempts 10
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Working on 819751203818 - us-east-1
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Marker: AWSLogs/819751203818/GuardDuty/us-east-1/2022/11/20
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: ++ Found new log: AWSLogs/819751203818/GuardDuty/us-east-1/2022/11/21/ukkce94f-eb7e-4f45-b324-834a0d0cas43.jsonl.gz
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: ++ Found new log: AWSLogs/819751203818/GuardDuty/us-east-1/2022/11/23/465ce94f-eb7e-4f45-b324-834a0d0cas43.jsonl.gz
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: ++ Found new log: AWSLogs/819751203818/GuardDuty/us-east-1/2022/11/26/987ce94f-eb7e-4f45-b324-834a0d0ccd08.jsonl.gz
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ DB Maintenance
DEBUG    wazuh_testing:cli_utils.py:23 Calling AWS module with: '[PosixPath('/var/ossec/wodles/aws/aws-s3'), '--bucket', 'wazuh-native-guardduty-integration-tests', '--type', 'guardduty', '--regions', 'us-east-1', '--aws_profile', 'qa', '--debug', '2', '--only_logs_after', '2022-NOV-20']'
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Debug mode on - Level: 2
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: Generating default configuration for retries: mode standard - max_attempts 10
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Working on 819751203818 - us-east-1
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Marker: AWSLogs/819751203818/GuardDuty/us-east-1/2022/11/20
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: ++ Skipping previously processed file: AWSLogs/819751203818/GuardDuty/us-east-1/2022/11/21/ukkce94f-eb7e-4f45-b324-834a0d0cas43.jsonl.gz
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: ++ Skipping previously processed file: AWSLogs/819751203818/GuardDuty/us-east-1/2022/11/23/465ce94f-eb7e-4f45-b324-834a0d0cas43.jsonl.gz
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: ++ Skipping previously processed file: AWSLogs/819751203818/GuardDuty/us-east-1/2022/11/26/987ce94f-eb7e-4f45-b324-834a0d0ccd08.jsonl.gz
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ DB Maintenance
DEBUG    wazuh_testing:cli_utils.py:23 Calling AWS module with: '[PosixPath('/var/ossec/wodles/aws/aws-s3'), '--bucket', 'wazuh-native-guardduty-integration-tests', '--type', 'guardduty', '--regions', 'us-east-1', '--aws_profile', 'qa', '--debug', '2', '--only_logs_after', '2022-NOV-22']'
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Debug mode on - Level: 2
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: Generating default configuration for retries: mode standard - max_attempts 10
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Working on 819751203818 - us-east-1
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Marker: AWSLogs/819751203818/GuardDuty/us-east-1/2022/11/22
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: ++ Skipping previously processed file: AWSLogs/819751203818/GuardDuty/us-east-1/2022/11/23/465ce94f-eb7e-4f45-b324-834a0d0cas43.jsonl.gz
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: ++ Skipping previously processed file: AWSLogs/819751203818/GuardDuty/us-east-1/2022/11/26/987ce94f-eb7e-4f45-b324-834a0d0ccd08.jsonl.gz
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ DB Maintenance
DEBUG    wazuh_testing:cli_utils.py:23 Calling AWS module with: '[PosixPath('/var/ossec/wodles/aws/aws-s3'), '--bucket', 'wazuh-native-guardduty-integration-tests', '--type', 'guardduty', '--regions', 'us-east-1', '--aws_profile', 'qa', '--debug', '2']'
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Debug mode on - Level: 2
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: Generating default configuration for retries: mode standard - max_attempts 10
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Working on 819751203818 - us-east-1
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Marker: AWSLogs/819751203818/GuardDuty/us-east-1/2023/02/02
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: ++ Found new log: AWSLogs/819751203818/GuardDuty/us-east-1/2023/02/02/dcf07e79-60ab-4e93-a3ca-e4a5a9eda2e9.jsonl.gz
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ DB Maintenance
ERROR    wazuh_testing:cli_utils.py:62 The AWS module did not use the correct marker
ERROR    wazuh_testing:cli_utils.py:63 Results found: 0
ERROR    wazuh_testing:cli_utils.py:64 Results expected: 1
=========================== short test summary info ============================
FAILED test_only_logs_after.py::test_multiple_calls[guardduty_only_logs_after_multiple_calls]
FAILED test_only_logs_after.py::test_multiple_calls[native_guardduty_only_logs_after_multiple_calls]
================ 2 failed, 31 deselected, 3 warnings in 55.90s =================
nico-stefani commented 1 year ago

WAF Results

Tier 0

root@ubuntu-jammy:/home/vagrant/qa/tests/integration/test_aws# pytest -k 'waf' --tier 0
================================================================================================ test session starts ================================================================================================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 24 items / 16 deselected / 8 selected

test_basic.py .                                                                                                                                                                                               [ 12%]
test_discard_regex.py .                                                                                                                                                                                       [ 25%]
test_only_logs_after.py ..                                                                                                                                                                                    [ 50%]
test_path.py ...                                                                                                                                                                                              [ 87%]
test_remove_from_bucket.py .                                                                                                                                                                                  [100%]

================================================================================================= warnings summary ==================================================================================================
../../../../../../usr/local/lib/python3.10/dist-packages/_pytest/cacheprovider.py:433
  /usr/local/lib/python3.10/dist-packages/_pytest/cacheprovider.py:433: PytestCacheWarning: cache could not write path /home/vagrant/qa/tests/integration/.pytest_cache/v/cache/nodeids
    config.cache.set("cache/nodeids", sorted(self.cached_nodeids))

../../../../../../usr/local/lib/python3.10/dist-packages/_pytest/stepwise.py:52
  /usr/local/lib/python3.10/dist-packages/_pytest/stepwise.py:52: PytestCacheWarning: cache could not write path /home/vagrant/qa/tests/integration/.pytest_cache/v/cache/stepwise
    session.config.cache.set(STEPWISE_CACHE_DIR, [])

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
============================================================================= 8 passed, 16 deselected, 2 warnings in 172.80s (0:02:52) ==============================================================================

Tier 1

root@ubuntu-jammy:/home/vagrant/qa/tests/integration/test_aws# pytest -k waf --disable-warnings --tier 1 --tb=no
================================================================================================ test session starts ================================================================================================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 195 items / 194 deselected / 1 selected

test_only_logs_after.py F                                                                                                                                                                                     [100%]

============================================================================================== short test summary info ==============================================================================================
FAILED test_only_logs_after.py::test_bucket_multiple_calls[waf_only_logs_after_multiple_calls] - wazuh_testing.modules.aws.cli_utils.OutputAnalysisError: The AWS module did not use the correct marker
================================================================================== 1 failed, 194 deselected, 3 warnings in 35.06s ===================================================================================
nico-stefani commented 1 year ago

Server Access Results

Tier 0

root@ubuntu-jammy:/home/vagrant/qa/tests/integration/test_aws# pytest -k server_access --disable-warnings --tier 0 --tb=no
================================================================================================ test session starts ================================================================================================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 195 items / 187 deselected / 8 selected

test_basic.py .                                                                                                                                                                                               [ 12%]
test_discard_regex.py .                                                                                                                                                                                       [ 25%]
test_only_logs_after.py ..                                                                                                                                                                                    [ 50%]
test_path.py ...                                                                                                                                                                                              [ 87%]
test_remove_from_bucket.py .                                                                                                                                                                                  [100%]

============================================================================= 8 passed, 187 deselected, 2 warnings in 173.15s (0:02:53) =============================================================================

Tier 1

root@ubuntu-jammy:/home/vagrant/qa/tests/integration/test_aws# pytest -k server_access --disable-warnings --tier 1 --tb=no
================================================================================================ test session starts ================================================================================================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 195 items / 194 deselected / 1 selected

test_only_logs_after.py F                                                                                                                                                                                     [100%]

============================================================================================== short test summary info ==============================================================================================
FAILED test_only_logs_after.py::test_bucket_multiple_calls[server_access_only_logs_after_multiple_calls] - wazuh_testing.modules.aws.cli_utils.OutputAnalysisError: The AWS module did not use the correct marker
================================================================================== 1 failed, 194 deselected, 3 warnings in 34.67s ===================================================================================
nico-stefani commented 1 year ago

Inspector Results

Tier 0

============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 21 items / 16 deselected / 5 selected

test_basic.py .                                                          [ 20%]
test_only_logs_after.py .                                                [ 40%]
test_regions.py ..F                                                      [100%]

=================================== FAILURES ===================================
______________ test_service_regions[inspector_inexistent_region] _______________

------------------------------ Captured log call -------------------------------
ERROR    wazuh_testing:monitoring.py:465 The AWS module did not show correct message non-existent region
ERROR    wazuh_testing:monitoring.py:466 Results accumulated: 0
ERROR    wazuh_testing:monitoring.py:468 Results expected: 1
=========================== short test summary info ============================
FAILED test_regions.py::test_service_regions[inspector_inexistent_region] - T...
====== 1 failed, 4 passed, 16 deselected, 3 warnings in 125.92s (0:02:05) ======

Tier 1

============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 21 items / 20 deselected / 1 selected

test_only_logs_after.py F                                                [100%]

=================================== FAILURES ===================================
____ test_service_multiple_calls[inspector_only_logs_after_multiple_calls] _____
------------------------------ Captured log call -------------------------------
DEBUG    wazuh_testing:cli_utils.py:23 Calling AWS module with: '[PosixPath('/var/ossec/wodles/aws/aws-s3'), '--service', 'inspector', '--regions', 'us-east-1', '--aws_profile', 'qa', '--debug', '2']'
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Debug mode on - Level: 2
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Getting alerts from "us-east-1" region.
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: Generating default configuration for retries: mode standard - max_attempts 10
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Table does not exist; create
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Listing findings starting from 2023-01-31 00:00:00
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ There are no new events in the "us-east-1" region
DEBUG    wazuh_testing:cli_utils.py:23 Calling AWS module with: '[PosixPath('/var/ossec/wodles/aws/aws-s3'), '--service', 'inspector', '--regions', 'us-east-1', '--aws_profile', 'qa', '--debug', '2', '--only_logs_after', '2023-JAN-30']'
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Debug mode on - Level: 2
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Getting alerts from "us-east-1" region.
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: Generating default configuration for retries: mode standard - max_attempts 10
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ Listing findings starting from 2023-01-31 12:29:52.336149
DEBUG    wazuh_testing:cli_utils.py:52 DEBUG: +++ There are no new events in the "us-east-1" region
ERROR    wazuh_testing:cli_utils.py:62 The AWS module did not process the expected number of events
ERROR    wazuh_testing:cli_utils.py:63 Results found: 0
ERROR    wazuh_testing:cli_utils.py:64 Results expected: 1
=========================== short test summary info ============================
FAILED test_only_logs_after.py::test_service_multiple_calls[inspector_only_logs_after_multiple_calls]
================ 1 failed, 20 deselected, 3 warnings in 28.31s =================
nico-stefani commented 1 year ago

Cloudwatch Results

Tier 0 results

================================================================================================ test session starts ================================================================================================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 195 items / 186 deselected / 9 selected

test_basic.py .                                                                                                                                                                                               [ 11%]
test_log_groups.py ..                                                                                                                                                                                         [ 33%]
test_only_logs_after.py ..                                                                                                                                                                                    [ 55%]
test_regions.py ..F                                                                                                                                                                                           [ 88%]
test_remove_from_bucket.py .                                                                                                                                                                                  [100%]

===================================================================================================== FAILURES ======================================================================================================
/home/vagrant/qa/deps/wazuh_testing/wazuh_testing/tools/monitoring.py:469: TimeoutError: The AWS module did not show correct message non-existent region
============================================================================================== short test summary info ==============================================================================================
FAILED test_regions.py::test_service_regions[cloudwatchlogs_inexistent_region] - TimeoutError: The AWS module did not show correct message non-existent region
======================================================================== 1 failed, 8 passed, 186 deselected, 3 warnings in 198.25s (0:03:18) ========================================================================

Tier 1

============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 25 items / 24 deselected / 1 selected

test_only_logs_after.py .                                                [100%]
================ 1 passed, 24 deselected, 2 warnings in 41.45s =================
nico-stefani commented 1 year ago

Umbrella Results

Tier 0

============================= test session starts ==============================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 24 items / 16 deselected / 8 selected

test_basic.py .                                                          [ 12%]
test_discard_regex.py .                                                  [ 25%]
test_only_logs_after.py ..                                               [ 50%]
test_path.py ...                                                         [ 87%]
test_remove_from_bucket.py .                                             [100%]
=========== 8 passed, 16 deselected, 2 warnings in 174.76s (0:02:54) ===========

Tier 1 results

root@ubuntu-jammy:/home/vagrant/qa/tests/integration/test_aws# pytest -k umbrella --disable-warnings --tier 1 --tb=line
================================================================================================ test session starts ================================================================================================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 195 items / 194 deselected / 1 selected

test_only_logs_after.py F                                                                                                                                                                                     [100%]

===================================================================================================== FAILURES ======================================================================================================
/home/vagrant/qa/deps/wazuh_testing/wazuh_testing/modules/aws/cli_utils.py:65: wazuh_testing.modules.aws.cli_utils.OutputAnalysisError: The AWS module did not use the correct marker
============================================================================================== short test summary info ==============================================================================================
FAILED test_only_logs_after.py::test_bucket_multiple_calls[cisco_umbrella_only_logs_after_multiple_calls] - wazuh_testing.modules.aws.cli_utils.OutputAnalysisError: The AWS module did not use the correct marker
================================================================================== 1 failed, 194 deselected, 3 warnings in 34.52s ===================================================================================
nico-stefani commented 1 year ago

Parser Results

Tier 0

root@ubuntu-jammy:/home/vagrant/qa/tests/integration/test_aws# pytest -k parser --disable-warnings --tier 0 --tb=line
================================================================================================ test session starts ================================================================================================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/vagrant/qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.2, html-3.1.1, testinfra-5.0.0
collected 195 items / 169 deselected / 26 selected

test_parser.py .....FFFF.FFF.F.FFF..FFF..                                                                                                                                                                     [100%]

===================================================================================================== FAILURES ======================================================================================================
============================================================================================== short test summary info ==============================================================================================
FAILED test_parser.py::test_empty_values_in_bucket[parser_empty_only_logs_after_in_bucket] - TimeoutError: The AWS module did not show the expected message about empty value
FAILED test_parser.py::test_empty_values_in_bucket[parser_empty_regions_in_bucket] - TimeoutError: The AWS module did not show the expected message about empty value
FAILED test_parser.py::test_empty_values_in_bucket[parser_empty_path_in_bucket] - TimeoutError: The AWS module did not show the expected message about empty value
FAILED test_parser.py::test_empty_values_in_bucket[parser_empty_path_suffix_in_bucket] - TimeoutError: The AWS module did not show the expected message about empty value
FAILED test_parser.py::test_empty_values_in_service[parser_empty_log_groups_in_service] - TimeoutError: The AWS module did not show the expected message about empty value
FAILED test_parser.py::test_empty_values_in_service[parser_empty_only_logs_after_in_service] - TimeoutError: The AWS module did not show the expected message about empty value
FAILED test_parser.py::test_empty_values_in_service[parser_empty_regions_in_service] - TimeoutError: The AWS module did not show the expected message about empty value
FAILED test_parser.py::test_invalid_values_in_bucket[parser_invalid_name_in_bucket] - TimeoutError: The AWS module did not show the expected message about invalid value
FAILED test_parser.py::test_invalid_values_in_bucket[parser_invalid_regions_in_bucket] - TimeoutError: The AWS module did not show the expected message about invalid value
FAILED test_parser.py::test_invalid_values_in_bucket[parser_invalid_path_in_bucket] - TimeoutError: The AWS module did not show the expected message about invalid value
FAILED test_parser.py::test_invalid_values_in_bucket[parser_invalid_path_suffix_in_bucket] - TimeoutError: The AWS module did not show the expected message about invalid value
FAILED test_parser.py::test_invalid_values_in_service[parser_invalid_log_groups_in_service] - TimeoutError: The AWS module did not show the expected message about invalid value
FAILED test_parser.py::test_invalid_values_in_service[parser_invalid_only_logs_after_in_service] - TimeoutError: The AWS module did not show the expected message about invalid value
FAILED test_parser.py::test_invalid_values_in_service[parser_invalid_regions_in_service] - TimeoutError: The AWS module did not show the expected message about invalid value
======================================================================= 14 failed, 12 passed, 169 deselected, 3 warnings in 288.18s (0:04:48) =======================================================================
nico-stefani commented 1 year ago

After doing the changes I ran the tests again and these are the results:

root@ip-172-31-82-0:/home/ubuntu/wazuh-qa/tests/integration/test_aws# pytest --disable-warnings --tb=line --tier 0 -x
================================================================================================ test session starts ================================================================================================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/ubuntu/wazuh-qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.4, html-3.1.1, testinfra-5.0.0
collected 195 items / 16 deselected / 179 selected

test_basic.py ................                                                                                                                                                                                [  8%]
test_discard_regex.py ..............                                                                                                                                                                          [ 16%]
test_log_groups.py ..                                                                                                                                                                                         [ 17%]
test_only_logs_after.py ...............................                                                                                                                                                       [ 35%]
test_parser.py ..........................                                                                                                                                                                     [ 49%]
test_path.py ..........................................                                                                                                                                                       [ 73%]
test_path_suffix.py .........                                                                                                                                                                                 [ 78%]
test_regions.py ........................                                                                                                                                                                      [ 91%]
test_remove_from_bucket.py ...sss.........                                                                                                                                                                    [100%]

============================================================================ 176 passed, 3 skipped, 16 deselected in 3607.33s (1:00:07) =============================================================================
root@ip-172-31-82-0:/home/ubuntu/wazuh-qa/tests/integration/test_aws# pytest --disable-warnings --tb=line --tier 1 -x
================================================================================================ test session starts ================================================================================================
platform linux -- Python 3.10.6, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/ubuntu/wazuh-qa/tests/integration, configfile: pytest.ini
plugins: metadata-2.0.4, html-3.1.1, testinfra-5.0.0
collected 195 items / 179 deselected / 16 selected

test_only_logs_after.py ..............x.                                                                                                                                                                      [100%]

============================================================================= 15 passed, 179 deselected, 1 xfailed in 362.08s (0:06:02) =============================================================================