wazuh / wazuh-qa

Wazuh - Quality Assurance
GNU General Public License v2.0
60 stars 30 forks source link

DTT1 - Possible flaky test in uninstall test #5241

Closed rauldpm closed 1 week ago

rauldpm commented 2 weeks ago

Description

A flaky test has been discovered in DTT1 when using the following YAML input, I have only reproduced it once in three executions

YAML ``` version: 0.1 description: This workflow is used to test agents' deployment for DDT1 PoC variables: manager-os: - linux-ubuntu-22.04-amd64 - linux-ubuntu-18.04-amd64 - linux-ubuntu-20.04-amd64 - linux-amazon-2-amd64 - linux-redhat-7-amd64 - linux-redhat-8-amd64 - linux-redhat-9-amd64 - linux-centos-7-amd64 - linux-centos-8-amd64 - linux-oracle-9-amd64 - linux-debian-10-amd64 - linux-debian-11-amd64 - linux-debian-12-amd64 infra-provider: aws working-dir: /tmp/dtt1-poc tasks: # Unique manager allocate task - task: "allocate-manager-linux-ubuntu-22.04-amd64" description: "Allocate resources for the manager." do: this: process with: path: python3 args: - modules/allocation/main.py - action: create - provider: "{infra-provider}" - size: large - composite-name: "linux-ubuntu-22.04-amd64" - inventory-output: "{working-dir}/manager-linux-ubuntu-22.04-amd64/inventory.yaml" - track-output: "{working-dir}/manager-linux-ubuntu-22.04-amd64/track.yaml" - ssh-key: "" - label-termination-date: "1d" - label-team: "qa" cleanup: this: process with: path: python3 args: - modules/allocation/main.py - action: delete - track-output: "{working-dir}/manager-linux-ubuntu-22.04-amd64/track.yaml" # Unique manager allocate task - task: "allocate-manager-linux-ubuntu-18.04-amd64" description: "Allocate resources for the manager." do: this: process with: path: python3 args: - modules/allocation/main.py - action: create - provider: "{infra-provider}" - size: large - composite-name: "linux-ubuntu-18.04-amd64" - inventory-output: "{working-dir}/manager-linux-ubuntu-18.04-amd64/inventory.yaml" - track-output: "{working-dir}/manager-linux-ubuntu-18.04-amd64/track.yaml" - ssh-key: "" - label-termination-date: "1d" - label-team: "qa" cleanup: this: process with: path: python3 args: - modules/allocation/main.py - action: delete - track-output: "{working-dir}/manager-linux-ubuntu-18.04-amd64/track.yaml" # Unique manager allocate task - task: "allocate-manager-linux-ubuntu-20.04-amd64" description: "Allocate resources for the manager." do: this: process with: path: python3 args: - modules/allocation/main.py - action: create - provider: "{infra-provider}" - size: large - composite-name: "linux-ubuntu-20.04-amd64" - inventory-output: "{working-dir}/manager-linux-ubuntu-20.04-amd64/inventory.yaml" - track-output: "{working-dir}/manager-linux-ubuntu-20.04-amd64/track.yaml" - ssh-key: "" - label-termination-date: "1d" - label-team: "qa" cleanup: this: process with: path: python3 args: - modules/allocation/main.py - action: delete - track-output: "{working-dir}/manager-linux-ubuntu-20.04-amd64/track.yaml" # Unique manager allocate task - task: "allocate-manager-linux-amazon-2-amd64" description: "Allocate resources for the manager." do: this: process with: path: python3 args: - modules/allocation/main.py - action: create - provider: "{infra-provider}" - size: large - composite-name: "linux-amazon-2-amd64" - inventory-output: "{working-dir}/manager-linux-amazon-2-amd64/inventory.yaml" - track-output: "{working-dir}/manager-linux-amazon-2-amd64/track.yaml" - ssh-key: "" - label-termination-date: "1d" - label-team: "qa" cleanup: this: process with: path: python3 args: - modules/allocation/main.py - action: delete - track-output: "{working-dir}/manager-linux-amazon-2-amd64/track.yaml" # Unique manager allocate task - task: "allocate-manager-linux-redhat-7-amd64" description: "Allocate resources for the manager." do: this: process with: path: python3 args: - modules/allocation/main.py - action: create - provider: "{infra-provider}" - size: large - composite-name: "linux-redhat-7-amd64" - inventory-output: "{working-dir}/manager-linux-redhat-7-amd64/inventory.yaml" - track-output: "{working-dir}/manager-linux-redhat-7-amd64/track.yaml" - ssh-key: "" - label-termination-date: "1d" - label-team: "qa" cleanup: this: process with: path: python3 args: - modules/allocation/main.py - action: delete - track-output: "{working-dir}/manager-linux-redhat-7-amd64/track.yaml" # Unique manager allocate task - task: "allocate-manager-linux-redhat-8-amd64" description: "Allocate resources for the manager." do: this: process with: path: python3 args: - modules/allocation/main.py - action: create - provider: "{infra-provider}" - size: large - composite-name: "linux-redhat-8-amd64" - inventory-output: "{working-dir}/manager-linux-redhat-8-amd64/inventory.yaml" - track-output: "{working-dir}/manager-linux-redhat-8-amd64/track.yaml" - ssh-key: "" - label-termination-date: "1d" - label-team: "qa" cleanup: this: process with: path: python3 args: - modules/allocation/main.py - action: delete - track-output: "{working-dir}/manager-linux-redhat-8-amd64/track.yaml" # Unique manager allocate task - task: "allocate-manager-linux-redhat-9-amd64" description: "Allocate resources for the manager." do: this: process with: path: python3 args: - modules/allocation/main.py - action: create - provider: "{infra-provider}" - size: large - composite-name: "linux-redhat-9-amd64" - inventory-output: "{working-dir}/manager-linux-redhat-9-amd64/inventory.yaml" - track-output: "{working-dir}/manager-linux-redhat-9-amd64/track.yaml" - ssh-key: "" - label-termination-date: "1d" - label-team: "qa" cleanup: this: process with: path: python3 args: - modules/allocation/main.py - action: delete - track-output: "{working-dir}/manager-linux-redhat-9-amd64/track.yaml" # Unique manager allocate task - task: "allocate-manager-linux-centos-7-amd64" description: "Allocate resources for the manager." do: this: process with: path: python3 args: - modules/allocation/main.py - action: create - provider: "{infra-provider}" - size: large - composite-name: "linux-centos-7-amd64" - inventory-output: "{working-dir}/manager-linux-centos-7-amd64/inventory.yaml" - track-output: "{working-dir}/manager-linux-centos-7-amd64/track.yaml" - ssh-key: "" - label-termination-date: "1d" - label-team: "qa" cleanup: this: process with: path: python3 args: - modules/allocation/main.py - action: delete - track-output: "{working-dir}/manager-linux-centos-7-amd64/track.yaml" # Unique manager allocate task - task: "allocate-manager-linux-centos-8-amd64" description: "Allocate resources for the manager." do: this: process with: path: python3 args: - modules/allocation/main.py - action: create - provider: "{infra-provider}" - size: large - composite-name: "linux-centos-8-amd64" - inventory-output: "{working-dir}/manager-linux-centos-8-amd64/inventory.yaml" - track-output: "{working-dir}/manager-linux-centos-8-amd64/track.yaml" - ssh-key: "" - label-termination-date: "1d" - label-team: "qa" cleanup: this: process with: path: python3 args: - modules/allocation/main.py - action: delete - track-output: "{working-dir}/manager-linux-centos-8-amd64/track.yaml" # Unique manager allocate task - task: "allocate-manager-linux-debian-10-amd64" description: "Allocate resources for the manager." do: this: process with: path: python3 args: - modules/allocation/main.py - action: create - provider: "{infra-provider}" - size: large - composite-name: "linux-debian-10-amd64" - inventory-output: "{working-dir}/manager-linux-debian-10-amd64/inventory.yaml" - track-output: "{working-dir}/manager-linux-debian-10-amd64/track.yaml" - ssh-key: "" - label-termination-date: "1d" - label-team: "qa" cleanup: this: process with: path: python3 args: - modules/allocation/main.py - action: delete - track-output: "{working-dir}/manager-linux-debian-10-amd64/track.yaml" # Unique manager allocate task - task: "allocate-manager-linux-debian-11-amd64" description: "Allocate resources for the manager." do: this: process with: path: python3 args: - modules/allocation/main.py - action: create - provider: "{infra-provider}" - size: large - composite-name: "linux-debian-11-amd64" - inventory-output: "{working-dir}/manager-linux-debian-11-amd64/inventory.yaml" - track-output: "{working-dir}/manager-linux-debian-11-amd64/track.yaml" - ssh-key: "" - label-termination-date: "1d" - label-team: "qa" cleanup: this: process with: path: python3 args: - modules/allocation/main.py - action: delete - track-output: "{working-dir}/manager-linux-debian-11-amd64/track.yaml" # Unique manager allocate task - task: "allocate-manager-linux-debian-12-amd64" description: "Allocate resources for the manager." do: this: process with: path: python3 args: - modules/allocation/main.py - action: create - provider: "{infra-provider}" - size: large - composite-name: "linux-debian-12-amd64" - inventory-output: "{working-dir}/manager-linux-debian-12-amd64/inventory.yaml" - track-output: "{working-dir}/manager-linux-debian-12-amd64/track.yaml" - ssh-key: "" - label-termination-date: "1d" - label-team: "qa" cleanup: this: process with: path: python3 args: - modules/allocation/main.py - action: delete - track-output: "{working-dir}/manager-linux-debian-12-amd64/track.yaml" # Unique manager allocate task - task: "allocate-manager-linux-oracle-9-amd64" description: "Allocate resources for the manager." do: this: process with: path: python3 args: - modules/allocation/main.py - action: create - provider: "{infra-provider}" - size: large - composite-name: "linux-oracle-9-amd64" - inventory-output: "{working-dir}/manager-linux-oracle-9-amd64/inventory.yaml" - track-output: "{working-dir}/manager-linux-oracle-9-amd64/track.yaml" - ssh-key: "" - label-termination-date: "1d" - label-team: "qa" cleanup: this: process with: path: python3 args: - modules/allocation/main.py - action: delete - track-output: "{working-dir}/manager-linux-oracle-9-amd64/track.yaml" # Unique manager provision task - task: "provision-manager-linux-oracle-9-amd64" description: "Provision the manager." do: this: process with: path: python3 args: - modules/provision/main.py - inventory: "{working-dir}/manager-linux-oracle-9-amd64/inventory.yaml" - install: - component: tar depends-on: - "allocate-manager-linux-oracle-9-amd64" on-error: "abort-all" # Generic manager test task - task: "run-manager-tests" description: "Run tests install for the manager." do: this: process with: path: python3 args: - modules/testing/main.py - targets: - wazuh-1: "{working-dir}/manager-linux-ubuntu-22.04-amd64/inventory.yaml" - wazuh-2: "{working-dir}/manager-linux-ubuntu-18.04-amd64/inventory.yaml" - wazuh-3: "{working-dir}/manager-linux-ubuntu-20.04-amd64/inventory.yaml" - wazuh-4: "{working-dir}/manager-linux-amazon-2-amd64/inventory.yaml" - wazuh-5: "{working-dir}/manager-linux-redhat-7-amd64/inventory.yaml" - wazuh-6: "{working-dir}/manager-linux-redhat-8-amd64/inventory.yaml" - wazuh-7: "{working-dir}/manager-linux-redhat-9-amd64/inventory.yaml" - wazuh-8: "{working-dir}/manager-linux-centos-7-amd64/inventory.yaml" - wazuh-9: "{working-dir}/manager-linux-centos-8-amd64/inventory.yaml" - wazuh-10: "{working-dir}/manager-linux-oracle-9-amd64/inventory.yaml" - wazuh-11: "{working-dir}/manager-linux-debian-10-amd64/inventory.yaml" - wazuh-12: "{working-dir}/manager-linux-debian-11-amd64/inventory.yaml" - wazuh-13: "{working-dir}/manager-linux-debian-12-amd64/inventory.yaml" - tests: "install,restart,stop,uninstall" - component: "manager" - wazuh-version: "4.7.3" - wazuh-revision: "40714" - live: "True" depends-on: - "provision-manager-linux-oracle-9-amd64" - "allocate-manager-linux-ubuntu-22.04-amd64" - "allocate-manager-linux-ubuntu-18.04-amd64" - "allocate-manager-linux-ubuntu-20.04-amd64" - "allocate-manager-linux-amazon-2-amd64" - "allocate-manager-linux-redhat-7-amd64" - "allocate-manager-linux-redhat-8-amd64" - "allocate-manager-linux-redhat-9-amd64" - "allocate-manager-linux-centos-7-amd64" - "allocate-manager-linux-centos-8-amd64" - "allocate-manager-linux-oracle-9-amd64" - "allocate-manager-linux-debian-10-amd64" - "allocate-manager-linux-debian-11-amd64" - "allocate-manager-linux-debian-12-amd64" - "allocate-manager-linux-oracle-9-amd64" ```

image

As the log file is stored in /tmp, I have not been able to save it due to a power outage

Based on the image, we need to check the uninstall test cases, as probably, the /var/ossec directory is probably still present due to an uninstall failed process

mhamra commented 1 week ago

UPDATE

I've modified the original workflow by removing the ssh-key: entries.

First run

I've found no failures in the first run. I've launched the workflow with threads=13

workflow-run-1.log

Second run

I've found no failures in the second run. I've launched the workflow with threads=13

workflow-run-2.log

Third run

I've found no failures in the third run. I've launched the workflow with threads=13.

workflow-run-3.log

rauldpm commented 1 week ago

As the issue cannot be reproduced we will close this issue as not planned