Review the qa-system-framework repository for relevant code to migrate

[QU3B1M]

Main issue
https://github.com/wazuh/wazuh-qa/issues/5527

Description

Analyze the content of the repository qa-system-framework, detect and document the code pertinent to migrate to the new QA repository.

Tasks

• [x] Review the qa-system-framework repository
• [x] Identify relevant code for migration to the new QA repository
• [x] Document a proposal for migration with a possible directory structure

[!Important] This repository doesn't have new activity since September 2023

Tasks conclusion

Review the qa-system-framework repository

This repository contains a proposed framework for the QA system tests, which development was stopped since September 2023. It has an interesting design, and it could be a good starting point for the development of a new QA framework.

Identify relevant code for migration to the new QA repository

As mentioned previously, the design of this repository is interesting. Still, the code is outdated and some of it was taken and improved on the qa-integration-framework by the Core team. So, we can mostly only take the 'ideas' for the design and some parts of the code from this repository.

Document a proposal for migration with a possible directory structure

Code

Considering that most of this repository's code is in 'Work in Progress' status, we can highlight some of it that could be migrated to the new QA repository, in some cases, after some improvements. The relevant code is the following:

• src/wazuh_qa_framework/
  • generic_modules: A set of generic tools and scripts that can be used in any kind of test. A lot of this code could be taken and improved if necessary.
    • exceptions/: A generic set of exceptions.
    • file/: A set of functions to work with files. A lot of functions are missing that could be taken from qa-integration-framework utils.
    • logging/framework_logger.py: A logger for the framework. (the repo also has another logger named pytest_logger tha is not needed)
    • process/: A set of classes to run and monitor OS processes.
    • requests/: A set of functions to make HTTP requests. Could be improved
    • threading/: A class to handle python threads.
    • time/: A class make the work with time easier.
    • tools/: Includes simplified version of the FileMonitor. This was also implemented and improved in qa-integration-framework tools
  • global_variables: Could be renamed to Constants, a set of global constants that are used in the framework.
  • meta_testing: A set of not-so-related functions. From here, only the functions inside utils.py are useful
    • utils.py: A set of functions to work with the framework, if migrated, most of the functions should be moved to other directories. functions read_file, write_file, remove_file, clean_file, append_log must be stored inside generic_modules/file/ directory, the remaining custom_callback function could be left here.
  • wazuh_components: A set of modules that are used to interact with Wazuh components. The majority of the modules are empty, but the wazuh_api module is interesting to migrate and improve.
    • api: A set of functions to interact with the Wazuh API. This module could be migrated and improved.
• tests/: Unit tests for the framework. This must be migrated and improved.

File structure

This could be the most relevant feature of this repository; it is well structured, and I think we could use it as inspiration for our new QA repository. Something like:

./
├── README.md
├── wazuh_qa_framework/
│   ├── __init__.py
│   ├── requirements.txt
│   ├── setup.py
│   ├── generic/
│   ├── constants/
│   ├── utils/
│   └── wazuh/
└── tests
    ├── __init__.py
    ├── generic/
    ├── constants/
    ├── utils/
    └── wazuh/

[QU3B1M]

QA System Framework

This repository contains a proposed framework for the QA system tests, which development was stopped since September 2023. It has an interesting design, and it is a good starting point for the development of a new QA framework.

What to migrate from this repository?

As mentioned previously, the design of this repository is interesting, and it is a good starting point for the development of a new QA framework, anyway the code is outdated and some if it was taken for the qa-integration-framework and improved by the Core team.

Code

Parts of the code that could be taken from this repository to the new QA framework are:

• src/wazuh_qa_framework/

  • generic_modules: A set of generic tools and scripts that can be used in any kind of test. A lot of this code could be taken and improved if necesary.
    • exceptions/: A generic set of exceptions.
    • file/: A set of functions to work with files. A lot of functions are missing that could be taken from qa-integration-framework utils.
    • logging/framework_logger.py: A logger for the framework. (the repo also has another logger named pytest_logger tha is not needed)
    • process/: A set of classes to run and monitor OS processes.
    • requests/: A set of functions to make HTTP requests. Could be improved
    • threading/: A class to handle python threads.
    • time/: A class make the work with time easier.
    • tools/: Includes simplified version of the FileMonitor. This was also implemented and improved in qa-integration-framework tools
  • global_variables: Could be renamed to Constants, a set of global constants that are used in the framework.
  • meta_testing: A set of not so related functions. From here, only the functions inside utils.py are useful
    • utils.py: A set of functions to work with the framework, if migrated, most of the functions should be moved to other directory. functions read_file, write_file, remove_file, clean_file, append_log must be stored inside generic_modules/file/ directory, the remaining custom_callback function could be left here.
  • wazuh_components: A set of modules that are used to interact with Wazuh components. The mayority of the modules are empty, but the wazuh_api module is interesting to migrate and improve.
    • api: A set of functions to interact with the Wazuh API. This module could be migrated and improved.

• tests/: Unit tests for the framework. This must be migrated and improved.

File strcuture

The file structure of this repository is interesting and could be taken as a base for the new QA framework. Using this structure as base I could propose something like this:

./
├── README.md
├── wazuh_qa_framework/
│   ├── __init__.py
│   ├── requirements.txt
│   ├── setup.py
│   ├── generic/
│   ├── constants/
│   ├── utils/
│   └── wazuh/
└── tests
    ├── __init__.py
    ├── generic/
    ├── constants/
    ├── utils/
    └── wazuh/

[damarisg]

Review

We decided to use part of the structure and some folders, but we will improve the files before migration.

The proposed proposal will remain in the issue of definition.