Investigate fail test in `/overview/agents/` endpoint

[nico-stefani]

Description

During the RC1 stage of 4.8.0, we detect a failure in the API performance tests in the /overview/agents/ endpoint:

test_case = {'body': {}, 'endpoint': '/overview/agents', 'method': 'get', 'parameters': {}}
set_api_test_environment = None, api_healthcheck = None

    @pytest.mark.parametrize('test_case', test_data['test_cases'], ids=case_ids)
    def test_api_endpoints(test_case, set_api_test_environment, api_healthcheck):
        """Make an API request for each `test_case`.

        Args:
            test_case (dict): Dictionary with the endpoint to be tested and the necessary parameters for the test.
            set_api_test_environment (fixture): Fixture that modifies the API security options.
            api_healthcheck (fixture): Fixture used to check that the API is ready to respond requests.
        """
        # Apply xfails
        if test_case['endpoint'] in xfailed_items.keys() and \
                test_case['method'] == xfailed_items[test_case['endpoint']]['method']:
            pytest.xfail(xfailed_items[test_case['endpoint']]['message'])

        base_url = api_details['base_url']
        headers = api_details['auth_headers']
        response = None
        try:
            response = getattr(requests, test_case['method'])(f"{base_url}{test_case['endpoint']}", headers=headers,
                                                              params=test_case['parameters'], json=test_case['body'],
                                                              verify=False)
>           assert response.status_code == 200
E           assert 500 == 200
E            +  where 500 = <Response [500]>.status_code

tests/performance/test_api/test_api_endpoints_performance.py:47: AssertionError
 ------------------------------Captured stdout call------------------------------ 
Request elapsed time: 10.120s

Status code: 500

Full response: 
{
  "title": "Wazuh Internal Error",
  "detail": "Timeout executing API request",
  "dapi_errors": {
    "master": {
      "error": "Timeout executing API request",
      "logfile": "WAZUH_HOME/logs/api.log"
    }
  },
  "error": 3021
}

Is needed to investigate the root reason of this problem and solve it.

Checks

The following elements have been updated or reviewed (should also be checked if no modification is required):

• [x] Tests (unit tests, API integration tests).
• [x] Changelog.
• [x] Documentation.
• [x] Integration test mapping (using api/test/integration/mapping/_test_mapping.py).

[javiersanchz]

Update

• I have been investigating the error regarding the failure of the test for the /overview/agents/ endpoint.

• Just yesterday, the test_overview_endpoints.tavern.yaml was launched at https://github.com/wazuh/wazuh/issues/23309 for v4.8.0-rc1, and it worked correctly.

• No changes were found in the code since its last release at https://github.com/wazuh/wazuh/issues/23065

• The issue appears to originate from wazuh-db, as the accumulation of wazuh-db queries seems to be causing excessive load, resulting in errors because the operation cannot be completed within the expected time (timeout).

• This has already been discussed and addressed in :

  • https://github.com/wazuh/wazuh/issues/6747
  • https://github.com/wazuh/wazuh/pull/6754

• As discussed, some minor changes were investigated and made for somewhat more efficient operation.

• The question is why wazuh-db is apparently performing worse in 4.8.0, at the moment, this is being investigated at https://github.com/wazuh/wazuh/issues/23331.

[GGP1]

Review

The issue was already present in previous versions and the solution made the endpoint faster but did not resolve it. From the Pyserver team standpoint there's no optimizations to be done and we rely on the rapidness of wazuh-db to answer our queries.

The investigation from the issue mentioned will give us more context about the overall performance of the manager, possibly resulting in changes that resolve this as well. I agree with the conclusion, LGTM

[Selutario]

GJ, I agree with both of you. It could also be related to:

• https://github.com/wazuh/wazuh/issues/23347