DataDog / datadog-ci

Use Datadog from your CI.
https://datadoghq.com
Apache License 2.0
129 stars 55 forks source link

[SYNTH-15675] Fix bug where timed out retries are not reported #1396

Closed AntoineDona closed 2 months ago

AntoineDona commented 4 months ago

What and why?

We recently discovered a bug where timed out fast retries were not reported as fast retries. This lead to executeTests exiting without the reporter spinner being stopped.

This was due to the fact that getResultsToReport didn’t handle still non-final results after the end of a batch, resulting in an empty resultsToReport array. This PR patches this issue.

How?

Review checklist

Drarig29 commented 4 months ago

@AntoineDona @etnbrd this PR is more of a draft of a solution we thought was working. But I now think we missed something, and we'll probably need to add a property in the backend.

For now, consider this PR on-hold.

datadog-datadog-prod-us1[bot] commented 4 months ago

Datadog Report

Branch report: antoine.donascimento/SYNTH-15675/fix-non-reported-timing-out-fast-retries Commit report: 6d23211 Test service: datadog-ci-tests

:x: 41 Failed (0 Known Flaky), 3533 Passed, 0 Skipped, 2m 29.74s Total duration (1m 2.63s time saved)

:x: Failed Tests (41)

This report shows up to 5 failed tests.

  • Proxy configuration use proxy defined in configuration - src/commands/synthetics/__tests__/tunnel/websocket.test.ts - Details
<details>
<summary>Expand for error</summary>

```
Failed to get batch: could not query base/synthetics/ci/batch/bid
connect ECONNREFUSED 127.0.0.1:80

```
</details>