search

DataDog / datadog-ci-rb

Ruby library for Datadog test visibility
https://docs.datadoghq.com/continuous_integration/tests/ruby
Other
8 stars 4 forks source link

[SDTEST-437] Auto test retries for RSpec #213

Closed anmarchenko closed 2 months ago

anmarchenko commented 2 months ago

What does this PR do? Adds auto test retries functionality for RSpec test framework

How to test the change? Tested using middleman test suite with this flaky test: 

    it "is new flaky" do
      res = rand(3)
      expect(res).to eq 1
    end

Locally test process completed normally:

image

Results of the test run in Datadog

New flaky test detected:

image

All the retries are correctly displayed in Datadog test runs:

image

Test session has status "pass":

image
codecov-commenter commented 2 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 98.89%. Comparing base (f00caf6) to head (9e6f62e). Report is 12 commits behind head on main.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #213 +/- ## ========================================== + Coverage 98.84% 98.89% +0.04% ========================================== Files 253 261 +8 Lines 11630 12096 +466 Branches 521 544 +23 ========================================== + Hits 11496 11962 +466 Misses 134 134 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.