DataDog / datadog-ci-rb

Ruby library for Datadog test visibility
https://docs.datadoghq.com/continuous_integration/tests/ruby
Other
8 stars 4 forks source link

[SDTEST-138] code coverage extension fixes and improvements #171

Closed anmarchenko closed 4 months ago

anmarchenko commented 5 months ago

What does this PR do? Applies the following changes and optimizations to per test code coverage:

How to test the change? Tested using test-environment integration test stand: https://github.com/DataDog/test-environment/pull/283

The following changes in performance overhead were recorded:

Rubocop From 109% to 86% => 27% improvement

Middleman From 29,7% to 28,6% => no change

Jekyll From 0,4% to 0,3% => no change

devdocs From 16,4% to 16,9% => no change

Overall result: it does not change anything for projects where code coverage overhead was low enough but provides meaningful improvement for the project where code coverage overhead was the largest.

codecov-commenter commented 5 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 98.85%. Comparing base (f58bb35) to head (b04133e).

Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #171 +/- ## ========================================== - Coverage 98.86% 98.85% -0.01% ========================================== Files 228 228 Lines 10096 10082 -14 Branches 466 465 -1 ========================================== - Hits 9981 9967 -14 Misses 115 115 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

anmarchenko commented 4 months ago

@ivoanjo yes, the number for performance improvements is final and was measured with the current state of this PR. Indeed, the switch to rb_profile_frames caused more overhead than rb_sourcefile but it is safer and solves an issue with UTF-8 filenames, so I consider it being worth it.