[CIVIS-10061] Fix Knapsack Pro integration

Fixes #147

This PR fixes the following issues with our current Knapsack Pro support:

Dropping tests because of missing test session

When running knapsack_pro:rspec:queue command, Datadog.configure block gets called when rails_helper is required which in this case happens after KnapsackPro::Extensions::RSpecExtension.setup! had been run.

To fix this, we check whether RSpec::Core::Runner was already instrumented by knapsack: if yes, we instrument RSpec::Core::Runner directly. If not, we hook into KnapsackPro::Extensions::RSpecExtension to instrument the runner after knapsack injects its methods:

if defined?(::KnapsackPro::Extensions::RSpecExtension::Runner) &&
  ::RSpec::Core::Runner.ancestors.include?(::KnapsackPro::Extensions::RSpecExtension::Runner)
  # knapsack already patched rspec runner
  require_relative "runner"
  ::RSpec::Core::Runner.include(KnapsackPro::Runner)
else
  # knapsack didn't patch rspec runner yet
  require_relative "extension"
  ::KnapsackPro::Extensions::RSpecExtension.include(KnapsackPro::Extension)
end

Noise from knapsack_pro:rspec_test_example_detector test command

We noticed that in many cases when using knapsack, we got a lot more test runs than there are unique tests; additional test runs all have very short duration and are part of knapsack_pro:rspec_test_example_detector.

It turned out, that we trace rspec session that is running with --dry-run: test example detector from knapsack uses dry run to generate a report with all rspec examples.

This issue is fixed by never tracing rspec tests when dry run is enabled:

return super if ::RSpec.configuration.dry_run?

How to test the change? Unit tests are provided and additionally tested with Github Actions: https://github.com/DataDog/rails-app-with-knapsack_pro/actions/workflows/main.yaml

The test runs arrived correctly:

There are no additional test runs from rspec_test_example_detector - the number of tests is equal to the number of test runs (51 vs 51)

DataDog / datadog-ci-rb

[CIVIS-10061] Fix Knapsack Pro integration #180

Codecov Report