newrelic / newrelic-ruby-agent

New Relic RPM Ruby Agent
https://docs.newrelic.com/docs/apm/agents/ruby-agent/getting-started/introduction-new-relic-ruby/
Apache License 2.0
1.2k stars 599 forks source link

Generate comprehensive code coverage statistics for our tests #1113

Closed fallwith closed 2 years ago

fallwith commented 2 years ago

We currently do not have comprehensive code coverage data for our tests, given that a) they are split between separate "unit" and "multiverse" sections, and b) the multiverse tests operate with nested Bundler instances whose activity will not be observed by the parent process. Once we layer in our CI - GitHub Actions - we also run into the challenge of c) different groups of tests are ran on separate containers.

Idea: Simplecov is a tool I am very familiar and satisfied with, and I like for us to use it here. Simplecov supports JSON as an output format. I'd like for us to be able to implement Simplecov, have it generate multiple .json output files, and then have a little utility that parses these files and produces a wholistic overview of all that was tested. For CI will have to wire up a separate step for storing the generated .json files in a centrally accessible location.

fallwith commented 2 years ago

Addressed with #1171