search

newrelic / newrelic-ruby-agent

New Relic RPM Ruby Agent
https://docs.newrelic.com/docs/apm/agents/ruby-agent/getting-started/introduction-new-relic-ruby/
Apache License 2.0
1.2k stars 596 forks source link

Create automated Envfiles #1184

Closed kaylareopelle closed 2 years ago

kaylareopelle commented 2 years ago

Our multiverse suites have a filetype called "Envfile" that holds heredocs that describe the Gemfiles that a given suite should be tested against.

All Envfiles share some similarities, however, there is enough difference between each implementation to cause friction when trying to determine whether all desired combinations of gems are being tested.

Task

Create a strategy to automate Envfile creation and standardize the format of these files.

kaylareopelle commented 2 years ago

Putting this on hold while I work on the GRPC milestone. Currently, SimpleCov is reporting a much lower line/branch coverage threshold with the refactored Envfiles. We're not sure about the root cause of this issue and do not want to merge in the change until it can be addressed.

kaylareopelle commented 2 years ago

After chatting with James, it seems the thresholds are varying on more PRs than this one. The SimpleCov threshold investigation will be moved to a separate ticket and the current thresholds lowered to 5% below the lowest observed values.