search

codeclimate / test-reporter

Code Climate Test Reporter
MIT License
154 stars 76 forks source link

Support uploading partial test coverage for distributed testing? #486

Open schwern opened 2 years ago

schwern commented 2 years ago

Hi, I'm trying to get coverage reporting working with parallel testing on Heroku CI. Parallel testing on HerokuCI works by distributing the tests across multiple dynos each with their own ephemeral filesystem.

I could upload the test results to S3 as you suggest, but Heroku CI lacks a way to run a command when all the CI dynos have completed. I can't find where to put the command to sum-coverage and upload-coverage after all dynos have finished running (I'll contact support to ask).

Rather than having to push partial results to S3 and sum them, would it be possible to use cc-test-reporter to upload partial test reports directly to CC? For example, if I have 4 dynos running my tests in parallel I would do:

cc-test-reporter before-build
bundle exec parallel_rspec spec/ -n $CI_NODE_TOTAL --only-group $CI_NODE_INDEX
cc-test-reporter after-build --part $CI_NODE_INDEX --of $CI_NODE_TOTAL

Then Code Climate knows it has received, for example, part 3 of 4. Once it's received all 4 parts it does the sum itself.

This would make setting up distributed testing easier.