Custom minitest reporter is run once on the first split, twice on the second, ..., N times on the Nth split

[davidwessman]

Command: bundle exec rake knapsack_pro:queue:minitest Version: knapsack_pro 7.6.1 Test runner: Github Actions Minitest version: 5.24.0

We are using https://blinka.app (my own tool) to report test results to the PR as a comment. This is the minitest plugin: https://github.com/davidwessman/blinka_reporter/blob/main/lib/minitest/blinka_plugin.rb

For the 1-2 weeks we sometimes get more tests that we are actually running, that is when I realized it looked like this:

Did anything change with the Minitest runner? Or something with Minitest itself?

Maybe I need to update something with my plugin as well.

Here seems to be some change regarding plugins. https://github.com/minitest/minitest/commit/2c5f1b70d782e74c90b1fd1fd7675d43da34c9b1

[ArturT]

Do you use simplecov with Minitest? If so, we recently discovered a breaking change in the knapsack_pro 6.0.4 version. You can find here info on how to adjust your project.

I think there were no other changes.

Please note that in Queue Mode, tests are executed at least once. For example, if one of parallel CI nodes can't connect to the Knapsack Pro API, then it would run tests in fallback mode which means the same test could be executed on multiple nodes.

Could you provide a link to the CI build in the Knapsack Pro dashboard?

You can set KNAPSACK_PRO_LOG_LEVEL=debug and when the issue happens again please send a txt output from CI to support@knapsackpro.com. I'd like to investigate that. Thank you.