search

KnapsackPro / knapsack_pro-ruby

Knapsack Pro gem splits tests across parallel CI nodes and makes sure that tests run in optimal time
https://knapsackpro.com
MIT License
131 stars 27 forks source link

Create a replication command on failed node #228

Closed kholbekj closed 6 months ago

kholbekj commented 10 months ago

Hey there,

I don't know if I'm missing something, but I'm finding it very hard to replicate a failing CI node locally.

We have a flaky test, and would like to track it down, but am unsure how to get all the right information.

Would it be possible to (optionally?) have a command output on failing tests that could be used to run it locally?

ArturT commented 10 months ago

If you are using Knapsack Pro in Queue Mode for RSpec then you can see the copy&paste command that you can use to run tests locally. It's at the end of the output for executing each batch of tests fetched from Queue API. The command uses the same RSpec seed number that you were using to run tests. Please ensure you are running the latest version of the knapsack_pro gem.

To make it work you need to use the debug log level (it's a default, KNAPSACK_PRO_LOG_LEVEL=debug). https://docs.knapsackpro.com/ruby/reference/#knapsack_pro_log_level

You can also find info how to reproduce tests locally here if you need to run tests using Regular or Queue Mode locally. https://docs.knapsackpro.com/ruby/troubleshooting/#debug-knapsack-pro-on-your-development-environmentmachine

ArturT commented 6 months ago

We have released version 7.0.0 of the knapsack_pro gem. This version has many improvements in Queue Mode for RSpec.

It also produces the RSpec copy & paste command that you can use to reproduce tests executed on a given CI node.

@kholbekj Please update the knapsack_pro gem version.

Please find the migration steps at the following link: https://github.com/KnapsackPro/knapsack_pro-ruby/blob/master/CHANGELOG.md#700