Open gap777 opened 9 months ago
@gap777 I use rspec --only-failures
for that. Currently, this is not supported with turbo_tests
.
I expect that passing the extra options to RSpec will be enough to support this feature (https://github.com/serpapi/turbo_tests/issues/14#issuecomment-898405912).
I work with @gap777.
I realized that rspec --only-failures
couldn't find failed examples reliably because the parallel runners invoked by turbo_tests
were writing to the same status file and clobbering each other's output. I changed our spec/spec_helper.rb
to use a different file name per runner:
config.example_status_persistence_file_path = "spec/examples#{ENV['TEST_ENV_NUMBER']}.txt"
We already use a shell script to invoke turbo_tests
, so I added a liine to consolidate those files at the end:
cat spec/examples?*.txt > spec/examples.txt
So for now, we can at least run (non-parallel) rspec --only-failures
after turbo_tests
.
@douglasshuang Thanks to your feedback!
Please note that combining multiple examples*.txt
may lead to the RSpec failure. RSpec status persistence files have a header which is expected to be present once (ref: ExampleStatusParser#initialize
).
# Concatenate multiple files with the same header
# https://unix.stackexchange.com/a/170692/33308
$ (head -2 spec/examples1.txt && tail -n +3 -q spec/examples?*.txt) > spec/examples.txt
# Checking it worked
$ wc -l spec/examples*.txt
73015 spec/examples.txt
46467 spec/examples1.txt
26550 spec/examples2.txt
146032 total
# Header is present only once
$ grep example_id spec/examples*.txt
spec/examples.txt:example_id | status | run_time |
spec/examples1.txt:example_id | status | run_time |
spec/examples2.txt:example_id | status | run_time |
@ilyazub Thank you very much for the refinement!
Is it possible to re-run the tests limited to those that failed on the last execution?