grosser / parallel_tests

Ruby: 2 CPUs = 2x Testing Speed for RSpec, Test::Unit and Cucumber
3.37k stars 493 forks source link

Some processes die silently, but the whole test finishes with exit 0 #907

Open mnmandahalf opened 1 year ago

mnmandahalf commented 1 year ago

Running parallel_rspec on CircleCI, I figured out some process died silently (propably with its memory issue, but I'm not sure) and finished logs are not shown. But the whole test was finished with exit code 0.

Here is the command:

bin/parallel_rspec spec --serialize-stdout --combine-stderr

bin/parallel_rspec:

#!/usr/bin/env ruby
load File.expand_path("spring", __dir__)
require 'bundler/setup'
load Gem.bin_path('parallel_tests', 'parallel_rspec')

Environment: parallel_test 4.2.1 Ruby 2.7.8(cimg/ruby:2.7.8-node)

The log (some warnings of my app omitted):

#!/bin/bash -eo pipefail
bin/parallel_rspec spec --serialize-stdout --combine-stderr
Running via Spring preloader in process 8254
Using recorded test runtime
8 processes for 1152 specs, ~ 144 specs per process
..
Running via Spring preloader in process 8311
.
Running via Spring preloader in process 8306
.

Finished in 4 minutes 36.4 seconds (files took 3.19 seconds to load)
2119 examples, 0 failures, 18 pending

.

Running via Spring preloader in process 8286

Finished in 5 minutes 9 seconds (files took 2.85 seconds to load)
2090 examples, 0 failures

Running via Spring preloader in process 8291

Finished in 5 minutes 9 seconds (files took 3.17 seconds to load)
1869 examples, 0 failures, 1 pending

.
Running via Spring preloader in process 8296

Finished in 5 minutes 39 seconds (files took 1.94 seconds to load)
1557 examples, 0 failures, 7 pending

Running via Spring preloader in process 8301

Finished in 6 minutes 39 seconds (files took 3.48 seconds to load)
2739 examples, 0 failures, 43 pending

10374 examples, 0 failures, 69 pendings

Took 405 seconds (6:45)
CircleCI received exit code 0

I use spring as preloader, but not sure if this affects. I expect if one process died, exit whole test with exit code 1.

I thought once #wait_for_other_processes_to_finish suits above case, but this methods checks the number of processes. https://github.com/grosser/parallel_tests/blob/163f53c67689e8362dc8eae635a839416e680fe0/lib/parallel_tests.rb#L82

I will appliciate if there is any idea to solve the problem.

grosser commented 1 year ago

in general I'd recommend against spring, recent rails also removed it if you can somehow reproduce this or ideally make a failing test that would help a lot