grosser / parallel_tests

Ruby: 2 CPUs = 2x Testing Speed for RSpec, Test::Unit and Cucumber
3.39k stars 495 forks source link

Allow TEST_ENV_NUMBER to support duplicate file groups #943

Closed jaydorsey closed 7 months ago

jaydorsey commented 8 months ago

This is a follow up to #940

After that PR was merged, I was writing a little helper script that basically let me do <script> <file> and it would basically duplicate that file name sysctl -n hw.ncpu, and then run parallel_rspec w/ the --allow-duplicates flag.

What I discovered was that the TEST_ENV_NUMBER was set to the same value for each file. I traced the root behavior back to the small change I made in this PR

Checklist

jaydorsey commented 8 months ago

hmm so I think the issue is that the groups all look the same to the code since we use index so maybe instead of the current code we could iterate the groups with each_with_index and get our index that way?

Yeah that's a good point. I wasn't 100% confident that I understood what was going on inside of the execute_in_parallel block so I went down the rabbit hole of trying to shim in some new behavior, conditionally.

I just pushed up another branch that uses Parallel#map_with_index that looks right in my head, but I'm also having trouble visualizing/tracing/wrapping my head around the block yields.

I need to test it on my other machine to make triple-sure it's doing what I think it's doing (will do that later today)

jaydorsey commented 7 months ago

I've been testing this with a little helper script I wrote to help me run 1 test multiple times in parallel:

(Edit: I've made this part of my dotfiles, with some tweaks here)

#!/usr/bin/env ruby
# frozen_string_literal: true

# Useful for running a single test multiple times, locally
#
# 1. Save this file locally as something like `bin/runit`, or put it in your path
# 2. chmod +x bin/runit
# 3. Add it to your gitignore and/or gitignore_global file
#
# Usage:
#
#    bin/runit spec/models/user_spec.rb
cpus  = (`sysctl -n hw.ncpu`.to_i - 1)
files = "#{ARGV[0]} " * cpus
exec("bin/parallel_rspec --allow-duplicates --verbose-command --prefix-output-with-test-env-number -- #{files}")

I've tested it with and without the first-is-1 flag and it all runs exactly like I expect:

  1. Test numbers appear correct w/ and w/out the flag
  2. Zero database conflicts (which it had before) because it's using the correct/distinct envs for the same file now

(I get a lot of activerecord errors before, because it was using the same test env number/database)

Before

All the same test env (not intended/desired behavior):

image

These would happen pretty often:

image

After

With this PR, test envs all look correct

image
grosser commented 7 months ago

4.6.1