serpapi / turbo_tests

Run RSpec tests on multiple cores. Like parallel_tests but with incremental summarized output. Originally extracted from the Discourse and Rubygems source code.
https://rubygems.org/gems/turbo_tests
MIT License
173 stars 26 forks source link

Support parallel_test options: pattern, exclude-pattern & group-by #41

Open inkstak opened 1 year ago

inkstak commented 1 year ago

This PR add suport for some parallel_test options:

    -p, --pattern [PATTERN]          run tests matching this regex pattern
        --exclude-pattern [PATTERN]  exclude tests matching this regex pattern
        --group-by [TYPE]            group tests by:
                                     found - order of finding files
                                     steps - number of cucumber/spinach steps
                                     scenarios - individual cucumber scenarios
                                     filesize - by size of the file
                                     runtime - info from runtime log
                                     default - runtime when runtime log is filled otherwise filesize
inkstak commented 1 year ago

There could be one issue around --group-by default: it relies on private methods from parallel_tests : https://github.com/serpapi/turbo_tests/pull/41/files#diff-36313b614b64b302e0ceec80e7a9a2dbaea194523dc46a44b7fffda7741674a5R109

inkstak commented 1 year ago

I would also suggest another PR to handle properly Fuubar #23 depending on --pattern and --exclude-pattern. It'll also required one of those private methods.

kalashnikovisme commented 10 months ago

Hey, @ilyazub! Do you have a decision about the level of dependency on parallel_tests? :slightly_smiling_face:

NOTE: as I know, these options belong to rspec.

steobrien commented 10 months ago

+1, this doesn't increase the dependency on parallel_tests (although this gem is a drop-in replacement for that, so should support anyway), because --pattern and --exclude-pattern are RSpec flags

Any chance you could merge, @ilyazub?

inkstak commented 10 months ago

FYI: --pattern & --exclude-pattern are both supported by RSpec & parallel_tests but they don't use the same implementation and doesn't support the same arguments.

For example, we use a CI script to launch our specs :

if parallel
  command = "bundle exec parallel_rspec"
  command = "bundle exec turbo_tests" if parallel == "turbo_tests"

  command += " --exclude-pattern spec/system" if arg == "unit"
  command += " --pattern spec/system"         if arg == "system"
else
  command = "bundle exec rspec"

  command += " --exclude-pattern 'system/**/*_spec.rb'" if arg == "unit"
  command += " --pattern 'system/**/*_spec.rb'"         if arg == "system"
end

In this PR, I forward those arguments to parallel_tests so it uses its implementation and not the one from RSpec.

inkstak commented 7 months ago

Because we want to support most of the ParallelTests options, may we call ParallelTests::CLI.send(:parse_options!, @argv) in TurboTests::CLI#run?

PR updated ! You can pass any options supported by parallel_tests after -- :

bundle exec turbo_tests -n 4 -- --only-group 1 --pattern spec/system

Because OptionParser raises an exception on unknown options, I found it easier to split options like this.
Parallel_tests suggests the same syntax to pass test options :

parallel_rspec -- -t acceptance -f progress -- spec/foo_spec.rb spec/acceptance
darokel commented 6 months ago

Thanks for all your work on this. @ilyazub Is this close to being merged? @ilyazub looks like the changes have been addressed - are we good to merge this?