Rake task does not respect 'pattern' flag

[enkessler]

Subject of the issue

RSpec CLI options read from a configuration file are treated differently depending on if they are used with the RSpec CLI or the RSpec Rake task. In particular, the --pattern flag is being ignored when running with the Rake task but works fine with the CLI.

Your environment

• Ruby version: Ruby 3.0
• rspec-core version: 3.10.1

Steps to reproduce

Here's a tiny repository: https://github.com/enkessler/RSpecRakeTaskIssue

From the project root,

1. Run bundle exec rspec
2. See both tests run and an HTML file is created
3. Delete the HTML report file
4. Run bundle exec rake run_rspec_tests
5. See that no tests are run but an HTML file is still created (thus showing that the options in .rspec are, in fact, being used)

Expected behavior

I would expect the pattern specified in the .rspec file to be used in both cases.

Actual behavior

The Rake task does use the values in the .rspec file but appears to explicitly override the pattern flag. I see rspec --pattern 'spec/**{,/*/**}/*_spec.rb' appearing in the console output of the second command. As if the Rake task runs its own RSpec process but doesn't just run it as plain old rspec.

[pirj]

The rake task passes --pattern option to ruby rspec command, and CLI options take precedence over .rspec options, see https://relishapp.com/rspec/rspec-core/v/3-10/docs/configuration/read-command-line-configuration-options-from-files

I suggest setting the t.pattern = option in your Rakefile to the same pattern you use in .rspec as a workaround.

It's a mystery to me why we always explicitly pass the default --pattern from the rake task to exe/rspec.

[JonRowe]

It's a mystery to me why we always explicitly pass the default --pattern from the rake task to exe/rspec.

The history here is that the rake file allows configuring a pattern, independently of rspec. It always has, since before RSpec was RSpec and before the pattern cli option existing. Originally it applied the pattern itself and produced a list of files, but this caused an edge case where no matching files would cause all files to run (because an empty file list just causes `rspec spec). So changing to passing the pattern through was the fix for that.

I'm not sure why when there is no pattern, we don't just let RSpec do it's thing, I think it's an oversight, and we should only set the pattern when there is a pattern to set.

[pirj]

Sounds like something that we should better change in 4.0?

[JonRowe]

I don't know, there should be no behaviour change by not sending the pattern when its not set, it should be the same as rspec default

[enkessler]

The rake task passes --pattern option to ruby rspec command

Right...and that seems bad? Having a default pattern is a fine idea but having the Rake task provide that default seems like making the decision too early in the process because there could be (and in this case is) another source of configuration information to use.

I suggest setting the t.pattern = option in your Rakefile to the same pattern you use in .rspec as a workaround.

That'll be my approach in the short term and I'll have to stick the pattern into an ENV variable that I can then embed in my .rspec file with ERB so that I don't have to maintain two duplicate patterns but, as you say, it's a workaround. So I look forward to the new behavior in 4.0. :)

[pirj]

Would you like to hack on it @enkessler ?

[enkessler]

@pirj Unfortunately, I am busy trying to update half a dozen projects already (for which I am the sole maintainer). So, while I tend to stumble into issues like this one, I usually don't have the time to dig in and fix it myself.

[pirj]

Sure, I understand, @enkessler. No problem.

[pirj]

Fixed in #2868, will be released as part of RSpec 4.0.