Intermittent CI failures

[sandbergja]

Expected behavior

CI passes

Actual behavior

Sometimes it passes, sometimes it fails

[sandbergja]

There are a few tests that pass when run individually, but fail if you run the entire test suite.

Locally when I run the whole test suite locally, I get these failures. Each of these tests is fine in isolation.

rspec ./spec/features/login_account_spec.rb:41 # Account login Library Account login as an unauthenticated user redirects to the log in page and then to alma
rspec ./spec/requests/left_anchor_spec.rb:73 # left anchor search solr operator characters are handled in advanced search
rspec ./spec/requests/request_spec.rb:20 # blacklight tests advanced handling when multiple fields handles it
rspec ./spec/requests/request_spec.rb:383 # blacklight tests series_display in search results is fetched when doing a more in this series search
rspec ./spec/requests/request_spec.rb:404 # blacklight tests notes field in advanced search record with notes field is retrieved
rspec './spec/system/left_anchor_spec.rb[1:1:2:1]' # left anchored searching old query dsl and advanced search behaves like an advanced search uses a wildcard for left anchored search
rspec './spec/system/left_anchor_spec.rb[1:1:2:2]' # left anchored searching old query dsl and advanced search behaves like an advanced search uses a wildcard for left anchored search from the second search box
rspec ./spec/system/searching_spec.rb:106 # Searching searching for series title from advanced search displays the online availability
rspec ./spec/system/searching_spec.rb:125 # Searching with an invalid field list parameter in the advanced search will return results without an error
rspec ./spec/system/searching_spec.rb:196 # Searching with the built-in advanced search form can add a facet to an existing search

When I ssh into CircleCI's runner and run the whole test suite, I get these failures. Each of these tests is fine in isolation.

rspec ./spec/requests/left_anchor_spec.rb:73 # left anchor search solr operator characters are handled in advanced search
rspec ./spec/requests/request_spec.rb:20 # blacklight tests advanced handling when multiple fields handles it
rspec ./spec/requests/request_spec.rb:383 # blacklight tests series_display in search results is fetched when doing a more in this series search
rspec ./spec/requests/request_spec.rb:404 # blacklight tests notes field in advanced search record with notes field is retrieved
rspec './spec/system/left_anchor_spec.rb[1:1:2:1]' # left anchored searching old query dsl and advanced search behaves like an advanced search uses a wildcard for left anchored search
rspec './spec/system/left_anchor_spec.rb[1:1:2:2]' # left anchored searching old query dsl and advanced search behaves like an advanced search uses a wildcard for left anchored search from the second search box
rspec ./spec/system/searching_spec.rb:106 # Searching searching for series title from advanced search displays the online availability
rspec ./spec/system/searching_spec.rb:125 # Searching with an invalid field list parameter in the advanced search will return results without an error

[sandbergja]

From bundle exec rspec --bisect spec:

The minimal reproduction command is:
  rspec './spec/features/more_in_this_series_spec.rb[1:2:2]' './spec/requests/left_anchor_spec.rb[1:7:2]' './spec/requests/request_spec.rb[1:2:1,1:21:1,1:22:1]' './spec/system/left_anchor_spec.rb[1:1:2:1,1:1:2:2]' './spec/system/searching_spec.rb[1:10:1,1:12:1,1:17:4]'