mysociety / alaveteli

Provide a Freedom of Information request system for your jurisdiction
https://alaveteli.org
Other
389 stars 195 forks source link

Add minimal CI test run for quicker feedback #6295

Open garethrees opened 3 years ago

garethrees commented 3 years ago

We know the full spec suite takes a long time to run https://github.com/mysociety/alaveteli/issues/626.

One problem this causes is that just getting a sense check of some feature development takes quite a bit of time.

We've made a couple of attempts at adding parallel specs over the years, but Xapian makes life difficult there.

Instead of parallelising the whole suite, what we could do is add a ci-minimal.yml which runs individual spec directories as a poor-man's parallelisation. FMS do this for their coverage specs. Could split up like:

- name: Test models
  if: matrix.part == 1
  run: bundle exec rspec spec/models

- name: Test views
  if: matrix.part == 2
  run: bundle exec rspec spec/models

- name: Test controllers
  if: matrix.part == 3
  run: bundle exec rspec spec/models

- name: Test integration
  if: matrix.part == 4
  run: bundle exec rspec spec/integration

# This is a bit of a gotcha in that we might easily miss a new directoy
# (e.g. spec/jobs). Maybe we can figure out a file glob that excludes the above
# directories
- name: Test everything else
  if: matrix.part == 5
  run: bundle exec rspec spec/lib spec/mailers spec/script spec/services spec/validators

This might at least make life a bit easier for one of the problems caused by slow specs.

garethrees commented 3 years ago

Oops, like I just missed spec/helpers out of the above list. Definitely want a glob.