search

benwbrum / fromthepage

FromThePage is a wiki-like application for crowdsourcing transcription of handwritten documents.
http://fromthepage.com
GNU Affero General Public License v3.0
170 stars 50 forks source link

Isolate feature tests #1227

Open jackweinbender opened 5 years ago

jackweinbender commented 5 years ago

A persistent source of pain for me has been the fact that our test suite is very linear and ordered which means running a single test (or set of tests) in isolation often results in a failure because they rely on some test having been run prior. This makes it difficult to utilize feature tests for anything other than a final check on a PR, since it takes so long to run the full suite.

In theory, Rspec should be rolling back transactions after each test, but my very unscientific and anecdotal intuition is saying that it's not working like that for our tests. Maybe something to do with AJAX stuff?

Anyhow, If we can get our tests isolated, it will make refactoring them easier in the future. This also opens us to being able to parallelize them, should we so desire.

So, I'm suggesting we do a little detective work to see how many of our tests actually are "ordered". As a first step, we could try running each of _spec.rb files, beginning with the last, in isolation to see which ones presuppose others. At least then we would be isolating any orderedness to particular files.

jackweinbender commented 5 years ago

Currently in the spec_helper.rb file we have transactional fixtures disabled. Is this intentional? It will definitely break our tests if we enable it, but it's probably something we should look at using.