slow tests

[thenrio]

how test do run on your laptop ?

~/src/ruby/propile [kill*]$ bundle exec rspec spec
Run options:
  include {:focus=>true}
  exclude {:broken=>true}

All examples were filtered out; ignoring {:focus=>true}
..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................

Finished in 1 minute 44.5 seconds
834 examples, 0 failures

this is damn too slow!

how do you other are doing ? ( yes I know my laptop is dying )

[dboissier]

Postgre probably...

[oelmekki]

Hello Thierry,

I have such "poor" performances - albeit less than two minutes full suite spec run is quite acceptable for a mature application, that's usually the time when we introduce continuous integration. We could do way better, here.

The main problem is that test suite is rather poorly designed. At first, there was no integration test layer, which I added. Controllers specs test about everything. They are sometime unit testing, some other times, they check generated view content, as a pseudo integration test.

Anyway, FactoryGirl is always misused, in those : FactoryGirl#create is called and #save is then stubbed on instances, rather than using FactoryGirl#build_stubbed from the start. Suite execution time could be dramatically reduced using the later.

[thbar]

One common issue with small test suites that last too long is unexpected connections to the internet in the background (which we can forbid with webmock for instance).

We can profile slow specs with rspec --profile (http://myronmars.to/n/dev-blog/2013/07/rspec-2-14-is-released) to figure out which ones are the culprit, so we figure out where an effort is useful.

[thenrio]

good catch --profile

Top 10 slowest examples (23.27 seconds, 22.2% of total time):
  Account creation visiting config page as a maintainer has a link to create account
    7.14 seconds ./spec/features/account_creation_spec.rb:15
  Account confirm maintainer it should behave like a confirmable account with_password with valid password makes it not reset
    2.33 seconds ./spec/models/account_spec.rb:197
  Account::PasswordsController when logged in PUT 'update' with valid params removes the reset state if needed
    2.11 seconds ./spec/controllers/account/passwords_controller_spec.rb:42
  Account confirm presenter it should behave like a confirmable account with_password with valid password makes it not reset
    2.09 seconds ./spec/models/account_spec.rb:197
  Account creation visiting config page as a maintainer clicking new account link filling account form to create maintainer behaves like account_creation hides form
    1.8 seconds ./spec/features/account_creation_spec.rb:37
  Account creation visiting config page as a maintainer clicking new account link filling account form to create maintainer behaves like account_creation hides form
    1.75 seconds ./spec/features/account_creation_spec.rb:37
  SessionsController GET all activity rss returns all sessions

now still might have to dig, or might not cheers all

[oelmekki]

Be aware, though, that it's perfectly normal tests under features/ are the longer one to run, individually. They are integration tests, and use database, browser, and realistic user visits simulations.

On the contrary, we should not see any controllers/ tests in the slower tests, since they are supposed to be fully mocked up (which is not the case).

Furthermore, on my machine, controller specs takes about 45 secs to run, while feature specs, albeit containing the slowest individual specs, is only about 9 secs.

We could certainly reduce feature specs execution time, for example :

• we could only go through login page once, then force auth state
• we could create elements in db in before :all rather than before :each

I'm a bit reluctant to implement the second one on a community project, because it implies strict policy in spec design, like not reusing two times the same record in two different tests (or else, rspec random spec order would sometime run the delete test before the edit test which would try to use the deleted record).

Anyway, that's quite insignificant right now since controller specs take about 5 times more time. We should ensure controller specs do not hit the database, using #build_stubbed.