Any chance to speed up tests?

[lacco]

In my current test suite, each test takes about 2000-3000 ms to complete. Is there anything I can do to speed this up? Following example shows that using any kind of database interaction in tests make them slow (although I initially thought the firing up phantom js browser instances takes so long):

suite "perf", ->
  test "test 1", (done, server, client) ->
    done()

  test "test 2", (done, server, client) ->
    server.evalSync ->
      emit("return")

    done()

  test "test 3", (done, server, client) ->
    server.evalSync ->
      (new Meteor.Collection("authorss")).find("1")
      emit("return")

    done()

  test "test 4", (done, server, client) ->
    client.evalSync ->
      (new Meteor.Collection("authorss")).find("1")
      emit("return")

    done()

Output:

(I have no clue why test 4 gets a timeout, it is just a copy of test 3).

I am using Laika 0.3.0 and Meteor 0.6.5.1 .

[rickihastings]

Did you ever get anywhere with this? Right now laika is unusable for me, even running the same as your test 1 it takes nearly 2 seconds and it's doing nothing. Development seems pretty much dead aswell.

[lacco]

No, currently I am living with my slow tests, but TDD is nearly impossible :(. I really like Laika's approach, so I really hope that somebody might find a solution to speed them up.

[arunoda]

I tried to improve a bit, but not successful yet. There is two bottlenecks.

• each server per test
• each client per test

If your app has some db initialisations, that time also add up. We can get around with this. Other one is the hardest one. Since we are launching a phantomjs page, time taken to load static files and render them also add up. I tried to use implement a full DDP based client. But it contains a lot of work.

On Wed, Dec 4, 2013 at 1:43 PM, Kai Schlichting notifications@github.comwrote:

No, currently I am living with my slow tests, but TDD is nearly impossible :(. I really like Laika's approach, so I really hope that somebody might find a solution to speed them up.

— Reply to this email directly or view it on GitHubhttps://github.com/arunoda/laika/issues/62#issuecomment-29786591 .

Arunoda Susiripala

@arunoda http://twitter.com/arunoda http://gplus.to/arunodahttps://github.com/arunoda http://www.linkedin.com/in/arunoda

[rickihastings]

I think a good improvement could be to run meteor in production mode to minimize the amount of requests phantomJS has to make, and caching will come into play when running multiple tests. It should really boot things up, see;

https://github.com/arunoda/laika/blob/master/lib/meteor.js#L9

How is the mongo code working, is it dropping the database and recreating it per test? If so perhaps we could speed that up a little, I've removed the db initialisation stuff from my code if it can detect its running in a laika test suite, with a bit of hackery, it would be nice to set an environment variable, so can manipulate our Meteor app to help out laika.

I'll have a play around with some things and see if I can make some improvements and maybe do a pull request if anything works.

[arunoda]

Running the project at first with production mode sounds good, but it will slow down at initially. We can add an option for that of course.

Some better way of handling the db might help. But still we need to figure out the full db isolations.

Of course, I'd love to have some pull requests :)

On Wed, Dec 4, 2013 at 4:48 PM, Ricki notifications@github.com wrote:

I think a good improvement could be to run meteor in production mode to minimize the amount of requests phantomJS has to make, and caching will come into play when running multiple tests. It should really boot things up, see;

https://github.com/arunoda/laika/blob/master/lib/meteor.js#L9

How is the mongo code working, is it dropping the database and recreating it per test? If so perhaps we could speed that up a little, I've removed the db initialisation stuff from my code if it can detect its running in a laika test suite, with a bit of hackery, it would be nice to set an environment variable, so can manipulate our Meteor app to help out laika.

I'll have a play around with some things and see if I can make some improvements and maybe do a pull request if anything works.

— Reply to this email directly or view it on GitHubhttps://github.com/arunoda/laika/issues/62#issuecomment-29796970 .

Arunoda Susiripala

@arunoda http://twitter.com/arunoda http://gplus.to/arunodahttps://github.com/arunoda http://www.linkedin.com/in/arunoda

[rickihastings]

Yeah I've just ran with --production and it speeded the initial tests up by a little. I've not ran it with a huge suite just a few though.

It seems the bulk of my problems are coming from when I'm doing on startup checks and some file manipulations, I don't specifically need these running during tests, I can stub them. Is a reliable way to detect whether laika started Meteor up? At the moment I'm passing in a changed config file through --settings and detecting there.

[dandv]

Just wanted to report a speed difference of about 2x on the startup phase (the test fails right away) between using a regular mongod instance (56 seconds), and one instantiated with mongod --smallfiles --noprealloc --nojournal (34 seconds, of which 4 are to launch mongod and then shut it down). System is a Linode VPS with 1GB RAM, running Ubuntu 13 LTS.

[nilsjanse]

Would it be possible to optionally "reuse" server and client(s) across tests?

Conceptually (using the mocha bdd-syntax), how it seems to work now is that laika is setting up/tearing down a server and client(s) as beforeEach()/afterEach(), but if it was possible to make it as before()/after() instead, a lot of time could be saved, right?

For my app, setup of one client takes about 2 seconds, and so each two-client test takes just above 4 seconds, and this almost regardless of the complexity of the test.

I have been tinkering with making just a few ginormous tests, that run through a several user stories, and in terms of run-speed-at-success it works great. And using verbose laika and custom logging messages, I can make debugging work ok. But, it feels a bit like I am "working against the mocha framework".

Being able to run several mocha tests within one laika setup/teardown would make it possible to run acceptance tests at reasonable speeds mocha-style.

The main drawback to this approach would be that the tests would not be fully independent. One way of mitigating test interdependence could be to add a custom afterEach()-function that basically resets the database and moves all clients back to the starting page.

Thoughts?

Also, thanks @arunoda for making laika! Apart from this speed issue, I think it is great as an acceptance testing framework for meteor. You rock!

[subhog]

Would it be possible to replace Phantom.js with Zombie.js? That should speed things up significantly.

[apendua]

@subhog :+1: