search

jmreidy / grunt-mocha-webdriver

Grunt task to run Mocha tests against a WebDriver source - either PhantomJS or Sauce Labs
MIT License
36 stars 17 forks source link

Spawn mocha as a child process #83

Closed binarykitchen closed 9 years ago

binarykitchen commented 10 years ago

IMO Mocha tests must be respawned as a child process each time to ensure test are executed in independent environments, see https://github.com/pghalliday/grunt-mocha-test/issues/64#issuecomment-56356198

Any chance this could be introduced here as a new option?

jmreidy commented 10 years ago

Hm, maybe. @saadtazi @ChrisWren thoughts on this?

saadtazi commented 10 years ago

I am not sure i understand what @pghalliday means by: "However some use cases rely on running mocha more than once in the same context (eg. collecting coverage data and running multiple reporters".

I know that spawning a new process per browser will allow grunt-mocha-webdriver to reuse existing reporters even with selenium/saucelabs: the main process can buffer stdin/stdout if concurrency > 1 and flush when the spawned process ends. I did a PoC a couple of months ago (see branch url in my comment here) and seems to work well. The only issue I had (minor in my opinion): the logs are not "in color".

pghalliday commented 10 years ago

@saadtazi I meant that in the context of grunt-mocha-test specifying multiple reporters means running mocha multiple times. Eg. to run once to report test results and collect coverage data and then run it again to run, for example, the travis-cov reporter to test if the coverage data falls short of a certain limit. A beneficial side effect of running both in the same node context is that the tests actually only run once and the coverage data collected then is still available (in a global variable) for use by the travis-cov reporter. This seems like a code smell but is probably more a problem with the Mocha architecture than the grunt plugin (it might be nicer if the results could be piped through multiple reporters at the same time)

Other users have also expressed an interest in having some test state cached between test runs (eg. mock data that takes time to setup). In this case a sensible option would be to split the set up into a separate grunt task to run before watch say and then use that data each time tests are run on changes - if the tests were spawned in a new process then this might not work.

Basically it boils down to the principle that you want your unit tests to run fast so you can run them as often as possible. Of course in the second case there maybe a trade off to make in terms of ensuring a clean and consistent state when the tests start. How that is done though, I think should be left to the developer.

So I think saying that tests must be spawned in new processes in this context is a bit short sighted and far too binary a point of view. There is value in having opinionated tools though so I leave it up to you guys ;) (I don't think i'll be doing it in grunt-mocha-test)