iamchrismiller / grunt-casper

Run CasperJS Scripts/Functional Tests
Other
69 stars 38 forks source link

If a test fails, subsequent test files are not run #20

Closed bigethan closed 10 years ago

bigethan commented 10 years ago

I have

tests/integration
 test1.js
 test2.js 

My options file is:

var casper = {
  options: {
    test: true,
    'xunit_out': function(src) {
      var file = src.split('/').pop();
      return 'build/integration-xunit/' + file + '.xml';
    },
    //this doesn't work when set here.
    //viewportSize: {width: 320, height: 500},
    silentErrors: true,
    'fail-fast' : false, //don't exit after a failure
  },
  integration: {
    src: ['test/integration/*.js'],
  }
};

module.exports = casper;

If there's a failure in test1.js, the test2.js suite doesn't run. the task exits after the first suite with:

Warning: Task "casper:integration" failed. Use --force to continue.

If I runt the task with --force it still doesn't fix the behavior. Is there something that I'm missing?

YukonSaint commented 10 years ago

I am seeing the same thing. It seems that fail-fast is always set to true regardless of the options setting. This is unfortunate for us because we always have a few broken tests (yes we are very messy) - but need to refactor and know we didn't break any more tests.

iamchrismiller commented 10 years ago

I just returned to the country and will look into updating the fail fast option, updating to 1.1x and publishing to npm in the next couple of days.

iamchrismiller commented 10 years ago

I have updated grunt-casper to support the new version of CasperJS v1.1.x in the master branch and have also fixed the issue with the --fail-fast parameter. Sorry for the delay.

If you need support for CasperJS v1.0 then checkout the 1.0 tag.

YukonSaint commented 10 years ago

I am not able to download v0.2.0.

npm http 304 https://registry.npmjs.org/grunt-casper npm ERR! notarget No compatible version found: grunt-casper@'>=0.2.0-0 <0.3.0-0' npm ERR! notarget Valid install targets: npm ERR! notarget ["0.1.0","0.1.1","0.1.3","0.1.4"]

Is this something that takes a while to propagate?

iamchrismiller commented 10 years ago

I forgot to publish.. :-/

Published to npm now...

YukonSaint commented 10 years ago

Am I doing something wrong or does non-parallel only run one test then quit? Also fail-fast seems to be ignored in parallel mode. I'm running casperjs 1.1 beta 3 and grunt-casper .2.0. Here are the settings in grunt:

casper : {
  options : {
    test : true,
    includes: 'app/shared/test/setup.js',
    parallel : false,
    "fail-fast": false,
    concurrency : 3,
    'log-level' : 'warning',
  },
  files : [
    "app/entertainment/**/*.test.js",
  ]
},

If I run in parallel mode it tests all the files. If I set parallel to true, it runs one test, which passes, then quits.

iamchrismiller commented 10 years ago

Yes, The fail-fast doesn't work in parallel mode. I have to figure out how I want to catch the failure and stop spawning new casper instances. You will also want to update your files array :

files : { src : ["app/entertainment/*/.test.js"] }

chapmanc commented 8 years ago

This error still seems to be present. @iamchrismiller are you still working in this?