silent failures

[choxi]

I tried running the example in the docs but with an obvious error at the top:

// example.js
FOOBAR

describe('Google searching', function() {
  before(function() {
    casper.start('http://www.google.fr/')
  })

  it('should retrieve 10 or more results', function() {
    casper.then(function() {
      'Google'.should.matchTitle
      'form[action="/search"]'.should.be.inDOM.and.be.visible
      this.fill('form[action="/search"]', {
        q: 'casperjs'
      }, true)
    })

    casper.waitForUrl(/q=casperjs/, function() {
      (/casperjs/).should.matchTitle
    })
  })
})

and I ran it with:

$ mocha-casperjs example.js

  0 passing (1ms)

I would have expected an error to be raised like mocha does:

mocha example.js                                                                                                           

example.js:1
(function (exports, require, module, __filename, __dirname) { FOOBAR
                                                              ^

ReferenceError: FOOBAR is not defined
    at Object.<anonymous> (/Users/Choxi/Code/Bloc/mocha-casperjs/example.js:1:63)
    at Module._compile (module.js:413:34)
    at Object.Module._extensions..js (module.js:422:10)
    at Module.load (module.js:357:32)
    at Function.Module._load (module.js:314:12)
    at Module.require (module.js:367:17)
    at require (internal/module.js:16:19)
    at /usr/local/lib/node_modules/mocha/lib/mocha.js:220:27
    at Array.forEach (native)
    at Mocha.loadFiles (/usr/local/lib/node_modules/mocha/lib/mocha.js:217:14)
    at Mocha.run (/usr/local/lib/node_modules/mocha/lib/mocha.js:469:10)
    at Object.<anonymous> (/usr/local/lib/node_modules/mocha/bin/_mocha:404:18)
    at Module._compile (module.js:413:34)
    at Object.Module._extensions..js (module.js:422:10)
    at Module.load (module.js:357:32)
    at Function.Module._load (module.js:314:12)
    at Function.Module.runMain (module.js:447:10)
    at startup (node.js:146:18)
    at node.js:404:3

Am I doing something wrong?

[nathanboktae]

Am I doing something wrong?

Nope not at all. Thanks for bringing up the issue. mocha-casperjs should be spitting out the error. using plain casperjs it looks something like this:

ReferenceError: Can't find variable: FOO

  phantomjs://code/derp.js:1 in global code
  :0 in injectJs
  phantomjs://code/bootstrap.js:435

Which is the best it can get likely, since it's not V8.

[brettjonesdev]

If casper is printing these errors, is there no way for mocha-casperjs to pick those up and echo them back??

[nathanboktae]

It should be outputing them, yes. It is a legit issue.

[brettjonesdev]

OK cool, I thought your previous comment meant it was unfixable, phew! :) Btw awesome work on this project, we've been using casper's built-in tester module for our integration test suite, but it's so janky compared to mocha/chai. This is exactly what we were looking for, and I'm excited to start migrating our tests over!

[TimPerry]

Any update on this?

[nathanboktae]

No, I'm quite busy in my life for the next few months. PRs are always welcome.

[TimPerry]

Fair enough, would you be too busy to point me in the right direction?

[nathanboktae]

In cli.js when the tests are required, it should throw with an exception and information if the js file has syntax errors or if it throws, i may just be swallowing it. Casper at 1.1 beta3 had this monkey patched maybe it was swallowing it. Not sure currently.

[TimPerry]

Righto, thank you. We are going to skip using this project for now but hope to circle back round so perhaps then I can get a PR sorted.

[stretchkennedy]

The error might be in phantomjs (more specifically, in qtwebkit). According to https://github.com/ariya/phantomjs/issues/14399 the fix for https://github.com/ariya/phantomjs/issues/12973 (https://github.com/Vitallium/qtwebkit/commit/e09b87bfe2b3ad6004d31894ee58bd611c8e3e39) is on master but hasn't been released yet.

If I'm right, a workaround would be to use phantomjs 1.9.8 instead of 2.1.1. I'll play around a bit more and try to confirm.

[thom-nic]

Experiencing this as well. Has anyone found a workaround that works with phantom 2.x?

[mchampanis]

@thom-nic I built phantomjs 2.1.1 with the supplied patch and it now displays errors, but doesn't exit; not sure if that is expected behaviour or not

[thom-nic]

I'm running phantomjs v2.1.1 installed via homebrew on Mac.

If I have any test that throws an error when it is evaluated, e.g. syntax error, dereference an undefined, etc. it just silently skips that test. Or if I require() anything that generates an error, the require() just returns {} instead of throwing an exception.

EDIT: I just dropped down to running a straight-up casperjs test (using the same casperjs module and system phantomjs) and errors propagate as expected, even from require()d non-test modules. So I'm not sure - at least in my case - this has anything to do with the phantomjs version...?

[andre-bonfatti-movile]

@thom-nic in my case, require() inside cli.js is always returning {}, even if the require was successful. I'm using phantomjs v2.1.7. Same thing with v2.1.1 on Linux

[thom-nic]

I'll be honest I moved to a webdriver-based E2E testing framework and, while I miss some of the convenience and features that phantom/casper offered, webdriverio + chromedriver ended up being a much better experience overall. Their gitter channel is very active and helped me figure out the differences in idiomatic testing patterns using their API vs what I'm familiar with in Phantom. Plus I have the option to conceivably run the tests on Firefox and IE, etc as well.

See this gist which gives you a pretty similar experience (it auto-starts the chromedriver so you don't have to think about running a selenium server) versus what you'd expect with Phantom/ Casper/ Capybara.

[nathanboktae]

in my case, require() inside cli.js is always returning {}, even if the require was successful.

That is casperjs' patchedRequire stuff that also caused me a lot of pain when creating this project. Not sure the shape of it now that n1k0 no longer runs casperjs.

while I miss some of the convenience and features that phantom/casper offered, webdriverio + chromedriver ended up being a much better experience overall.

Totally understood. while phantomjs / casperjs is great when it's going (I was seeing tests run 5x faster than selenium + chrome in 2013 and very derministically) it's a pain to debug. Maybe phantomjs2 has this story a bit better with better dev tools? Also now Chrome can go headless on linux via xvbf.