search

nathanboktae / mocha-casperjs

Write CasperJS tests using Mocha
MIT License
120 stars 30 forks source link

Using Mocha > 2.3.5 and retry options, tests that fail are considered successful #77

Closed youcandanch closed 5 years ago

youcandanch commented 8 years ago

@nathanboktae Planning on hacking on this a bit tonight but I'll flesh out the issue if it turns out I don't have time - long story short, Mocha 2.3.5 introduced the concept of retrying failed tests, which is rad for tests that may flake due to network issues, timeouts, etc. That said, trying to run it locally with mocha-casperjs results in the tests passing instead of going back into the retry.

Nothing broken here as mocha-casperjs was never designed with this functionality in mind, but it'd be cool to get it in. I'll post some more info if it turns out I don't have time to hack on this.

nathanboktae commented 8 years ago

hmmm... seems tricky. Considering how stateful your remote web page is under test, and casperjs too, it seems very problematic - definitely something you'd want to opt-in too.

Good luck, let me know what you learn.

youcandanch commented 8 years ago

Yeah, it's definitely an opt-in. It's basically:

describe 'a test that should retry three times if it fails', ->
    @retries 3

...which is neat if you're running post-deployment tests on a server that's got a spin-up penalty, and the failures you see may be transient (which is definitely the case for a lot of smoke tests).