search

jejacks0n / teaspoon

Teaspoon: Javascript test runner for Rails. Use Selenium, BrowserStack, or PhantomJS.
1.43k stars 243 forks source link

Fail test on parse error #425

Closed pratik60 closed 9 years ago

pratik60 commented 9 years ago

Hi Guys

Really wondering why tests are not considered to fail when there is a parse error? Because of a parse error, it didn't run all the tests, and definitely should have ultimately result in a fail.

Any reason why the current behaviour is like that?

Starting the Teaspoon server... ... Teaspoon running default suite at http://127.0.0.1:61748/teaspoon/default SyntaxError: Parse error

...................................................................................................................................................................................................................................................................................

Finished in 4.42400 seconds 277 examples, 0 failures, 2 pending

mikepack commented 9 years ago

How are you producing the parse error? The success or failure of a build is determined by whether you have any failing specs. In your example, nothing failed, so that's why it's a successful build.

jejacks0n commented 9 years ago

We get these issues from time to time. It's because the order of how JavaScript is loaded and parsed. If there's a parse error, it's long before any JavaScript has the opportunity to do anything, and regardless, there's not a JavaScript exception raised or anything.

We could punt on execution of the specs (maybe), but a parse error is not indicative of if your javascript execute in the real world or not. So in stead you get the message.

pratik60 commented 9 years ago

@mikepack @jejacks0n - We have 302 tests that are there...Because of a parse error, I'm assuming in the tests itself, some of them just didn't run!

Surely that is something that should be caught by teaspoon?

jejacks0n commented 9 years ago

I'm willing to entertain it. Feel free to submit a pr.

mikepack commented 9 years ago

I still don't understand how this error is being produced. Syntax errors in the specs or implementation bubble up as errors in the console output. @pratik60 can you explain how to produce this error?