mikepack
commented
9 years ago Hi Arin. My gut tells me it is not Teaspoon that is causing the random failures, but the tests within your application. If Teaspoon reported a failure and you're seeing a failure in your suite, Teaspoon has done its job at that point and the test is truly failing. If you're seeing a dropped or missing test, then it could be Teaspoon.
Random failures are often due to the order in which the tests run. When trying to debug them, I usually look for some global state that could be corrupt when the tests run in a specific order. Things like storing data on classes instead of instances, or on the prototype that is shared between objects, or some underlying persistent store, like Local Storage.
Can you confirm that you are seeing a test failure and not a dropped test? Are other tests passing?
Rails 4.1.0 Teaspoon ~> 0.9.1 Using Qunit Ember JS
I have manually tested over and over in my teaspoon environment with same data after failures. The coverage of operations in these tests are functional. However at random, our teaspoon environment will send a failure on a couple individual tests within a specific folder or pass them all. This also happens when using codeship as well. I know you can't really track down the issue, but even providing some advice or possible problems that could cause these failures would be helpful.
I'm sure lots of developers ran into these issues before.
Thank you, Arin