Flaky tests on 1.0.6

[sebakerckhof]

My tests have gotten flaky since going to 1.0.6 (the version wasn't locked, so I'm not sure what version I was running before, but it was about 2 weeks ago they were last being run).

Before, collection hooks weren't being called at all, and now suddenly they are being called. But since I typically don't want these hooks to be called when running unit tests (well, except if I'm testing the hook itself), I changed all my insert/update/remove calls to be on the .direct collection.

And while the changelog says these should also be stubbed, the behavior seems to be very flaky.

It seems to run correct on the first run, but on a rebuild (e.g. when changing a test) things seem to go wrong every so often and in random ways.

Errors coming from npm-mongo on collections that should have been stubbed, like:

‣
Error: E11000 duplicate key error collection: meteor.meetings index: _id_ dup key: { : "aaa" }WriteError({"code":11000,"index":0,"errmsg":"E11000 duplicate key error collection: meteor.meetings index: _id_ dup key: { : \"aaa\" }","op":{"_id":"aaa","state":2,"endDate":"2017-11-28T17:52:37.668Z"}})
    at Function.MongoError.create (

And find-s that seems to not find results that have clearly been inserted.

[hwillson]

Interesting, thanks for the heads up @sebakerckhof. Sounds like we need to take https://github.com/hwillson/meteor-stub-collections/pull/21 a bit further. For now pinning to 1.0.3 should let you move forward. Thanks!

[Floriferous]

I've seen the same behavior, took me a while to track it down to this package, was pulling my hair out as the failure counter on my tests would change every time I refresh the page!!

The update to 1.0.6 created lots of failing tests with very inconsistent behavior for me. I'm not sure what exactly happened, but among them, the following:

• E11000 duplicate key error
• simple-schema defaultValues were suddenly set (expected behavior, but didn't work before)
• collection2 validationContext was suddenly set on all collections, and complex update errors started appearing ($push in a nested array would trigger an error on another nested string not being there, even though it was...)
• stubbed collections would stop working from time to time (documents became undefined in a few random tests)

The interplay between this package, simple-schema, collection2, meteor-factory, and xolvio:cleaner has been pretty messy for me for a long time. Too bad the meteor docs only provide a few examples, from which I've had trouble extrapolating for my own needs, a year into meteor, I still don't know how to properly test meteor-related stuff 😂