lorenzofox3 / zora

Lightest, yet Fastest Javascript test runner for nodejs and browsers
MIT License
539 stars 93 forks source link

Add ability to check the exact number of assertion calls per-test #102

Closed bompus closed 3 years ago

bompus commented 3 years ago

Over the years, with various test runners / assertion libraries, I've encountered several false-passing tests due to assertions inside of async / promise resolutions, callbacks, and mocks not actually executing.

The only test runner that I'm aware of which has this check built-in is Jest with expect.assertions(number).

Developers have requested this feature of other libraries (7-year long chai issue/discussion) , but most have not come up with an eloquent way to handle it.

bompus commented 3 years ago

I believe this can eloquently be handled by Zora, using either the test() expectAssertions option proposed in PR #103 or by some other means, but the end goal can be achieved easily because Zora stores assertion results as an array already, instead of throwing, making it easier to add features like this.

lorenzofox3 commented 3 years ago

It is actually implemented in AvA and Tape too (under .plan()). Although that 's kind for legacy reasons, when node was all CPS (continuous passing style). Nowadays with new control flows (such ascync/await) I find very few usage of it. See the take of AvA on the topic.

Do you have concrete examples to share where you feel it really has an added value ?

bompus commented 3 years ago

I actually solved it in userland yesterday while playing around in mocha/chai/expect on another project. The (weird) but valid example can also be seen at https://github.com/chaijs/chai/issues/94#issuecomment-925403907

lorenzofox3 commented 3 years ago

Yes that is what I thought. Usually for this kind of legacy code, I simply test on side effects:

test('check callback error', async (t) => {
    let error;

    await promiseCbAsync(badFnAsync(), (err, rtn) => {
      error = err
    });

    t.eq(error?.message, 'expectedErrorMessage')
})
lorenzofox3 commented 3 years ago

So I close this for now

bompus commented 3 years ago

Yes, right after I sent the last reply and thought about it some more, just assign something a variable inside the callback, then assert it after the await.

Consider this closed, as the example you posted back is both much simpler and easier for folks to understand looking at the test.