ValentinH / jest-fail-on-console

Utility to make jest tests fail when console.error() or any other methods are used
MIT License
141 stars 20 forks source link

Error after test has passed is not catched #40

Closed janpe closed 1 year ago

janpe commented 1 year ago

If a test has passed succesfully but a error is logged after all assertions, this is not catched by the utility.

ValentinH commented 1 year ago

Do you have an example of such a case?

janpe commented 1 year ago

Do you have an example of such a case?

This works as expected:

it('tests something', () => {
  console.error('hello');
  expect(1).toEqual(1);
});

But if the error comes from something asynchronous that errors after the assertion has successfully passed, catching the error won't happen. Something like this:

it('tests something', () => {
  setTimeout(() => {
      console.error('hello');
    }, 100);
  expect(1).toEqual(1);
});
ValentinH commented 1 year ago

The issue is that the console error happens when the rest is supposed to be finished. Having something run later like this without telling Jest about it is error prone. You can use the done callback to handle this properly: https://jestjs.io/docs/asynchronous#callbacks

janpe commented 1 year ago

I ran into this problem unexpectedly and Jest is able to show me these errors but this utility is not able to make the test fail because of this error. And since it was unexpected I might not be able to take it into account using callbacks. So the problem I have is that I get an error that I'd like to have fixed before a PR is merged but Github actions is unable to let me know this happens because I'm not able to make the test fail even though there is an error.

I understand that this is problematic. Do you have any suggestions or is this just something I have to live with not being able to catch these in my Github actions?

ValentinH commented 1 year ago

I'm not sure there is much we can do. Indeed, the lib relies on the afterAll callback from Jest. If the error happens after Jest calls it, we will not be able to detect it.

janpe commented 1 year ago

I'm not sure there is much we can do. Indeed, the lib relies on the afterAll callback from Jest. If the error happens after Jest calls it, we will not be able to detect it.

I appreciate you taking the time to reply in a quick manner. Thanks! I'll close this one.

ValentinH commented 1 year ago

You are welcome. Thank for your understanding!