sinonjs / sinon

Test spies, stubs and mocks for JavaScript.
https://sinonjs.org/
Other
9.63k stars 769 forks source link

Test suite is not re-runnable using `--watch` due to keeping state #2502

Closed fatso83 closed 1 year ago

fatso83 commented 1 year ago

Describe the bug When using the --watch option of Mocha the tests start failing on any change

To Reproduce Steps to reproduce the behavior:

  1. npx mocha --watch test
  2. Change any test file
  3. See the test suite break with 4 failing tests
``` 1259 passing (3s) 2 pending 4 failing 1) assert message assert.calledWith match.defined exception message: AssertionError: [assert.equals] 'expected doSomething to be called with arguments \n \x1B[31m\x1B[31mdefined\x1B[0m\x1B[0m' expected to be equal to 'expected doSomething to be called with arguments \n \x1B[31mdefined\x1B[0m' + expected - actual expected doSomething to be called with arguments - defined + defined at Object.fail (node_modules/@sinonjs/referee/lib/create-fail.js:5:21) at Object.fail (node_modules/@sinonjs/referee/lib/define-assertion/index.js:47:17) at assertion (node_modules/@sinonjs/referee/lib/define-assertion/index.js:65:11) at Function.referee.. [as equals] (node_modules/@sinonjs/referee/lib/define-assertion/index.js:92:22) at Context. (test/assert-test.js:1847:20) at processImmediate (node:internal/timers:466:21) 2) assert message assert.calledWith match.truthy exception message: AssertionError: [assert.equals] 'expected doSomething to be called with arguments \n \x1B[31m\x1B[31mtruthy\x1B[0m\x1B[0m' expected to be equal to 'expected doSomething to be called with arguments \n \x1B[31mtruthy\x1B[0m' + expected - actual expected doSomething to be called with arguments - truthy + truthy at Object.fail (node_modules/@sinonjs/referee/lib/create-fail.js:5:21) at Object.fail (node_modules/@sinonjs/referee/lib/define-assertion/index.js:47:17) at assertion (node_modules/@sinonjs/referee/lib/define-assertion/index.js:65:11) at Function.referee.. [as equals] (node_modules/@sinonjs/referee/lib/define-assertion/index.js:92:22) at Context. (test/assert-test.js:1858:20) at processImmediate (node:internal/timers:466:21) 3) assert message assert.calledWith match.falsy exception message: AssertionError: [assert.equals] 'expected doSomething to be called with arguments \n' + '\x1B[32mtrue\x1B[0m \x1B[31m\x1B[31mfalsy\x1B[0m\x1B[0m' expected to be equal to 'expected doSomething to be called with arguments \n' + '\x1B[32mtrue\x1B[0m \x1B[31mfalsy\x1B[0m' + expected - actual expected doSomething to be called with arguments -true falsy +true falsy at Object.fail (node_modules/@sinonjs/referee/lib/create-fail.js:5:21) at Object.fail (node_modules/@sinonjs/referee/lib/define-assertion/index.js:47:17) at assertion (node_modules/@sinonjs/referee/lib/define-assertion/index.js:65:11) at Function.referee.. [as equals] (node_modules/@sinonjs/referee/lib/define-assertion/index.js:92:22) at Context. (test/assert-test.js:1869:20) at processImmediate (node:internal/timers:466:21) 4) stub non enumerable properties create stub from function on object: AssertionError: [assert.equals] [ 'aProp', 'func2' ] expected to be equal to [ 'aProp' ] + expected - actual [ "aProp" - "func2" ] at Object.fail (node_modules/@sinonjs/referee/lib/create-fail.js:5:21) at Object.fail (node_modules/@sinonjs/referee/lib/define-assertion/index.js:47:17) at assertion (node_modules/@sinonjs/referee/lib/define-assertion/index.js:65:11) at Function.referee.. [as equals] (node_modules/@sinonjs/referee/lib/define-assertion/index.js:92:22) at Context. (test/stub-test.js:180:20) at processImmediate (node:internal/timers:466:21) ```

Expected behavior Immutable, non-failing test runs with no shared global state.

Context (please complete the following information):

Additional context

This is due to two things that I could see

  1. We change the prototype of Function in https://github.com/sinonjs/sinon/pull/1968/files#diff-ebffa5a039be75d22925d266e16de1f0dbf14a625e5998bb0982e8523ce791c9R1332-R1337
  2. Some tests seem to keep state that makes re-runs of tests fail
fatso83 commented 1 year ago

Was closed in #2504