jasmine / jasmine-npm

A jasmine runner for node projects.
MIT License
376 stars 145 forks source link

Regression in jasmine 5: settings bleed over to next `Jasmine` instance #207

Closed nicojs closed 1 year ago

nicojs commented 1 year ago

Since Jasmine 5, settings seem to bleed over in the next Jasmine instance. See this example:

import Jasmine from 'jasmine';

const jasmine = new Jasmine();
jasmine.exitOnCompletion = false;
jasmine.loadConfigFile()
await jasmine.execute();

console.log('jasmine 2')
const jasmine2 = new Jasmine();
jasmine2.exitOnCompletion = false;
await jasmine2.execute();

Expected:

Randomized with seed 78563
Started
..........

10 specs, 0 failures
Finished in 0.019 seconds
Randomized with seed 78563 (jasmine --random=true --seed=78563)
jasmine 2
Randomized with seed 93183
Started

No specs found
Finished in 0.001 seconds
Incomplete: No specs found
Randomized with seed 93183 (jasmine --random=true --seed=93183)

Actual:

Randomized with seed 20839
Started
..........

10 specs, 0 failures
Finished in 0.018 seconds
Randomized with seed 20839 (jasmine --random=true --seed=20839)
jasmine 2
Randomized with seed 98398
Started
Randomized with seed 98398
Started
FFFFFFFFFFFFFFFFFFFF

Failures:
1) Player should be able to play a Song
  Message:
    TypeError: Cannot read properties of null (reading 'length')
  Stack:
        at <Jasmine>
        at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
 [...]

 10) Player 2 when song has been paused should be possible to resume
  Message:
    TypeError: Cannot read properties of null (reading 'length')
  Stack:
        at <Jasmine>
        at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

See reproduction repo: jasmine-issue.zip

node run-jasmine.js reproduces it.

Downgrading jasmine fixes the issue:

npm i -D jasmine@4

Background info: StrykerJS relies on the jasmine API to run mutation testing. Although we generally only create 1 instance of jasmine, we still want to rely in settings not bleeding over to the next instance.

sgravrock commented 1 year ago

Can you try the npm-multiple-runs branch e.g. npm install --save-dev github:jasmine/jasmine-npm#npm-multiple-runs? It works for your minimal repro and I think it fixes all the relevant failures in the stryker-js repo, but I get some (probably extraneous) failures even on master so I wanted to double-check before releasing the fix.

nicojs commented 1 year ago

💚 Seems to work: https://github.com/stryker-mutator/stryker-js/pull/4190 https://github.com/stryker-mutator/stryker-js/actions/runs/5187773793/jobs/9350550686?pr=4190 (unsure if you can see this link)

sgravrock commented 1 year ago

TY. I'll try to get that released later this week.

sgravrock commented 1 year ago

This is released in 5.0.1.

Note that the bug is still present if you do new Jasmine({globals: false}). And it always has been, all the way back to 4.0 when that option was first introduced. Fixing that in a sane way probably requires a major rethink of the no-globals design, so I'm probably going to leave it alone unless somebody actually says that they need that particular combination of features.