jasmine / jasmine-npm

A jasmine runner for node projects.
MIT License
376 stars 145 forks source link

Feature Request: Allow re-running test suites without restarting #200

Closed Schommer475 closed 2 years ago

Schommer475 commented 2 years ago

Jasmine should allow users to rerun test suites without needing to restart the NodeJS process.

Expected Behavior

Jasmine should allow users to run the same test suite multiple times in the same NodeJS process, especially if Jasmine is being run programmatically using the library.

Current Behavior

Currently, Jasmine provides no way to rerun a test suite without either starting a new NodeJS process or hacking around it by modifying the require cache (which doesn't even work if you wish to write tests in ECMAScript modules).

Possible Solution

When the describe function is called, you gain access to the provided callback, give us a way to tell Jasmine to call those callbacks again. There are many ways that could be done.

Context

My team writes web services in NodeJS that we deploy inside a Docker container. During development, we connect an inspector module to Chrome Dev Tools and the tool allows us to run our jasmine tests on command from within the dev tools. We would like to be able to set breakpoints in the dev tools and then rerun our test suite with those breakpoints in place, and we would like to be able to do this without having to restart our entire service each time. However, because Jasmine will only run a test suite once within a NodeJS process, our only options are either to restart the service every time or hack around the problem by keeping our test suites commonjs modules and modifying the require cache.

sgravrock commented 2 years ago

If all you need to do is re-run the suite, that should be possible. See jasmine#1934, or create a new Env and or Jasmine. But my understanding is that you're trying to reload modified modules, replace the specs/suites/etc defined by the old copies of the modules with different ones defined by the new copies, and run the result.

The problem is that the feature you're asking for probably can't ever work well with ES modules. ES modules are immutable. There are various loader hacks that try to enable hot reloading, but all of them have serious bugs. My understanding, from the Node loader working group discussions I've seen, is that there's little hope for a seamless, bug-free way to do hot reloading or any other technique that involves modifying already-loaded modules. The design decision of immutability is just too deeply baked in to both the ES module specification and Node's implementation. I believe this also applies to CommonJS modules loaded with dynamic import, which is what Jasmine does in the default configuration.

It's one thing for a user like you to decide that they're OK with serious memory leaks plus not being able to run on Windows, or serious memory leaks plus only being able to reload top-level modules, or any of the other trade-offs imposed by various loader hacks. It would be quite another thing for me to make that choice for all Jasmine users. I also don't think it would be a good idea to add CommonJS-only functionality at a time when the JS world is clearly moving in the direction of ES modules. That could encourage users to unknowingly paint themselves into a corner where they can't migrate to ES modules.

hack around the problem by keeping our test suites commonjs modules and modifying the require cache.

Unfortunately, I think that's the best that can be done. And for the reasons I've already outlined, I think it's better for that kind of hack to live in user code rather than in Jasmine.

Schommer475 commented 2 years ago

No, we were trying to re-run the suite without modifying the modules. We just could not find anything on how to do that in the same NodeJS process. Thank you.