debugging jest tests in vscode, stepping past inmemory mongodb triggers the test timeout

Zazcallabah commented 1 year ago

Environment

vscode-jest version: 5.2.2
node -v: 18.4.0
npm -v or yarn --version: 1.22.19
npm ls jest or npm ls react-scripts (if you haven’t ejected): 29.1.2

your vscode-jest settings if customized:

"jest.showTerminalOnLaunch": false,
"jest.testExplorer": {
"enabled": true
},
"jest.debugCodeLens.showWhenTestStateIn": ["pass", "skip", "fail", "unknown"],
"jest.autoRun": "off",
"jest.parserPluginOptions": null

Operating system: Win10

Prerequisite

Jest works fine except for the debugging and pausing on breakpoints thing. they run in our ci, from cli, through yarn, and from the vscode extension properly.

It is only when debugging, and stepping through code, and specifically only when passing mongoose/typegoose/inmemory mongodb querys that the issue appears.

Steps to Reproduce

I made a minimalistic demo repo here

clone it, yarn, then yarn test for good measure, to check the test is green.

Then open the repo in vscode, find the test/vscodejest.test.ts file, place a breakpoint at the beginning of the test, and start debug.

Step through a few times and you will be thrown out of the test, into a typegoose teardown procedure, then out of debug mode altogether.

Expected Behavior

Just keep stepping through code. Ignore the jest-timeout when debugging.

Actual Behavior

Test halts with a timeout and message

Test suite failed to run

    thrown: "Exceeded timeout of 500 ms for a hook.

connectdotz commented 1 year ago

Hmmm... this might not be related to the extension... see facebook/jest#11607 , does any suggestion there help?

Zazcallabah commented 1 year ago

I dont really see how that link is related? None of the comments are talking about step-by-step debugging through a test? They also seem to have long running tests? (given the start argument is to increase the runtime from 30 to 60 seconds. (?!)

I would say that "debugging" a test should override the test runtime timeout, no? It is not really acceptable for us to increase the runtime for our tests generally, since long-running tests is a signal we have not mocked properly. - and increasing them manually every time we debug is just asking for accidents.

Jule- commented 9 months ago

Same here; I feel like it is non-sense to continue monitoring timeout during step-by-step debugging and risking to commit stuff like --testTimeout=100000000 during development ~or even configuring the extension Jest command line accordingly and forget to revert it after (less problematic if you have proper CI though)~.

EDIT: setting --testTimeout=100000000 in the extension Jest command line doesn't work since it is removed from the issued command.

EDIT2: actually the extension is using the config from launch.json file for debugging, so you can setup --testTimeout=100000000 inside this file and it will use this param as expected. This is actually a decent workaround for step-by-step debugging.

I think that deactivating the timeout or overriding it to something that makes sense during step-by-step debugging could improve the usability of this feature. Moreover, it took me some time to understand why my afterAll hooks were triggered during the execution of my tests, making them erroneously fail with random unexpected behaviours.

On the other hand, I don't know if there could be some drawbacks to such modification, like if you want just to start debugging in order to find where legitimate timeouts are issued (still a niche case IMHO, and not even sure you can detect the root cause like that nor if it is the easiest way of doing it, most of the time you know what is causing timeout). So, I guess it could be at least an extension opt-in/opt-out setting.

connectdotz commented 9 months ago

EDIT2: actually the extension is using the config from launch.json file for debugging, so you can setup --testTimeout=100000000 inside this file and it will use this param as expected. This is actually a decent workaround for step-by-step debugging.

@Jule- Your suggestion is excellent. it's an appropriate solution for those encountering this issue. It makes a lot of sense to use testTimeout option for this scenario, and configuring it within the launch.json allows for customization tailored to individual user environments.

Creating an additional extension setting seems unnecessary, given the flexibility already offered by the testTimeout option in Jest. We just need to ensure that users are aware of this feature. This issue discussion here will serve as a valuable resource for that purpose.

An example debug config with the testTimeout might look like this:

{
  "type": "node",
  "name": "vscode-jest-tests.v2",
  "request": "launch",
  "program": "${workspaceFolder}/node_modules/.bin/jest",
  "args": [
    "--runInBand",
    "--watchAll=false",
    "--testTimeout=100000000",
    "--testNamePattern",
    "${jest.testNamePattern}",
    "--runTestsByPath",
    "${jest.testFile}"
  ],
  "cwd": "${workspaceFolder}",
  "console": "integratedTerminal",
  "internalConsoleOptions": "neverOpen",
  "disableOptimisticBPs": true,
  "windows": {
    "program": "${workspaceFolder}/node_modules/jest/bin/jest"
  }
}

For more information on customizing the Jest debug configuration, please refer to the debug config documentation.

jest-community / vscode-jest