hbenl / vscode-mocha-test-adapter

Mocha Test Adapter for the VS Code Test Explorer
MIT License
91 stars 31 forks source link

test location mismatch on partial name match with monkeyPatch:false #195

Open Spown opened 2 years ago

Spown commented 2 years ago

If a test that is being explicitly run/debugged and it's name being a substring of another test, that occurs earlier in the set, this earlier test is marked as currently being run:

Code_2022-02-23_02-13-08

I didn't try monkeyPatch: true since in my setup it won't work anyway.

Thank you.

hbenl commented 2 years ago

Yeah, without the monkey patch the extension just searches for text matches, without trying to understand the source code. Perhaps you could try to change your setup slightly to make the monkey patch work. (What the monkey patch does is create a stacktrace for each it() and describe() call and look for the first caller in the stacktrace that is in the test file that is currently being loaded). Essentially what you do is this:

const tests = {
  "should do stuff": {...},
  "should do other stuff": {...},
};
for (let test in tests) {
  const opts = tests[test];
  it(test, () => { ... });
}

If you write it like this:

myTest("should do stuff", {...});
myTest("should do other stuff", {...});

function myTest(test, opts) {
  it(test, () => { ... });
}

and move myTest() to a different file(!), then monkeyPatch: true should give you the right locations.

Spown commented 2 years ago

Perhaps you could try to change your setup slightly to make the monkey patch work.

that's what I did already. Just ensuring that it() descriptions do not intersect is easy. The problem is you can't really always keep it in mind. Or at all if you are new to the problem. If monkeyPatch is the one who is doing the matching, then I guess there is not much you can do here and the issue can be closed/upstreamed.

The fact that the dividing things in files helps doesn't really helps much, since tests that test for adjustment problems would naturally be grouped together and named similarly. So it only would help in a 1test : 1file setup.

Maybe the extension could do a simple analysis check on the descriptions after the parsing (or better yet before the explicit running/debugging) and throw a warning to the user about eventual discrepancies?