🚀 Feature: Add option to find the call site of a test (file name and location)

[nicojs]

Is your feature request related to a problem or a nice-to-have?? Please describe. I would love a way to discover where tests live inside the files. Could we attach this metadata to a Test, or Runnable?

runner.on(EVENT_TEST_BEGIN, function(test) {
  const { fileName, column, line } = test.getCallSite(); 
  //      ^ string  ^ numbers ^
});

Describe the solution you'd like The only generic way I can think of how to discover the location of the caller of our it or test functions is by using new Error().stack. This might come with a performance penalty, so we might want to add it behind a new configuration option. Similar to Jest's --testlocationinresults.

Describe alternatives you've considered

This workaround works:

const original = Mocha.Suite.prototype.addTest;
Suite.prototype.addTest = function addTest(test) {
  const suite = original.call(this, test);
  test.callSite = getCallSite(new Error()); // magic of parsing the stack
  return suite;
};

There are numerous libraries available that can parse the stack for us. Jest and tabjs are using stack-utils.

Additional context

You might have guessed it, but we want to use this new callSite to improve Stryker's reporting and performance 😅

[nicojs]

Let me do a proposal.

I want to create and maintain a good package that provides you with the most accurate location of the call site. I've already started it here: https://github.com/nicojs/call-id. It will support all actively used browsers (as long as sauce labs or BrowserStack allows me to test them). It can be used as an ESM or CJS module, so it should be easily bundled in browser packages.

I want to implement this feature in both here in mocha and jasmine (https://github.com/jasmine/jasmine/issues/1884). What do you think?

[JoshuaKGoldberg]

@nicojs is this still relevant given https://github.com/stryker-mutator/stryker-js/pull/3504?

[nicojs]

Hi Josh 🙋‍♂️. It's very cool to see you here! 😄

@nicojs is this still relevant given stryker-mutator/stryker-js#3504?

Yes, it's still very relevant. Since we only know in which file the test was declared, not its location.

1. This results in a plain list of tests right above the source code. Not very elegant IMO. See https://dashboard.stryker-mutator.io/reports/github.com/stryker-mutator/stryker-js/master?module=core#test/concurrent/resource-decorator.spec.js
2. We use the test location to implement incremental mode. This is a feature where Stryker tries to detect which tests have changed since a previous run. Since we currently don't know the exact location of a mocha test, we assume every test changed when a test file changed, which is bad for performance.

About the implementation. There are a couple of open questions:

1. Since the only way to get the location of the call site is to throw an error and catch it, it would add a linear performance penalty. Would this be acceptable?
   • We could make it configurable, like jest's --testlocationinresults.
2. Users that run Mocha tests often transpile to JS before running the tests (for example, using esbuild or tsc). Is resolving source files part of this feature?
   • I think not. Resolving source maps is a lot of work. There are also good solutions out there, like --enable-source-maps or source-map-support.
3. Would you be willing to add call-id as a dependency of mocha?
4. You've marked this issue with area: nodejs. This feature could also work in the browser. That was my goal originally with call-id.
   • Which brings me to: what browsers does Mocha need to support?