Should the `before`/`after` hooks run during Task warmup?

[scotttrinh]

I am benchmarking TypeScript's language service binary and I'm currently spawning the tsserver process in a beforeAll hook and then killing the process in an afterAll hook. However, I noticed that running the warmup does not run the task's beforeAll and afterAll hooks, but it does run the bench's setup and teardown hooks. Since I'm shooting out different versions of the TypeScript compiler against each other, I cannot move my setup and teardown into the bench's setup/teardown lifecycle since each Task needs to setup a different server.

Here is a portion of my benchmarking code:

for (const version of ["ts48", "ts49", "ts50", "ts51"]) {
  bench = bench.add(
    version,
    async () => {
      await getCompletion(activeServer);
    },
    {
      beforeAll: () => {
        activeServer = createServer(
          path.join(
            __dirname,
            "..",
            "..",
            "..",
            "node_modules",
            version,
            "bin",
            "tsserver"
          )
        );
        activeServer.send({
          command: "open",
          arguments: {
            file: mockFileName,
            fileContent: mockFileContent,
          },
        });
      },
      afterAll: async () => {
        await activeServer.close();
      },
    }
  );
}

[Aslemammad]

Reasonable, I believe it should run! Do you want to work on it?