Struggling to run eslint benchmarks on macOS

[StevenLove]

My current issue is likely just with vscode-eslint/ and not quicklint, but I wanted to share my experience and see if others are trying to run benchmarks on mac or not.

...

I followed the instructions from the benchmark/benchmark-lsp/README.md to prepare eslint for benchmarking, which boil down to

$ cd benchmark/benchmark-lsp/eslint
$ yarn
$ cd node_modules/vscode-eslint/
$ npm ci && npm run compile:server

but I get 62 errors trying to compile with tsc. Here's the first couple of errors

server/src/eslintServer.ts:1346:3 - error TS2322: Type 'number' is not assignable to type 'Immediate | undefined'.

1346   this.timer = setImmediate(() => {
       ~~~~~~~~~~

../../../../../../../../node_modules/@types/jest/index.d.ts:2:13 - error TS2403: Subsequent variable declarations must have the same type.  Variable 'beforeEach' must be of type 'HookFunction', but here has type 'Lifecycle'.

2 declare var beforeEach: jest.Lifecycle;
              ~~~~~~~~~~

  node_modules/@types/mocha/index.d.ts:2624:13
    2624 declare var beforeEach: Mocha.HookFunction;
                     ~~~~~~~~~~
    'beforeEach' was also declared here.

../../../../../../../../node_modules/@types/jest/index.d.ts:4:13 - error TS2403: Subsequent variable declarations must have the same type.  Variable 'afterEach' must be of type 'HookFunction', but here has type 'Lifecycle'.

4 declare var afterEach: jest.Lifecycle;
              ~~~~~~~~~

  node_modules/@types/mocha/index.d.ts:2642:13
    2642 declare var afterEach: Mocha.HookFunction;
                     ~~~~~~~~~
    'afterEach' was also declared here.

../../../../../../../../node_modules/@types/jest/index.d.ts:5:13 - error TS2403: Subsequent variable declarations must have the same type.  Variable 'describe' must be of type 'SuiteFunction', but here has type 'Describe'.

5 declare var describe: jest.Describe;
              ~~~~~~~~

  node_modules/@types/mocha/index.d.ts:2658:13
    2658 declare var describe: Mocha.SuiteFunction;
                     ~~~~~~~~
    'describe' was also declared here.

I tried with the local tsc version 4.5.2 located at /benchmark/benchmark-lsp/eslint/node_modules/vscode-eslint/node_modules/typescript/bin/tsc I tried with a global install of tsc version 5.3.3 and got the same 62 errors

Any ideas? Ultimately I would like to add another linter to be benchmarked

[strager]

I don't get the errors you're seeing on my Mac.

What do npm version and yarn --version report? Here's mine (inside .../benchmark/benchmark-lsp/eslint/node_modules/vscode-eslint):

$ npm version                         
{
  'vscode-eslint': '2.2.3',
  npm: '8.13.2',
  node: '18.6.0',
  v8: '10.2.154.13-node.8',
  uv: '1.43.0',
  zlib: '1.2.11',
  brotli: '1.0.9',
  ares: '1.18.1',
  modules: '108',
  nghttp2: '1.47.0',
  napi: '8',
  llhttp: '6.0.7',
  openssl: '3.0.5+quic',
  cldr: '41.0',
  icu: '71.1',
  tz: '2022a',
  unicode: '14.0',
  ngtcp2: '0.1.0-DEV',
  nghttp3: '0.1.0-DEV'
}
$ yarn --version
1.22.19

I also didn't get errors with Node.js version 20:

$ npm version
{
  'vscode-eslint': '2.2.3',
  npm: '10.2.3',
  node: '20.10.0',
  acorn: '8.10.0',
  ada: '2.7.2',
  ares: '1.20.1',
  base64: '0.5.0',
  brotli: '1.0.9',
  cjs_module_lexer: '1.2.2',
  cldr: '43.1',
  icu: '73.2',
  llhttp: '8.1.1',
  modules: '115',
  napi: '9',
  nghttp2: '1.57.0',
  openssl: '3.0.12',
  simdutf: '3.2.18',
  tz: '2023c',
  undici: '5.26.4',
  unicode: '15.0',
  uv: '1.47.0',
  uvwasi: '0.0.19',
  v8: '11.3.244.8-node.25',
  zlib: '1.3'
}

[StevenLove]

I checked the info and reverified I'm still getting those 62 errors.

$ npm version
{
  'vscode-eslint': '2.2.3',
  npm: '8.11.0',
  node: '16.15.1',
  v8: '9.4.146.24-node.21',
  uv: '1.43.0',
  zlib: '1.2.11',
  brotli: '1.0.9',
  ares: '1.18.1',
  modules: '93',
  nghttp2: '1.47.0',
  napi: '8',
  llhttp: '6.0.4',
  openssl: '1.1.1o+quic',
  cldr: '40.0',
  icu: '70.1',
  tz: '2021a3',
  unicode: '14.0',
  ngtcp2: '0.1.0-DEV',
  nghttp3: '0.1.0-DEV'
}
$ yarn --version
1.22.21

And since you mentioned the node version which wasn't really on my radar I updated to 20.0.0 and tried again, still getting 62 errors but perhaps due to another package as several are older than yours. Here are my versions after updating node.

$ npm version
{
  'vscode-eslint': '2.2.3',
  npm: '9.6.4',
  node: '20.0.0',
  acorn: '8.8.2',
  ada: '2.0.0',
  ares: '1.19.0',
  brotli: '1.0.9',
  cldr: '42.0',
  icu: '72.1',
  llhttp: '8.1.0',
  modules: '115',
  napi: '8',
  nghttp2: '1.52.0',
  nghttp3: '0.7.0',
  ngtcp2: '0.8.1',
  openssl: '3.0.8+quic',
  simdutf: '3.2.7',
  tz: '2023c',
  undici: '5.21.0',
  unicode: '15.0',
  uv: '1.44.2',
  uvwasi: '0.0.16',
  v8: '11.3.244.4-node.3',
  zlib: '1.2.13'
}
$ yarn --version
1.22.21

[strager]

When you say "but I get 62 errors trying to compile with tsc.", are you talking about the npm run compile:server command or some other command?

The error message you posted refers to a patch which I don't have on my machine (../../../../../../../../node_modules/@types/jest/index.d.ts). Do you know where this came from? ../../../../../../../../ isn't even inside the quick-lint-js repo if cwd is <repo>/benchmark/benchmark-lsp/eslint/node_modules/vscode-eslint.

[StevenLove]

Good insight noticing those suspicious paths. I mistakenly had a package.json up a few directories from where I forked quick-lint. Once I removed it, npm run compile:server worked just fine!