Unable to resolve vite aliases while running wallabyjs with pnpm workspace and vitest

[bhvngt]

Issue description or question

I am trying to setup wallabyjs with a new project that is using pnpm workspace and vitest.

I am trying to self-reference my project using vite aliases. While running vitest from cli, it picks up aliases. However, wallabyjs is not able to resolve them.

I had to either add my project module "@libs/foo": "workspace:*" as dependency within "./libs/foo/package.json" or add it at root level package.json

Here's the repo where you can reproduce this issue.

Wallaby diagnostics report

{
  editorVersion: 'IntelliJ IDEA 2022.2.1 RC',
  pluginVersion: '1.0.250',
  editorType: 'IntelliJ',
  osVersion: 'darwin 21.6.0',
  nodeVersion: 'v18.7.0',
  coreVersion: '1.0.1317',
  checksum: 'MmUxNzQ4YTIxNGUwZjZhODg5YWRhNzc5Y2E0OWY3MzgsMTY2OTU5MzYwMDAwMCww',
  config: {
    diagnostics: {
      vitest: {
        config: {
          allowOnly: true,
          watch: true,
          globals: false,
          environment: 'node',
          threads: true,
          clearMocks: false,
          restoreMocks: false,
          mockReset: false,
          include: [ '**/*.{test,spec}.{js,mjs,cjs,ts,mts,cts,jsx,tsx}' ],
          exclude: [ '**/node_modules/**', '**/dist/**', '**/cypress/**', '**/.{idea,git,cache,output,temp}/**' ],
          testTimeout: 5000,
          hookTimeout: 10000,
          teardownTimeout: 1000,
          isolate: true,
          watchExclude: [ '**/node_modules/**', '**/dist/**' ],
          forceRerunTriggers: [ '**/package.json/**', '**/vitest.config.*/**', '**/vite.config.*/**' ],
          update: false,
          reporters: [ 'default' ],
          silent: false,
          ui: false,
          uiBase: '/__vitest__/',
          open: true,
          css: { include: [ {} ] },
          coverage: {
            enabled: false,
            clean: true,
            cleanOnRerun: false,
            reportsDirectory: '<homeDir>/Projects/innerview/temp/pnpm-demo/coverage',
            excludeNodeModules: true,
            exclude: [
              'coverage/**',
              'dist/**',
              'packages/*/test{,s}/**',
              '**/*.d.ts',
              'cypress/**',
              'test{,s}/**',
              'test{,-*}.{js,cjs,mjs,ts,tsx,jsx}',
              '**/*{.,-}test.{js,cjs,mjs,ts,tsx,jsx}',
              '**/*{.,-}spec.{js,cjs,mjs,ts,tsx,jsx}',
              '**/__tests__/**',
              '**/{karma,rollup,webpack,vite,vitest,jest,ava,babel,nyc,cypress}.config.{js,cjs,mjs,ts}',
              '**/.{eslint,mocha,prettier}rc.{js,cjs,yml}'
            ],
            reporter: [ 'text', 'html', 'clover' ],
            allowExternal: false,
            extension: [
              '.js',  '.cjs',
              '.mjs', '.ts',
              '.tsx', '.jsx',
              '.vue', '.svelte'
            ],
            tempDirectory: '<homeDir>/Projects/innerview/temp/pnpm-demo/coverage/tmp'
          },
          fakeTimers: { loopLimit: 10000, shouldClearNativeTimers: true, toFake: [ 'setTimeout', 'clearTimeout', 'setInterval', 'clearInterval', 'setImmediate', 'clearImmediate', 'Date' ] },
          maxConcurrency: 5,
          dangerouslyIgnoreUnhandledErrors: false,
          defines: {},
          root: '<homeDir>/Projects/innerview/temp/pnpm-demo',
          deps: { inline: [ {}, {}, {}, {}, {}, {}, '@nuxt/test-utils' ], registerNodeLoader: false },
          snapshotOptions: { snapshotFormat: {}, updateSnapshot: 'new' },
          setupFiles: [],
          cache: { dir: '<homeDir>/Projects/innerview/temp/pnpm-demo/node_modules/.vitest' },
          sequence: {},
          package: {
            version: '0.21.1',
            urls: { hooks: 'file://<homeDir>/Library/Caches/JetBrains/IntelliJIdea2022.2/wallaby/wallaby/runners/node/hooks.mjs' },
            paths: {
              root: '<homeDir>/Projects/innerview/temp/pnpm-demo/node_modules/.pnpm/vitest@0.21.1/node_modules/vitest',
              dist: '<homeDir>/Projects/innerview/temp/pnpm-demo/node_modules/.pnpm/vitest@0.21.1/node_modules/vitest/dist'
            }
          }
        }
      }
    },
    testFramework: { version: 'vitest@0.14.0', configurator: 'vitest@0.14.0', reporter: 'vitest@0.14.0', starter: 'vitest@0.14.0', autoDetected: true },
    preserveComments: false,
    extractComments: true,
    files: [
      { pattern: '**/*.{test,spec}.{js,mjs,cjs,ts,mts,cts,jsx,tsx}', ignore: true, trigger: true, load: true, file: true },
      { pattern: '**/node_modules/**', ignore: true, trigger: true, load: true, file: true, test: true },
      { pattern: '**/dist/**', ignore: true, trigger: true, load: true, file: true, test: true },
      { pattern: '**/cypress/**', ignore: true, trigger: true, load: true, file: true, test: true },
      { pattern: '**/.{idea,git,cache,output,temp}/**', ignore: true, trigger: true, load: true, file: true, test: true },
      { pattern: '**/*.*', ignore: false, trigger: true, load: true, order: 1 }
    ],
    tests: [
      { pattern: '**/node_modules/**', ignore: true, trigger: true, load: true, test: true, file: false },
      { pattern: '**/dist/**', ignore: true, trigger: true, load: true, test: true, file: false },
      { pattern: '**/cypress/**', ignore: true, trigger: true, load: true, test: true, file: false },
      { pattern: '**/.{idea,git,cache,output,temp}/**', ignore: true, trigger: true, load: true, test: true, file: false },
      { pattern: '**/*.{test,spec}.{js,mjs,cjs,ts,mts,cts,jsx,tsx}', ignore: false, trigger: true, load: true, test: true, order: 2 }
    ],
    workers: { initial: 1, regular: 1, recycle: false },
    filesWithNoCoverageCalculated: [],
    runAllTestsInAffectedTestFile: false,
    updateNoMoreThanOneSnapshotPerTestFileRun: false,
    compilers: {},
    logLimits: { inline: { depth: 5, elements: 5000 }, values: { default: { stringLength: 200 }, autoExpand: { elements: 5000, stringLength: 8192, depth: 10 } } },
    preprocessors: {},
    maxConsoleMessagesPerTest: 100,
    autoConsoleLog: true,
    delays: { run: 0, edit: 100, update: 0 },
    teardown: undefined,
    hints: {
      ignoreCoverage: '__REGEXP /ignore coverage|istanbul ignore/',
      ignoreCoverageForFile: '__REGEXP /ignore file coverage/',
      commentAutoLog: '?',
      testFileSelection: { include: '__REGEXP /file\\.only/', exclude: '__REGEXP /file\\.skip/' }
    },
    automaticTestFileSelection: true,
    runSelectedTestsOnly: false,
    mapConsoleMessagesStackTrace: false,
    extensions: {},
    env: {
      type: 'node',
      params: { runner: '--experimental-loader=file://<homeDir>/Library/Caches/JetBrains/IntelliJIdea2022.2/wallaby/wallaby/runners/node/hooks.mjs' },
      runner: '<homeDir>/Library/Application Support/fnm/node-versions/v18.7.0/installation/bin/node',
      viewportSize: { width: 800, height: 600 },
      options: { width: 800, height: 600 },
      bundle: true
    },
    reportUnhandledPromises: true,
    slowTestThreshold: 75,
    lowCoverageThreshold: 80,
    loose: undefined,
    symlinkNodeModules: undefined,
    configCode: 'auto.detect#1187933729'
  },
  packageJSON: { dependencies: undefined, devDependencies: { vitest: '^0.21.1' } },
  fs: { numberOfFiles: 29 },
  debug: [
    '2022-08-14T15:59:22.981Z config Attempting automatic configuration for angular\n',
    '2022-08-14T15:59:22.982Z angular/cli config Angular CLI not found.\n',
    '2022-08-14T15:59:22.982Z config Finished attempting automatic configuration for angular (1ms)\n',
    '2022-08-14T15:59:22.982Z config Attempting automatic configuration for jest\n',
    "2022-08-14T15:59:22.983Z jest/config Error: Module jest-cli is not found in '<homeDir>/Projects/innerview/temp/pnpm-demo'.\n" +
      '    at Object.<anonymous> (<homeDir>/Library/Caches/JetBrains/IntelliJIdea2022.2/wallaby/wallaby/server.js:31:15814)\n' +
      '    at r (<homeDir>/Library/Caches/JetBrains/IntelliJIdea2022.2/wallaby/wallaby/server.js:30:31066)\n' +
      '    at Object.next (<homeDir>/Library/Caches/JetBrains/IntelliJIdea2022.2/wallaby/wallaby/server.js:30:30361)\n' +
      '    at <homeDir>/Library/Caches/JetBrains/IntelliJIdea2022.2/wallaby/wallaby/server.js:30:30270\n' +
      '    at new Promise (<anonymous>)\n' +
      '    at n (<homeDir>/Library/Caches/JetBrains/IntelliJIdea2022.2/wallaby/wallaby/server.js:30:30064)\n' +
      '    at Object.loadJest (<homeDir>/Library/Caches/JetBrains/IntelliJIdea2022.2/wallaby/wallaby/server.js:31:13412)\n' +
      '    at <homeDir>/Library/Caches/JetBrains/IntelliJIdea2022.2/wallaby/wallaby/server.js:19:17086\n' +
      '    at r (<homeDir>/Library/Caches/JetBrains/IntelliJIdea2022.2/wallaby/wallaby/server.js:19:21819)\n' +
      '    at Object.next (<homeDir>/Library/Caches/JetBrains/IntelliJIdea2022.2/wallaby/wallaby/server.js:19:21114)\n' +
      '    at <homeDir>/Library/Caches/JetBrains/IntelliJIdea2022.2/wallaby/wallaby/server.js:19:21023\n' +
      '    at new Promise (<anonymous>)\n' +
      '    at s (<homeDir>/Library/Caches/JetBrains/IntelliJIdea2022.2/wallaby/wallaby/server.js:19:20817)\n' +
      '    at r (<homeDir>/Library/Caches/JetBrains/IntelliJIdea2022.2/wallaby/wallaby/server.js:19:16774)\n' +
      '    at Object.<anonymous> (<homeDir>/Library/Caches/JetBrains/IntelliJIdea2022.2/wallaby/wallaby/server.js:19:20494)\n' +
      '    at r (<homeDir>/Library/Caches/JetBrains/IntelliJIdea2022.2/wallaby/wallaby/server.js:19:21819)\n' +
      '    at Object.next (<homeDir>/Library/Caches/JetBrains/IntelliJIdea2022.2/wallaby/wallaby/server.js:19:21114)\n' +
      '    at <homeDir>/Library/Caches/JetBrains/IntelliJIdea2022.2/wallaby/wallaby/server.js:19:21023\n' +
      '    at new Promise (<anonymous>)\n' +
      '    at s (<homeDir>/Library/Caches/JetBrains/IntelliJIdea2022.2/wallaby/wallaby/server.js:19:20817)\n' +
      '    at Object.n [as configure] (<homeDir>/Library/Caches/JetBrains/IntelliJIdea2022.2/wallaby/wallaby/server.js:19:20375)\n' +
      '    at Config.<anonymous> (<homeDir>/Library/Caches/JetBrains/IntelliJIdea2022.2/wallaby/wallaby/server.js:15:20330)\n' +
      '    at r (<homeDir>/Library/Caches/JetBrains/IntelliJIdea2022.2/wallaby/wallaby/server.js:15:12061)\n' +
      '    at Object.next (<homeDir>/Library/Caches/JetBrains/IntelliJIdea2022.2/wallaby/wallaby/server.js:15:11356)\n' +
      '    at o (<homeDir>/Library/Caches/JetBrains/IntelliJIdea2022.2/wallaby/wallaby/server.js:15:11105)\n',
    '2022-08-14T15:59:22.983Z config Finished attempting automatic configuration for jest (1ms)\n',
    '2022-08-14T15:59:22.983Z config Attempting automatic configuration for vitest\n',
    '2022-08-14T15:59:22.985Z vitest/config Detected Vitest.\n',
    '2022-08-14T15:59:23.258Z config Finished attempting automatic configuration for vitest (275ms)\n',
    '2022-08-14T15:59:23.259Z project Wallaby Node version: v18.7.0\n',
    '2022-08-14T15:59:23.259Z project Wallaby config: <homeDir>/Projects/innerview/temp/pnpm-demo/auto.detect\n',
    '2022-08-14T15:59:23.265Z fs File system starting\n',
    '2022-08-14T15:59:23.288Z fs File system scan completed\n',
    '2022-08-14T15:59:23.291Z project File cache: <homeDir>/Library/Caches/JetBrains/IntelliJIdea2022.2/wallaby/projects/618d18562d342ae2\n',
    '2022-08-14T15:59:23.320Z uiService Listening port 51235\n',
    '2022-08-14T15:59:23.326Z workers Parallelism for initial run: 1, for regular run: 1\n',
    '2022-08-14T15:59:23.326Z workers Starting run worker instance #0\n',
    '2022-08-14T15:59:23.326Z workers Web server is listening at 65419\n',
    '2022-08-14T15:59:23.330Z project Stopping process pool\n',
    '2022-08-14T15:59:23.330Z project File cache is up-to-date, starting full test run\n',
    '2022-08-14T15:59:23.333Z project Test run started; run priority: 3\n',
    '2022-08-14T15:59:23.333Z project Running all tests\n',
    '2022-08-14T15:59:23.334Z workers Starting test run, priority: 3\n',
    '2022-08-14T15:59:23.334Z nodeRunner Starting sandbox [worker #0, session #a6hol]\n',
    '2022-08-14T15:59:23.335Z nodeRunner Preparing sandbox [worker #0, session #a6hol]\n',
    '2022-08-14T15:59:23.426Z workers Started run worker instance (delayed) #0\n',
    '2022-08-14T15:59:23.426Z nodeRunner Prepared sandbox [worker #0, session #a6hol]\n',
    '2022-08-14T15:59:23.426Z workers [worker #0, session #a6hol] Running tests in sandbox\n',
    '2022-08-14T15:59:23.558Z workers Scheduling Vitest Run (a6hol): 2022-08-14T15:59:23.547Z\n',
    '2022-08-14T15:59:23.985Z workers Sandbox (active) [a6hol] error: [vite-node] Failed to load @libs/foo/demo\n' +
      '    at VitestRunner.directRequest (file://./node_modules/.pnpm/vitest@0.21.1/node_modules/vitest/dist/chunk-vite-node-client.fdd9592c.mjs:140:13)\n' +
      '    at async VitestRunner.cachedRequest (file://./node_modules/.pnpm/vitest@0.21.1/node_modules/vitest/dist/chunk-vite-node-client.fdd9592c.mjs:88:12)\n' +
      '    at async _VitestMocker.request (file://./node_modules/.pnpm/vitest@0.21.1/node_modules/vitest/dist/chunk-vite-node-client.fdd9592c.mjs:110:21)\n' +
      '    at async ./libs/foo/test/demo.test.ts:4:31\n' +
      '    at async VitestRunner.directRequest (file://./node_modules/.pnpm/vitest@0.21.1/node_modules/vitest/dist/chunk-vite-node-client.fdd9592c.mjs:192:5)\n' +
      '    at async VitestRunner.cachedRequest (file://./node_modules/.pnpm/vitest@0.21.1/node_modules/vitest/dist/chunk-vite-node-client.fdd9592c.mjs:88:12)\n' +
      '    at async _VitestMocker.request (file://./node_modules/.pnpm/vitest@0.21.1/node_modules/vitest/dist/chunk-vite-node-client.fdd9592c.mjs:110:21)\n' +
      '    at async collectTests (./node_modules/.pnpm/vitest@0.21.1/node_modules/vitest/dist/chunk-runtime-error.87a2b5a2.mjs:314:9)\n' +
      '    at async startTestsNode (./node_modules/.pnpm/vitest@0.21.1/node_modules/vitest/dist/chunk-runtime-error.87a2b5a2.mjs:634:17)\n' +
      '    at async ./node_modules/.pnpm/vitest@0.21.1/node_modules/vitest/dist/entry.mjs:76:9\n' +
      '    at async Module.withEnv (./node_modules/.pnpm/vitest@0.21.1/node_modules/vitest/dist/chunk-runtime-error.87a2b5a2.mjs:259:5)\n' +
      '    at async run (./node_modules/.pnpm/vitest@0.21.1/node_modules/vitest/dist/entry.mjs:71:5)\n' +
      '    at async file://./node_modules/.pnpm/tinypool@0.2.4/node_modules/tinypool/dist/esm/worker.js:99:20\n',
    '2022-08-14T15:59:23.987Z workers [a6hol.1] Loaded unknown number of test(s)\n',
    '2022-08-14T15:59:24.033Z workers [a6hol.2] Loaded unknown number of test(s)\n',
    '2022-08-14T15:59:24.033Z workers [a6hol.2] Test executed: sum\n',
    '2022-08-14T15:59:24.036Z workers Vitest Run Complete (a6hol): 2022-08-14T15:59:24.035Z\n',
    '2022-08-14T15:59:24.037Z workers [a6hol] Run 1 test(s), skipped 0 test(s)\n',
    '2022-08-14T15:59:24.037Z workers [a6hol] Sandbox is responsive, closing it\n',
    '2022-08-14T15:59:24.039Z project Test run finished\n',
    '2022-08-14T15:59:24.039Z project Processed console.log entries\n',
    '2022-08-14T15:59:24.039Z project Processed loading sequences\n',
    '2022-08-14T15:59:24.039Z project Processed executed tests\n',
    '2022-08-14T15:59:24.039Z project Processed code coverage\n',
    '2022-08-14T15:59:24.044Z project Test run result processed and sent to IDE\n',
    '2022-08-14T15:59:24.147Z fs No metadata for added file found: node_modules/.vitest/results.json\n'
  ]
}

[smcenlly]

I believe that this issue is the same as https://github.com/wallabyjs/public/issues/3049.

It appears that you are attempt to run vitest from your project root with Wallaby and there is no vite configuration within your project root so the default configuration is being used.

Right now, vitest does not support running as a mono-repo and so Wallaby also does not support running vitest from a mono-repo project root.

After fixing your dependencies in lib/foo, we were able to start Wallaby on this directory and the aliases worked for us.

[bhvngt]

@smcenlly Thanks for your reply.

I believe that this issue is the same as https://github.com/wallabyjs/public/issues/3049.

I agree.

It appears that you are attempt to run vitest from your project root with Wallaby and there is no vite configuration within your project root so the default configuration is being used.

It was the other way round. In command-line mode, I was running vitest from my module directory libs/foo which worked fine. When I ran it from project root on the command line, my tests failed with the same error. I believe, wallaby too runs from project root.

Right now, vitest does not support running as a mono-repo and so Wallaby also does not support running vitest from a mono-repo project root.

As per the vitest thread, currently, they recommend pnpm's recursion (-r) flag for monorepo. Is something similar can be used with wallaby? Alternatively, I currently organise my projects as intellij modules. Is it possible to run wallabyjs instance per module?

After fixing your dependencies in lib/foo, we were able to start Wallaby on this directory and the aliases worked for us.

I could make it work at the root level by adding my monorepo workspace projects at root level package.json and configuring vitest.config.js at the root level. However, it felt little hackish. So not sure if I would like to take a similar approach in a real project.

[smcenlly]

With Wallaby's Automatic Configuration, you can specify the root path to run your tests from in your Run Configuration. This would allow you to run the tests in lib/foo the same as when you use the CLI.

When you use pnpm's recursive flag, it runs multiple commands at the same time (e.g. npm run test for each project). Unfortunately Wallaby needs to run all of your project tests as a single unit of work so that it can correctly provide code coverage, etc. This is why we need vitest to support mono-repos natively to run in a mono-repo root.

If you make the root level work by adding and configuring vitest in your mono-repo root, then Wallaby should definitely work for you. If you choose not to take this approach then you will be limited to being able to run a single package at a time.

[bhvngt]

Thanks @smcenlly. That was very helpful and does make sense. I will tinker with my root folder configuration to see if I can make that workable.

[kaminskypavel]

@smcenlly quick question, if I migrate to jest, can wallaby run simultaneously on a monorepo managed with pnpm?

[ArtemGovorov]

@kaminskypavel Jest has the projects setting, that allows to run all monorepo tests regardless of the used package manager.