Broken source maps for bundles with packages from node_modules

dmaretskyi commented 3 days ago

Describe the bug

I have a workspace with two packages:

pkg-a build a bundle of its own code and packages from node_modules
pkg-b runs test from vitest that import code from pkg-a

The sourcemaps for errors thrown from inside pkg-a report incorrect source locations:

$ pnpm vitest

Error: Test error
 ❯ Module.foo ../node_modules/.pnpm/base32-decode@1.0.0/node_modules/base32-decode/index.js:5:9
 ❯ index.test.js:5:3

Running similar code in node JS produces the correct stack trace:

$ node --enable-source-maps script.js
/Users/dmaretskyi/vitest-sourcemap-bug/pkg-a/index.js:5
  throw new Error("Test error");
        ^

Error: Test error
    at foo (/Users/dmaretskyi/vitest-sourcemap-bug/pkg-a/index.js:5:9)
    at file:///Users/dmaretskyi/vitest-sourcemap-bug/pkg-b/script.js:3:1
    at ModuleJob.run (node:internal/modules/esm/module_job:217:25)
    at async ModuleLoader.import (node:internal/modules/esm/loader:316:24)
    at async loadESM (node:internal/process/esm_loader:34:7)
    at async handleMainPromise (node:internal/modules/run_main:66:12)

Reproduction

https://github.com/dmaretskyi/vitest-sourcemap-bug

See README for repro steps

System Info

System:
    OS: macOS 14.0
    CPU: (10) arm64 Apple M1 Pro
    Memory: 78.80 MB / 16.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 20.9.0 - ~/.nodenv/versions/20.9.0/bin/node
    Yarn: 1.22.21 - ~/.nodenv/versions/20.9.0/bin/yarn
    npm: 10.2.4 - ~/.nodenv/versions/20.9.0/bin/npm
    pnpm: 8.11.0 - ~/.nodenv/versions/20.9.0/bin/pnpm
    bun: 1.0.25 - ~/.bun/bin/bun
  Browsers:
    Chrome: 128.0.6613.138
    Safari: 17.0
  npmPackages:
    vitest: ^2.1.1 => 2.1.1

Used Package Manager

pnpm

Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.

dmaretskyi commented 3 days ago

For the bug to trigger it seems to be essential to have a package from node_modules in the bundle. Just bundling two local source files together does not repro the bug

dmaretskyi commented 2 days ago

I've started debugging this...

It seems like the source file paths get corrupted when the source map is loaded

dmaretskyi commented 2 days ago

After doing a dump with VITE_NODE_DEBUG_DUMP=1 it seems like those modules have broken source maps

dmaretskyi commented 2 days ago

I've traced it down to vite's source map combining code. The actual source positions are correct, but something corrupts the source paths

hi-ogawa commented 2 days ago

This looks similar to https://github.com/vitest-dev/vitest/issues/5523. Workaround is to prevent Vitest/Vite from processing already bundled pkg-a and you can do that by configuring test.server.deps.external. I don't think Vite currently has a way to read existing dist.js.map and use it for remapping during own transform.

// pkg-b/vitest.config.ts
import { defineConfig } from "vitest/config"

export default defineConfig({
  test: {
    server: {
      deps: {
        external: [/pkg-a/],
      }
    }
  }
})

With this config, I see this error on Vitest:

 FAIL  index.test.js > repro
Error: Test error
 ❯ Module.foo ../pkg-a/index.js:5:9
 ❯ index.test.js:5:3

AriPerkkio commented 2 days ago

Looks like bug in Vite. map.sources are incorrect when file is SSR transformed.

Error: Test error
 ❯ Module.foo ../node_modules/.pnpm/base32-decode@1.0.0/node_modules/base32-decode/index.js:5:9
                                                                                   ^^^ This is pkg-a/index.js, not contents of base32-decode

I renamed pkg-a/index.js to pkg-a/pkg-a-entrypoint.js here to make it unique from index.js.

When we request the pkg-a here, we get following. Notice the "...base32-decode/pkg-a-entrypoint.js" part:

https://github.com/vitest-dev/vitest/blob/2a50464d58e98f58fed513971a570a952081bfef/packages/vite-node/src/server.ts#L411

console.log(result.map.sources);
> {
>   "sources": [
>     "../node_modules/.pnpm/base32-decode@1.0.0/node_modules/node_modules/.pnpm/base32-decode@1.0.0/node_modules/base32-decode/index.js",
>     "../node_modules/.pnpm/base32-decode@1.0.0/node_modules/base32-decode/pkg-a-entrypoint.js"
>   ],
>   ...
> }

Minimal repro without Vitest:

import { createServer } from "vite";

const server = await createServer();
await server.listen();

const ssrResult = await server.transformRequest(
  "/Users/x/vitest-sourcemap-bug/pkg-a/dist.js",
  { ssr: true }
);
console.log(ssrResult.map);
/*
  sources: [
    '../node_modules/.pnpm/base32-decode@1.0.0/node_modules/node_modules/.pnpm/base32-decode@1.0.0/node_modules/base32-decode/index.js',
    '../node_modules/.pnpm/base32-decode@1.0.0/node_modules/base32-decode/pkg-a-entrypoint.js'
  ],
  file: '/Users/x/vitest-sourcemap-bug/pkg-a/dist.js'
*/

// Without SSR:
const webResult = await server.transformRequest(
  "/Users/x/vitest-sourcemap-bug/pkg-a/dist.js",
  { ssr: false }
);
console.log(webResult.map);
/*
  sources: [
    '../node_modules/.pnpm/base32-decode@1.0.0/node_modules/base32-decode/index.js',
    'pkg-a-entrypoint.js'
  ],
*/

server.close();

If I manually fix the path in vite-node we get proper stack trace in errror:

 FAIL  index.test.js > repro
Error: Test error
 ❯ Module.foo ../pkg-a/pkg-a-entrypoint.js:5:9
 ❯ index.test.js:5:3
      3| 
      4| test("repro", () => {
      5|   foo();
       |   ^
      6| });
      7|

I'll create upstream bug for this.

jwatte commented 1 day ago

I just ran into this yesterday, and this bug is very timely! Just a "me, too." Keep up the good work!

vitest-dev / vitest