search

vitejs / vite

Next generation frontend tooling. It's fast!
http://vite.dev
MIT License
68.08k stars 6.13k forks source link

"Could not read from file …" error thrown when using `?url` and `?worker` suffixes inside 3rd party module #10838

Open wojtekmaj opened 1 year ago

wojtekmaj commented 1 year ago

Describe the bug

Getting module URL in our own code like so:

import module2Url from "module-2/index?url";

works fine, but if that very same line is placed in node_modules/module-1/index.js, build crashes:

wmaj@MacBook-Pro vite-url-suffix-bug % yarn dev          
Port 5173 is in use, trying another one...

  VITE v3.2.3  ready in 83 ms

  ➜  Local:   http://localhost:5174/
  ➜  Network: use --host to expose
✘ [ERROR] Could not read from file: /vite-url-suffix-bug/node_modules/module-2/index.js?url

    node_modules/module-1/index.js:1:23:
      1 │ import module2Url from "module-2/index?url";
        ╵                        ~~~~~~~~~~~~~~~~~~~~

/vite-url-suffix-bug/node_modules/esbuild/lib/main.js:1566
  let error = new Error(`${text}${summary}`);
              ^

Error: Build failed with 1 error:
node_modules/module-1/index.js:1:23: ERROR: Could not read from file: /vite-url-suffix-bug/node_modules/module-2/index.js?url
    at failureErrorWithLog (/vite-url-suffix-bug/node_modules/esbuild/lib/main.js:1566:15)
    at /vite-url-suffix-bug/node_modules/esbuild/lib/main.js:1024:28
    at runOnEndCallbacks (/vite-url-suffix-bug/node_modules/esbuild/lib/main.js:1438:61)
    at buildResponseToResult (/vite-url-suffix-bug/node_modules/esbuild/lib/main.js:1022:7)
    at /vite-url-suffix-bug/node_modules/esbuild/lib/main.js:1134:14
    at responseCallbacks.<computed> (/vite-url-suffix-bug/node_modules/esbuild/lib/main.js:671:9)
    at handleIncomingPacket (/vite-url-suffix-bug/node_modules/esbuild/lib/main.js:726:9)
    at Socket.readFromStdout (/vite-url-suffix-bug/node_modules/esbuild/lib/main.js:647:7)
    at Socket.emit (node:events:513:28)
    at addChunk (node:internal/streams/readable:324:12) {
  errors: [
    {
      detail: undefined,
      id: '',
      location: {
        column: 23,
        file: 'node_modules/module-1/index.js',
        length: 20,
        line: 1,
        lineText: 'import module2Url from "module-2/index?url";',
        namespace: '',
        suggestion: ''
      },
      notes: [],
      pluginName: '',
      text: 'Could not read from file: /vite-url-suffix-bug/node_modules/module-2/index.js?url'
    }
  ],
  warnings: []
}

Node.js v18.12.0

At the same time, I was not able to find any mention regarding these suffixed working only on limited subset of modules when reading the following page: https://vitejs.dev/guide/assets.html

Reproduction

https://github.com/wojtekmaj/vite-url-suffix-bug

Steps to reproduce

System Info

System:
    OS: macOS 13.0
    CPU: (8) arm64 Apple M1 Pro
    Memory: 457.75 MB / 16.00 GB
    Shell: 5.8.1 - /bin/zsh
  Binaries:
    Node: 18.12.0 - /usr/local/bin/node
    Yarn: 3.2.4 - /usr/local/bin/yarn
    npm: 8.19.2 - /usr/local/bin/npm
  Browsers:
    Chrome: 107.0.5304.87
    Firefox: 106.0.3
    Safari: 16.1
  npmPackages:
    vite: ^3.2.3 => 3.2.3

Used Package Manager

yarn

Logs

No response

Validations

seo-rii commented 11 months ago

Is there any update about this?

TheMoonDawg commented 8 months ago

In my case, I was updating to React PDF v7 and ran into this issue. What I ended up doing was including the transitive dependency / 3rd party module in our Vite config to also optimize it when using the ?url suffix. Worked like a charm!

export default defineConfig({
  ...
  optimizeDeps: { include: ['pdfjs-dist'] },
})
Danielku15 commented 6 months ago

I dived a bit into the bug and the problem is in this line: https://github.com/vitejs/vite/blob/e526573cae8a2fb073bf9a38bef17aeb9adbebc3/packages/vite/src/node/plugins/workerImportMetaUrl.ts#L154

If the file system resolve fails here, we just stay on the artificially produced path like: 'D:\Dev\myproj\src\vite-react\node_modules\.vite\deps\mylib.worker.mjs'. This leads to two problems:

  1. The worker file is considered an "optimized dependency" because its file path starts with .vite\deps https://github.com/vitejs/vite/blob/f8e0791e3f7c7c39c041a563e77396eca706d05e/packages/vite/src/node/optimizer/index.ts#L912-L921
  2. This path is then encoded and decoded to URLs and used by modules but will end up with files which cannot be found.

Hence a fix would mean: resolve at this location the correct path to the worker file of the 3rd party module in the node_modules directory.

It seems a code bit like this will work:

file = tryFsResolve(file, fsResolveOptions) ?? tryOptimizedDepResolve(url, id) ?? file;

function tryOptimizedDepResolve(url, depId, fsResolveOptions) {
    if (depsOptimizer?.isOptimizedDepFile(depId)) {
        const depFile = cleanUrl(depId);
        const info = optimizedDepInfoFromFile(depsOptimizer.metadata, depFile);
        const depSrc = info.src;
        if (depSrc) {
            const resolvedFile = path.resolve(path.dirname(depSrc), url)
            return tryFsResolve(file, fsResolveOptions);
        }
    }

    return undefined;
}

It is a similar flow like the existing tryOptimizedResolve but simplified.

I'll check if I can make a change to Vite and test it locally.

Danielku15 commented 6 months ago

I prepared a fix which works nicely for my library at https://github.com/vitejs/vite/pull/16418 Would be great if some more people could give it a test. To test it locally:

  1. Clone and built vite from https://github.com/Danielku15/vite/tree/fix/workers-in-3rd-party-modules (pnpm install && pnpm build)
  2. Copy the local vite build to your project where you use vite with a 3rd party library.
  3. Run and test your application and check if the worker loads.
timephy commented 1 month ago

Running into the same issue, would love to see this fixed..