Duplicated modules under optimizeDeps

mbeckem commented 1 year ago

Describe the bug

Hi everyone,

as this is my first issue here i'd like to thank you for this great tool ❤.

I am having an issue with vite's optimization of dependencies during development: under some (admittingly very specific, but in my opinion legitimate) circumstances, vite will instantiate a module from node_modules twice. This in turn leads to side effects happening multiple times, confused object identities and instanceof checks failing, which can be very hard to debug. When the depsOptimizer is not present (e.g. during vite build), everything works as expected.

I'll do my best describing the issue. I have also attached a repository with as minimal of a reproduction as i could manage.

The problem

At my company, we are using a vite/rollup plugin to add some reflection capabilities to an application. Essentially, from a single import in the application's code to a virtual module, the apps entire dependency graph may be scanned. All transitive node dependencies are visited and for certain packages code is generated, including imports to modules or objects within those packages.

The reproduction repository uses a simplified system that produces the same problem. A single import "virtual:init-all-packages"; triggers analysis of the dependency graph, visits all referenced packages (just via dependencies), and if their package.json contains a "customAutoInit": "..." declaration, that init module will be imported to trigger the initialization code within that module.

The import "virtual:init-all-packages"; is thus turned into:

import "RESOLVED_LOCATION/dep1/init1.js";
import "RESOLVED_LOCATION/dep2/init2.js";
// ...

Note that this virtual module cannot use bare imports (e.g. import "dep1/init1.js") because transitive dependencies may not be reachable from the virtual modules "location" (nested node_modules etc. should be taken into account). This is why the plugin resolves the module ids passing some importer from where the import is possible. This results in an absolute path like /@fs/.../dep1/init1.js?v=d7007810, which will then be ignored by the depsOptimizer.

The problem here arises when the unoptimized modules mentioned above use node modules that are also used by the application directly (and are thus optimized in node_modules/.vite/...). There will be two copies of any modules used in this way: raw files in node_modules and a copy bundled by esbuild in .vite, with different parts of the application observing one or the other. In the reproduction repository, the module shared/inner is duplicated.

I don't think excluding certain packages from optimization would really help here, since only a static set of packages can be configured and dependencies may add additional imports whenever they feel like it, making any such solution very hard to maintain. Also optimization is a great feature in principle and i would like to make it work correctly in this case.

Additional observations

I have dug around vite's code a bit. All problems would be avoided if the resolve(...) call in our plugin would receive an optimized module id instead of the raw location. The implementation of resolve even has this capability, adding a missing import and then re-triggering optimization, but that branch is skipped in these specific circumstances because the (virtual) importer is in node_modules. However, i cannot use a location outside of node_modules because the module id in question might not be reachable from elsewhere.

I have verified locally that removing the (importer && isInNodeModules(importer)) check from the linked condition would fix the issue in this case. However that line is probably there for a good reason (?). The check was introduced by Evan in this commit, but I couldn't find any additional context.

An additional option for resolve, or an improved skip condition (although i can't think of one) or any other kind of workaround with would be greatly appreciated. However, because the optimizer can alter the observable behaviour of the app in a significant matter, I believe a fix should be internal.

Reproduction

https://github.com/mbeckem/vite-duplicated-modules-repro

Steps to reproduce

Run pnpm install followed by pnpm vite dev and then open the browser, for more details see the linked repository's README.

System Info

System:
    OS: Linux 5.15 Ubuntu 22.04.2 LTS 22.04.2 LTS (Jammy Jellyfish)
    CPU: (16) x64 AMD Ryzen 7 PRO 4750U with Radeon Graphics
    Memory: 17.12 GB / 30.59 GB
    Container: Yes
    Shell: 5.1.16 - /bin/bash
  Binaries:
    Node: 18.13.0 - ~/.nvm/versions/node/v18.13.0/bin/node
    npm: 8.19.3 - ~/.nvm/versions/node/v18.13.0/bin/npm
  Browsers:
    Chrome: 114.0.5735.133
    Firefox: 114.0.1
  npmPackages:
    vite: ^4.3.9 => 4.3.9

Used Package Manager

pnpm

Logs

Click to expand!

```shell $ pnpm vite dev --debug vite:config bundled config file loaded in 50.16ms +0ms vite:config using resolved config: { vite:config root: '/home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/src', vite:config plugins: [ vite:config 'vite:optimized-deps', vite:config 'vite:watch-package-data', vite:config 'vite:pre-alias', vite:config 'alias', vite:config 'vite:modulepreload-polyfill', vite:config 'vite:resolve', vite:config 'vite:html-inline-proxy', vite:config 'vite:css', vite:config 'vite:esbuild', vite:config 'vite:json', vite:config 'vite:wasm-helper', vite:config 'vite:worker', vite:config 'vite:asset', vite:config 'auto-init', vite:config 'vite:wasm-fallback', vite:config 'vite:define', vite:config 'vite:css-post', vite:config 'vite:worker-import-meta-url', vite:config 'vite:asset-import-meta-url', vite:config 'vite:dynamic-import-vars', vite:config 'vite:import-glob', vite:config 'vite:client-inject', vite:config 'vite:import-analysis' vite:config ], vite:config optimizeDeps: { vite:config disabled: 'build', vite:config force: undefined, vite:config esbuildOptions: { preserveSymlinks: false } vite:config }, vite:config server: { vite:config preTransformRequests: true, vite:config sourcemapIgnoreList: [Function: isInNodeModules], vite:config middlewareMode: false, vite:config fs: { strict: true, allow: [Array], deny: [Array] } vite:config }, vite:config configFile: '/home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/vite.config.ts', vite:config configFileDependencies: [ vite:config '/home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/vite.config.ts' vite:config ], vite:config inlineConfig: { vite:config root: undefined, vite:config base: undefined, vite:config mode: undefined, vite:config configFile: undefined, vite:config logLevel: undefined, vite:config clearScreen: undefined, vite:config optimizeDeps: { force: undefined }, vite:config server: {} vite:config }, vite:config base: '/', vite:config rawBase: '/', vite:config resolve: { vite:config mainFields: [ 'module', 'jsnext:main', 'jsnext' ], vite:config browserField: true, vite:config conditions: [], vite:config extensions: [ vite:config '.mjs', '.js', vite:config '.mts', '.ts', vite:config '.jsx', '.tsx', vite:config '.json' vite:config ], vite:config dedupe: [], vite:config preserveSymlinks: false, vite:config alias: [ [Object], [Object] ] vite:config }, vite:config publicDir: '/home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/src/public', vite:config cacheDir: '/home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/node_modules/.vite', vite:config command: 'serve', vite:config mode: 'development', vite:config ssr: { vite:config format: 'esm', vite:config target: 'node', vite:config optimizeDeps: { disabled: true, esbuildOptions: [Object] } vite:config }, vite:config isWorker: false, vite:config mainConfig: null, vite:config isProduction: false, vite:config esbuild: { jsxDev: true }, vite:config build: { vite:config target: [ 'es2020', 'edge88', 'firefox78', 'chrome87', 'safari14' ], vite:config cssTarget: [ 'es2020', 'edge88', 'firefox78', 'chrome87', 'safari14' ], vite:config outDir: 'dist', vite:config assetsDir: 'assets', vite:config assetsInlineLimit: 4096, vite:config cssCodeSplit: true, vite:config sourcemap: false, vite:config rollupOptions: {}, vite:config minify: 'esbuild', vite:config terserOptions: {}, vite:config write: true, vite:config emptyOutDir: null, vite:config copyPublicDir: true, vite:config manifest: false, vite:config lib: false, vite:config ssr: false, vite:config ssrManifest: false, vite:config ssrEmitAssets: false, vite:config reportCompressedSize: true, vite:config chunkSizeWarningLimit: 500, vite:config watch: null, vite:config commonjsOptions: { include: [Array], extensions: [Array] }, vite:config dynamicImportVarsOptions: { warnOnError: true, exclude: [Array] }, vite:config modulePreload: { polyfill: true }, vite:config cssMinify: true vite:config }, vite:config preview: { vite:config port: undefined, vite:config strictPort: undefined, vite:config host: undefined, vite:config https: undefined, vite:config open: undefined, vite:config proxy: undefined, vite:config cors: undefined, vite:config headers: undefined vite:config }, vite:config envDir: '/home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/src', vite:config env: { BASE_URL: '/', MODE: 'development', DEV: true, PROD: false }, vite:config assetsInclude: [Function: assetsInclude], vite:config logger: { vite:config hasWarned: false, vite:config info: [Function: info], vite:config warn: [Function: warn], vite:config warnOnce: [Function: warnOnce], vite:config error: [Function: error], vite:config clearScreen: [Function: clearScreen], vite:config hasErrorLogged: [Function: hasErrorLogged] vite:config }, vite:config packageCache: Map(2) { vite:config 'fnpd_/home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules' => { vite:config dir: '/home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules', vite:config data: [Object], vite:config hasSideEffects: [Function: hasSideEffects], vite:config webResolvedImports: {}, vite:config nodeResolvedImports: {}, vite:config setResolvedCache: [Function: setResolvedCache], vite:config getResolvedCache: [Function: getResolvedCache] vite:config }, vite:config 'fnpd_/home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/src' => { vite:config dir: '/home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules', vite:config data: [Object], vite:config hasSideEffects: [Function: hasSideEffects], vite:config webResolvedImports: {}, vite:config nodeResolvedImports: {}, vite:config setResolvedCache: [Function: setResolvedCache], vite:config getResolvedCache: [Function: getResolvedCache] vite:config }, vite:config set: [Function (anonymous)] vite:config }, vite:config createResolver: [Function: createResolver], vite:config worker: { vite:config format: 'iife', vite:config plugins: [ vite:config 'vite:optimized-deps', vite:config 'vite:watch-package-data', vite:config 'vite:pre-alias', vite:config 'alias', vite:config 'vite:modulepreload-polyfill', vite:config 'vite:resolve', vite:config 'vite:html-inline-proxy', vite:config 'vite:css', vite:config 'vite:esbuild', vite:config 'vite:json', vite:config 'vite:wasm-helper', vite:config 'vite:worker', vite:config 'vite:asset', vite:config 'vite:wasm-fallback', vite:config 'vite:define', vite:config 'vite:css-post', vite:config 'vite:worker-import-meta-url', vite:config 'vite:asset-import-meta-url', vite:config 'vite:dynamic-import-vars', vite:config 'vite:import-glob', vite:config 'vite:client-inject', vite:config 'vite:import-analysis' vite:config ], vite:config rollupOptions: {}, vite:config getSortedPlugins: [Function: getSortedPlugins], vite:config getSortedPluginHooks: [Function: getSortedPluginHooks] vite:config }, vite:config appType: 'spa', vite:config experimental: { importGlobRestoreExtension: false, hmrPartialAccept: false }, vite:config getSortedPlugins: [Function: getSortedPlugins], vite:config getSortedPluginHooks: [Function: getSortedPluginHooks] vite:config } +11ms vite:esbuild 31.21ms tsconfck init /home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules +0ms vite:deps scanning for dependencies... +0ms VITE v4.3.9 ready in 221 ms ➜ Local: http://localhost:5173/ ➜ Network: use --host to expose ➜ press h to show help vite:deps Crawling dependencies using entries: vite:deps /home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/src/index.html +0ms vite:resolve 0.34ms ./app/index.js -> /home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/src/app/index.js +0ms vite:resolve 0.80ms virtual:init-all-packages -> auto-init?from=/home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/src/app/package.json&noext +4ms vite:resolve 0.75ms shared -> /home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/external/node_modules/shared/index.js +3ms vite:deps Scan completed in 39.70ms: vite:deps shared -> /home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/external/node_modules/shared/index.js +26ms vite:deps Dependencies bundled in 7.41ms +0ms vite:html-fallback Rewriting GET / to /index.html +0ms vite:time 37.45ms /index.html +0ms vite:resolve 0.91ms /@vite/client -> /home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/node_modules/.pnpm/vite@4.3.9_@types+node@16.18.36/node_modules/vite/dist/client/client.mjs +0ms vite:resolve 0.21ms /app/index.js -> /home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/src/app/index.js +4ms vite:load 3.59ms [fs] /@vite/client +0ms vite:load 1.73ms [fs] /app/index.js +1ms vite:resolve 0.33ms shared -> /home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/node_modules/.vite/deps/shared.js?v=b189d285 +7ms vite:resolve 0.56ms virtual:init-all-packages -> auto-init?from=/home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/src/app/package.json&noext +0ms vite:import-analysis 4.26ms [2 imports rewritten] app/index.js +0ms vite:transform 7.34ms /app/index.js +0ms vite:time 11.76ms /app/index.js +486ms vite:resolve 3.00ms dep1/package.json -> /home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/external/node_modules/dep1/package.json +6ms vite:resolve 0.65ms dep1/init1.js -> /home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/external/node_modules/dep1/init1.js?v=077357fd +1ms vite:resolve 0.47ms dep2/package.json -> /home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/external/node_modules/dep2/package.json +0ms vite:resolve 0.61ms dep2/initExport -> /home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/external/node_modules/dep2/init2.js?v=077357fd +1ms vite:resolve 0.26ms shared/package.json -> /home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/external/node_modules/shared/package.json +0ms vite:load 7.29ms [plugin] auto-init?from=/home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/src/app/package.json&noext +14ms vite:import-analysis 0.96ms [2 imports rewritten] auto-init?from=/home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/src/app/package.json&noext +7ms vite:transform 1.46ms auto-init?from=/home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/src/app/package.json&noext +8ms vite:resolve 0.12ms @vite/env -> /home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/node_modules/.pnpm/vite@4.3.9_@types+node@16.18.36/node_modules/vite/dist/client/env.mjs +10ms vite:import-analysis 3.27ms [1 imports rewritten] /home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/node_modules/.pnpm/vite@4.3.9_@types+node@16.18.36/node_modules/vite/dist/client/client.mjs +9ms vite:transform 4.02ms /@vite/client +8ms vite:time 32.49ms /@vite/client +15ms vite:cache [memory] auto-init?from=/home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/src/app/package.json&noext +0ms vite:time 0.34ms /@id/__x00__auto-init?from=/home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/src/app/package.json&noext +2ms vite:load 11.68ms [fs] ../external/node_modules/dep2/init2.js?v=077357fd +13ms vite:resolve 0.77ms shared/inner -> /home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/external/node_modules/shared/inner.js?v=077357fd +8ms vite:import-analysis 3.53ms [1 imports rewritten] /home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/external/node_modules/dep2/init2.js?v=077357fd +8ms vite:transform 4.27ms ../external/node_modules/dep2/init2.js?v=077357fd +8ms vite:load 17.28ms [fs] ../external/node_modules/dep1/init1.js?v=077357fd +5ms vite:import-analysis 0.03ms [no imports] /home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/external/node_modules/dep1/init1.js?v=077357fd +2ms vite:transform 0.88ms ../external/node_modules/dep1/init1.js?v=077357fd +2ms vite:load 11.27ms [fs] ../node_modules/.pnpm/vite@4.3.9_@types+node@16.18.36/node_modules/vite/dist/client/env.mjs +3ms vite:load 3.35ms [fs] ../external/node_modules/shared/inner.js?v=077357fd +0ms vite:import-analysis 0.09ms [no imports] /home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/external/node_modules/shared/inner.js?v=077357fd +2ms vite:transform 0.38ms ../external/node_modules/shared/inner.js?v=077357fd +2ms vite:cache [memory] ../external/node_modules/dep1/init1.js?v=077357fd +11ms vite:time 0.44ms ../external/node_modules/dep1/init1.js?v=077357fd +11ms vite:cache [memory] ../external/node_modules/dep2/init2.js?v=077357fd +1ms vite:time 0.47ms ../external/node_modules/dep2/init2.js?v=077357fd +1ms vite:import-analysis 0.07ms [no imports] /home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/node_modules/.pnpm/vite@4.3.9_@types+node@16.18.36/node_modules/vite/dist/client/env.mjs +4ms vite:transform 0.49ms ../node_modules/.pnpm/vite@4.3.9_@types+node@16.18.36/node_modules/vite/dist/client/env.mjs +4ms vite:cache [304] ../node_modules/.pnpm/vite@4.3.9_@types+node@16.18.36/node_modules/vite/dist/client/env.mjs +0ms vite:time 0.40ms ../node_modules/.pnpm/vite@4.3.9_@types+node@16.18.36/node_modules/vite/dist/client/env.mjs +7ms vite:cache [memory] ../external/node_modules/shared/inner.js?v=077357fd +20ms vite:time 0.74ms ../external/node_modules/shared/inner.js?v=077357fd +13ms vite:deps ✨ static imports crawl ended +16s vite:deps ✨ using post-scan optimizer result, the scanner found every used dependency +0ms vite:optimize-deps load /home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/node_modules/.vite/deps/shared.js +0ms vite:deps ✨ dependencies optimized +2ms vite:load 86.71ms [plugin] ../node_modules/.vite/deps/shared.js?v=b189d285 +59ms vite:import-analysis 0.03ms [no imports] /home/michael/projects/pioneer/repro-depsoptimize-duplicated-modules/node_modules/.vite/deps/shared.js?v=b189d285 +54ms vite:transform 0.26ms ../node_modules/.vite/deps/shared.js?v=b189d285 +54ms vite:time 60.24ms ../node_modules/.vite/deps/shared.js?v=b189d285 +37ms ```

Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guidelines.
[X] Read the docs.
[X] Check that there isn't already an issue that reports the same bug to avoid creating a duplicate.
[X] Make sure this is a Vite issue and not a framework-specific issue. For example, if it's a Vue SFC related bug, it should likely be reported to vuejs/core instead.
[X] Check that this is a concrete bug. For Q&A open a GitHub Discussion or join our Discord Chat Server.
[X] The provided reproduction is a minimal reproducible example of the bug.

dardino commented 1 year ago

possible duplicate of https://github.com/vitejs/vite/issues/9171 ?

mbeckem commented 1 year ago

I can't say this with absolute certainty, but I think that this is a different problem (with similar error symptoms, though). #9171 describes duplicated modules within the chunks generated by the optimizer (chunk-xyz.js). This issue is related two duplicate modules where one copy is generated by the optimizer and the other one is not.

Essentially, there are two "import paths" that are not merged into the same module copy, hence the module existing twice.

Edit: I tried running the plugin from the linked issue's description. It does not detect any duplicated modules. But it does demonstrate the issue nicely. The following is a list of all modules rendered during the build (console.log(Object.keys(chunk.modules));) in renderChunk(). (Note: dependency optimization has been enabled for this build via optimizeDeps.disabled = false).

vite v4.3.9 building for production...
✓ 11 modules transformed.
[
  '\x00vite/modulepreload-polyfill',
  '/home/michael/projects/temp/vite-duplicated-modules-repro/external/node_modules/dep1/init1.js',
  /** This is the normal copy of the module */
  '/home/michael/projects/temp/vite-duplicated-modules-repro/external/node_modules/shared/inner.js', 
  '/home/michael/projects/temp/vite-duplicated-modules-repro/external/node_modules/dep2/init2.js',
   /** This module contains a bundled copy of `inner.js` as well! */
  '/home/michael/projects/temp/vite-duplicated-modules-repro/node_modules/.vite/deps_build-dist/shared.js',
  '/home/michael/projects/temp/vite-duplicated-modules-repro/src/app/index.js',
  '/home/michael/projects/temp/vite-duplicated-modules-repro/src/index.html'
]
dist/index.html                0.32 kB │ gzip: 0.23 kB
dist/assets/index-4ec38ee8.js  1.14 kB │ gzip: 0.57 kB
✓ built in 114ms

The contents of deps_build-dist/shared.js are as follows:

// external/node_modules/shared/inner.js
window.SHARED_INNER_EXECUTED ?? (window.SHARED_INNER_EXECUTED = 0);
window.SHARED_INNER_EXECUTED += 1;
//# sourceMappingURL=shared.js.map

Vite config (also see reproduction repo)

```js export default defineConfig({ root: resolve(__dirname, "src"), optimizeDeps: { disabled: false, }, build: { commonjsOptions: { include: [] }, }, plugins: [ autoInitPlugin(), { name: "Test", apply: "build", enforce: "pre", renderChunk(code, chunk) { console.log(Object.keys(chunk.modules)); return null; }, }, ], }); ```

trusktr commented 5 months ago

I don't know much about Vite, so I'm not sure this is related, but I am getting duplicate modules breaking my app, and using resolve.alias is not helping and only causes module not found errors.

With this config,

import {defineConfig} from 'vite'
import path from 'path'

// https://vitejs.dev/config/
export default defineConfig({
    optimizeDeps: {
        noDiscovery: true,
        include: [],
    },
    assetsInclude: ['**/*.glb'],
    build: {
        target: 'esnext',
        // FIXME ^ why do private fields get compiled to __pricateAdd(), __privateGet(), etc, although we use esnext?
        minify: false,
    },
    esbuild: {
        minify: false,
        jsx: 'automatic',
        jsxImportSource: 'preact',
        target: 'esnext',
        // FIXME ^ why do private fields get compiled to __pricateAdd(), __privateGet(), etc, although we use esnext?
    },
    resolve: {
        alias: {
            'solid-js': path.resolve('./node_modules/solid-js/dist/solid.js'),
            'solid-js/store': path.resolve('./node_modules/solid-js/store/dist/store.js'),
            'solid-js/web': path.resolve('./node_modules/solid-js/web/dist/web.js'),
            'solid-js/html': path.resolve('./node_modules/solid-js/html/dist/html.js'),
        },
    },
})

I get errors like these:

Failed to resolve import "solid-js/web" from "node_modules/lume/dist/index.js". Does the file exist?

I have verified that paths like path.resolve('./node_modules/solid-js/dist/solid.js') do actually exist.

Without the resolve.alias, I do not get the missing module errors, but then the app ends up having duplicate libraries (that makes other problems).

I do have an optimizeDeps config. Is it related?

trusktr commented 5 months ago

Maybe my issue is not optimizeDeps-related. Changing my alias config to following no longer has a missing module error, and duplicates are prevented:

        alias: {
            'solid-js': path.resolve('./node_modules/solid-js'),
        },

is-jonreeves commented 1 month ago

Dunno if this is related, but I just ran into an issue where running vite build produced two different sized results depending on if the packages were installed with npm ci vs pnpm i --frozen-lockfile.

I narrowed it down to the use of shared-workspace-lockfile = false in my .npmrc file. Removing, or setting this to true removed the duplication of bundled assets and the filesizes are the same/closer.

For context... This happened for me in a monorepo using "npm workspaces". We mainly use npm, but "npm workspaces" are a bit of nightmare for building and isolating/ejecting backend Node applications for production. Because of this, in CI we opted to use pnpm instead to at least pnpm install --frozen-lockfile the packages at the start, and then pnpm deploy at the end to export an isolated package that we can prep for dist.

This preparation looks a little like:

# Generate PNPM Workspace and Settings
cat 'package.json' | yq eval '{"packages": .workspaces}' -P - > 'pnpm-workspace.yaml'
echo 'link-workspace-packages = true' > '.npmrc'
echo 'shared-workspace-lockfile = false' >> '.npmrc'   # <---- this was causing the duplication
echo 'shamefully-hoist = true' >> '.npmrc'

# Generate PNPM Lockfile
pnpm import

# Install Dependencies
pnpm install --frozen-lockfile

The deployment/ejection looks like:

# Create isolated/deployable application
pnpm deploy --filter="$project_name" --prod /usr/src/deploy

In between those stages we would build and test etc... Using either npm or pnpm to do so, i.e... npm run build or pnpm run build, or npm run build --workspace="$project_name" or pnpm run --filter="$project_name" build.

This is where we noticed that the resulting assets for one of our very-small frontend packages was slightly larger than expected.

Running npm ci in the root of the repo to remove the pnpm installed deps and replace with npm installed ones, then re-running the build fixed the issue. So we determined it was something about how pnpm was structuring the node_modules/. After playing with the pnpm settings, we determined it was the shared-workspace-lockfile causing the problem for us.

Annoyingly, that option is kind of important for backend applications (as it seems to ensure deps with binaries are correctly exported when deploying), but for frontend applications/assets having it is causing this bundling bloat. I was hoping I could create an .npmrc in each package to tailor what we need where, but it doesn't look like these get inherited. The good news is that the setting can be added and removed just before/after pnpm deploy at least and its seems to work out. But that gymnastics is not going to be fun.

Anyways.... in my case at least, this issue seems to be related more to pnpm and not vite necessarily.

vitejs / vite