Dynamic imports/URL reference issues in node_modules

brunnerh commented 1 year ago

Describe the bug

There are various ways in which assets can be imported and referenced and there appear to be various issues when using a package that has assets with the Vite dev server. I also found one scenario in which the build output is also broken.

Use cases for this are components that require images as assets or translation data in separate JSON files by language (so only the file for the current language can be dynamically loaded.)

I constructed a component that tests various scenarios:	Type	`dev`
Globbing (`import.meta.glob(...)`)	✗	✓
URL + fetch without parameter	✗	✓
URL + fetch with parameter	✗	✓
Dynamic import without parameter	✓	✓
Dynamic import with parameter	✗	✗
CSS `url(...)` references	✗	✓
`<img src>` references (static import)	✓	✓

Component code:

<script>
    import imageStatic from './image.png';
    const languages = import.meta.glob('./data/*.json');

    export let lang;

    const url = new URL('./data/de.json', import.meta.url);
    const urlWithParameter = new URL(`./data/${lang}.json`, import.meta.url);
</script>

<p>Globbing</p>

<div>Keys: {Object.keys(languages).length}</div>
{#if Object.keys(languages).length == 0}
    <div class="error">Expected 2 keys.</div>
{/if}

{#each Object.keys(languages) as key}
    <div>Key: {key}</div>
    {#await languages[key]()}
        Loading JSON
    {:then json}
        <div>Value: {json.value}</div>
    {:catch e}
        Failed to load JSON ({key})
        <div class="error">Error: {e.message}</div>
    {/await}
{/each}

<p>URL + fetch without parameter</p>
<div>URL: {url}</div>
{#await fetch(url).then(r => r.json())}
    Fetching JSON
{:then json}
    <div>Value: {json.value}</div>
{:catch e}
    Failed to fetch JSON
    <div class="error">Error: {e.message}</div>
{/await}

<p>URL + fetch with parameter</p>
<div>URL: {urlWithParameter}</div>
{#await fetch(urlWithParameter).then(r => r.json())}
    Fetching JSON
{:then json}
    <div>Value: {json.value}</div>
{:catch e}
    Failed to fetch JSON
    <div class="error">Error: {e.message}</div>
{/await}

<p>Dynamic import without parameter</p>
{#await import('./data/de.json').then(m => m.default)}
    Importing JSON
{:then json}
    <div>Value: {json.value}</div>
{:catch e}
    Failed to import JSON (./data/de.json)
    <div class="error">Error: {e.message}</div>
{/await}

<p>Dynamic import with parameter</p>
{#await import(`./data/${lang}.json`).then(m => m.default)}
    Importing JSON
{:then json}
    <div>Value: {json.value}</div>
{:catch e}
    Failed to import JSON (./data/{lang}.json)
    <div class="error">Error: {e.message}</div>
{/await}

<p>CSS url(...) reference</p>
<div class="bg-img" />

<p>&lt;img src&gt; reference</p>
<img src={imageStatic} alt="" />

<style>
    p { font-weight: bold; }

    .error { color: red; }

    .bg-img {
        width: 100px;
        height: 100px;
        background: url('./image.png') red;
        background-size: 100px;
    }
    img {
        width: 100px;
        height: 100px;
    }
</style>

Workaround for issues that only appear in the dev server:

// vite.config.js
export default defineConfig(({ command }) => {
    return {
        // ...
        optimizeDeps: {
            exclude: command == 'serve' ? ['package-name-here'] : [],
        },
    };
});

Reproduction URL

https://github.com/brunnerh/svelte-package-assets-imports-issue

Reproduction

npm install
Copy test-package into node_modules
npm run dev to view the dev server results.
npm run build and npm run preview to view build results.

Logs

vite:deps Crawling dependencies using entries:
  vite:deps   D:/brunnerh/svelte-package-assets-imports-issue/index.html +0ms
  vite:resolve 1.54ms ./src/main.js -> D:/brunnerh/svelte-package-assets-imports-issue/src/main.js +0ms
  vite:resolve 0.80ms ./app.svelte -> D:/brunnerh/svelte-package-assets-imports-issue/src/app.svelte +3ms
  vite:vite-plugin-svelte resolveId resolved D:\brunnerh\svelte-package-assets-imports-issue\node_modules\test-package\index.js via package.json svelte field of test-package +145ms
  vite:resolve 1.51ms test-package -> D:\brunnerh\svelte-package-assets-imports-issue\node_modules\test-package\index.js +4ms
  vite:deps Scan completed in 41.09ms: {
  'test-package': 'D:/brunnerh/svelte-package-assets-imports-issue/node_modules/test-package/index.js'
} +21ms
  vite:deps dependencies found by scanner: test-package +42ms
  vite:resolve 1.57ms svelte/internal -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/svelte/internal/index.mjs +0ms
  vite:resolve 0.49ms ./image.png -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/test-package/image.png +1ms
  vite:deps deps bundled in 165.84ms +169ms
  vite:html-fallback Rewriting GET / to /index.html +0ms
  vite:time 18.59ms /index.html +0ms
  vite:html-fallback Rewriting GET / to /index.html +38ms
  vite:time 6.09ms /index.html +26ms
  vite:resolve 2.18ms D:\brunnerh\svelte-package-assets-imports-issue\node_modules\vite\dist\client\client.mjs -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/vite/dist/client/client.mjs +0ms
  vite:resolve 4.81ms /@vite/client -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/vite/dist/client/client.mjs +1ms
  vite:load 5.97ms [fs] /@vite/client +0ms
  vite:resolve 0.99ms /src/main.js -> D:/brunnerh/svelte-package-assets-imports-issue/src/main.js +40ms
  vite:resolve 0.59ms D:\brunnerh\svelte-package-assets-imports-issue\node_modules\vite\dist\client\env.mjs -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/vite/dist/client/env.mjs +6ms
  vite:resolve 1.13ms @vite/env -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/vite/dist/client/env.mjs +0ms
  vite:resolve 1.62ms /node_modules/vite/dist/client/env.mjs -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/vite/dist/client/env.mjs +2ms
  vite:import-analysis 7.08ms [1 imports rewritten] node_modules\vite\dist\client\client.mjs +0ms
  vite:transform 39.67ms /@vite/client +0ms
  vite:time 59.87ms /@vite/client +99ms
  vite:load 17.25ms [fs] /src/main.js +49ms
  vite:resolve 0.60ms ./app.svelte -> D:/brunnerh/svelte-package-assets-imports-issue/src/app.svelte +14ms
  vite:resolve 0.90ms /src/app.svelte -> D:/brunnerh/svelte-package-assets-imports-issue/src/app.svelte +1ms
  vite:import-analysis 5.27ms [1 imports rewritten] src\main.js +14ms
  vite:transform 6.67ms /src/main.js +14ms
  vite:time 28.89ms /src/main.js +14ms
  vite:load 11.80ms [fs] /node_modules/vite/dist/client/env.mjs +11ms
  vite:load 4.03ms [fs] /src/app.svelte +4ms
  vite:vite-plugin-svelte setting cssHash s-KKmO-HVyc8Kl for /src/app.svelte +2s
  vite:vite-plugin-svelte transform returns compiled js for D:/brunnerh/svelte-package-assets-imports-issue/src/app.svelte +32ms
  vite:resolve 0.39ms svelte/internal -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/.vite/deps/svelte_internal.js?v=69406817 +49ms
  vite:resolve 0.71ms /node_modules/.vite/deps/svelte_internal.js?v=69406817 -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/.vite/deps/svelte_internal.js?v=69406817 +2ms
  vite:resolve 0.16ms test-package -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/.vite/deps/test-package.js?v=1e0cc558 +6ms
  vite:resolve 0.09ms /node_modules/.vite/deps/test-package.js?v=1e0cc558 -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/.vite/deps/test-package.js?v=1e0cc558 +1ms
  vite:resolve 0.70ms D:/brunnerh/svelte-package-assets-imports-issue/node_modules/svelte-hmr/runtime/hot-api-esm.js -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/svelte-hmr/runtime/hot-api-esm.js?v=22e6423f +1ms
  vite:resolve 1.09ms /node_modules/svelte-hmr/runtime/hot-api-esm.js?v=22e6423f -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/svelte-hmr/runtime/hot-api-esm.js?v=22e6423f +2ms
  vite:resolve 0.94ms D:/brunnerh/svelte-package-assets-imports-issue/node_modules/svelte-hmr/runtime/proxy-adapter-dom.js -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/svelte-hmr/runtime/proxy-adapter-dom.js?v=22e6423f +2ms
  vite:resolve 1.12ms /node_modules/svelte-hmr/runtime/proxy-adapter-dom.js?v=22e6423f -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/svelte-hmr/runtime/proxy-adapter-dom.js?v=22e6423f +1ms
  vite:hmr [accepts-exports] src\app.svelte +0ms
  vite:import-analysis 21.95ms [4 imports rewritten] src\app.svelte +68ms
  vite:transform 59.49ms /src/app.svelte +68ms
  vite:import-analysis 0.03ms [no imports] node_modules\vite\dist\client\env.mjs +6ms
  vite:transform 66.89ms /node_modules/vite/dist/client/env.mjs +5ms
  vite:load 2.26ms [fs] /node_modules/svelte-hmr/runtime/hot-api-esm.js?v=22e6423f +67ms
  vite:resolve 0.63ms ../runtime/index.js -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/svelte-hmr/runtime/index.js?v=22e6423f +16ms
  vite:resolve 1.00ms /node_modules/svelte-hmr/runtime/index.js?v=22e6423f -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/svelte-hmr/runtime/index.js?v=22e6423f +2ms
  vite:import-analysis 7.71ms [1 imports rewritten] node_modules\svelte-hmr\runtime\hot-api-esm.js?v=22e6423f +10ms
  vite:transform 8.52ms /node_modules/svelte-hmr/runtime/hot-api-esm.js?v=22e6423f +11ms
  vite:load 14.07ms [fs] /node_modules/svelte-hmr/runtime/proxy-adapter-dom.js?v=22e6423f +12ms
  vite:resolve 0.45ms ./overlay.js -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/svelte-hmr/runtime/overlay.js?v=22e6423f +12ms
  vite:resolve 0.33ms /node_modules/svelte-hmr/runtime/overlay.js?v=22e6423f -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/svelte-hmr/runtime/overlay.js?v=22e6423f +1ms
  vite:import-analysis 3.95ms [2 imports rewritten] node_modules\svelte-hmr\runtime\proxy-adapter-dom.js?v=22e6423f +9ms
  vite:transform 4.92ms /node_modules/svelte-hmr/runtime/proxy-adapter-dom.js?v=22e6423f +9ms
  vite:load 1.51ms [fs] /node_modules/svelte-hmr/runtime/overlay.js?v=22e6423f +16ms
  vite:import-analysis 0.07ms [no imports] node_modules\svelte-hmr\runtime\overlay.js?v=22e6423f +12ms
  vite:transform 1.46ms /node_modules/svelte-hmr/runtime/overlay.js?v=22e6423f +11ms
  vite:load 19.25ms [fs] /node_modules/svelte-hmr/runtime/index.js?v=22e6423f +3ms
  vite:resolve 0.37ms ./hot-api.js -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/svelte-hmr/runtime/hot-api.js?v=22e6423f +15ms
  vite:resolve 0.76ms /node_modules/svelte-hmr/runtime/hot-api.js?v=22e6423f -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/svelte-hmr/runtime/hot-api.js?v=22e6423f +1ms
  vite:import-analysis 2.69ms [1 imports rewritten] node_modules\svelte-hmr\runtime\index.js?v=22e6423f +5ms
  vite:transform 4.66ms /node_modules/svelte-hmr/runtime/index.js?v=22e6423f +6ms
  vite:load 0.52ms [fs] /node_modules/svelte-hmr/runtime/hot-api.js?v=22e6423f +11ms
  vite:resolve 0.50ms ./proxy.js -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/svelte-hmr/runtime/proxy.js?v=22e6423f +11ms
  vite:resolve 0.49ms /node_modules/svelte-hmr/runtime/proxy.js?v=22e6423f -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/svelte-hmr/runtime/proxy.js?v=22e6423f +1ms
  vite:import-analysis 3.13ms [1 imports rewritten] node_modules\svelte-hmr\runtime\hot-api.js?v=22e6423f +12ms
  vite:transform 3.58ms /node_modules/svelte-hmr/runtime/hot-api.js?v=22e6423f +11ms
  vite:load 0.56ms [fs] /node_modules/svelte-hmr/runtime/proxy.js?v=22e6423f +7ms
  vite:resolve 1.66ms ./svelte-hooks.js -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/svelte-hmr/runtime/svelte-hooks.js?v=22e6423f +8ms
  vite:resolve 0.88ms /node_modules/svelte-hmr/runtime/svelte-hooks.js?v=22e6423f -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/svelte-hmr/runtime/svelte-hooks.js?v=22e6423f +2ms
  vite:import-analysis 5.55ms [1 imports rewritten] node_modules\svelte-hmr\runtime\proxy.js?v=22e6423f +10ms
  vite:transform 6.39ms /node_modules/svelte-hmr/runtime/proxy.js?v=22e6423f +10ms
  vite:load 0.49ms [fs] /node_modules/svelte-hmr/runtime/svelte-hooks.js?v=22e6423f +13ms
  vite:import-analysis 0.87ms [1 imports rewritten] node_modules\svelte-hmr\runtime\svelte-hooks.js?v=22e6423f +7ms
  vite:transform 1.63ms /node_modules/svelte-hmr/runtime/svelte-hooks.js?v=22e6423f +8ms
  vite:deps ✨ static imports crawl ended +2s
  vite:deps ✨ using post-scan optimizer result, the scanner found every used dependency +1ms
  vite:optimize-deps load D:/brunnerh/svelte-package-assets-imports-issue/node_modules/.vite/deps/svelte_internal.js +0ms
  vite:optimize-deps load D:/brunnerh/svelte-package-assets-imports-issue/node_modules/.vite/deps/test-package.js +0ms
  vite:deps ✨ dependencies optimized +4ms
  vite:load 212.12ms [plugin] /node_modules/.vite/deps/svelte_internal.js?v=69406817 +148ms
  vite:resolve 0.11ms ./chunk-L7AXC632.js -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/.vite/deps/chunk-L7AXC632.js?v=22e6423f +156ms
  vite:resolve 0.05ms /node_modules/.vite/deps/chunk-L7AXC632.js?v=22e6423f -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/.vite/deps/chunk-L7AXC632.js?v=22e6423f +0ms
  vite:import-analysis 1.28ms [1 imports rewritten] node_modules\.vite\deps\svelte_internal.js?v=69406817 +148ms
  vite:transform 1.70ms /node_modules/.vite/deps/svelte_internal.js?v=69406817 +147ms
  vite:optimize-deps load D:/brunnerh/svelte-package-assets-imports-issue/node_modules/.vite/deps/chunk-L7AXC632.js +4ms
  vite:load 214.92ms [plugin] /node_modules/.vite/deps/test-package.js?v=1e0cc558 +3ms
  vite:cache [memory] /src/app.svelte +0ms
  vite:time 1.17ms /src/app.svelte +347ms
  vite:cache [memory] /node_modules/vite/dist/client/env.mjs +3ms
  vite:time 1.34ms /node_modules/vite/dist/client/env.mjs +3ms
  vite:resolve 0.36ms /node_modules/test-package/image.png -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/test-package/image.png +71ms
  vite:resolve 0.28ms ./de-AKK4ABVL.js -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/.vite/deps/de-AKK4ABVL.js?v=22e6423f +1ms
  vite:resolve 0.14ms /node_modules/.vite/deps/de-AKK4ABVL.js?v=22e6423f -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/.vite/deps/de-AKK4ABVL.js?v=22e6423f +0ms
  vite:import-analysis 6.00ms [3 imports rewritten] node_modules\.vite\deps\test-package.js?v=1e0cc558 +74ms
  vite:transform 72.97ms /node_modules/.vite/deps/test-package.js?v=1e0cc558 +74ms
  vite:load 0.13ms [plugin] /node_modules/test-package/image.png +75ms
  vite:resolve 0.62ms /undefined -> null +159ms
  vite:html-fallback Rewriting GET /undefined to /index.html +3ms
  vite:time 4.75ms /index.html +6ms
  vite:optimize-deps load D:/brunnerh/svelte-package-assets-imports-issue/node_modules/.vite/deps/de-AKK4ABVL.js +235ms
  vite:resolve 0.25ms /node_modules/.vite/deps/data/en.json -> D:/brunnerh/svelte-package-assets-imports-issue/node_modules/.vite/deps/data/en.json +7ms
  vite:optimize-deps load D:/brunnerh/svelte-package-assets-imports-issue/node_modules/.vite/deps/data/en.json +1ms
  vite:time 5.34ms /node_modules/.vite/deps/data/en.json?import +8ms
  vite:load 7.84ms [plugin] /node_modules/.vite/deps/de-AKK4ABVL.js?v=22e6423f +148ms
  vite:import-analysis 0.02ms [no imports] node_modules\.vite\deps\de-AKK4ABVL.js?v=22e6423f +145ms
  vite:transform 0.43ms /node_modules/.vite/deps/de-AKK4ABVL.js?v=22e6423f +145ms
  vite:time 9.49ms /node_modules/.vite/deps/de-AKK4ABVL.js?v=22e6423f +3ms
  vite:time 20.54ms /node_modules/test-package/image.png +14ms
  vite:html-fallback Not rewriting GET /image.png because the path includes a dot (.) character. +39ms
  vite:time 0.66ms /image.png +11ms
  vite:deps ✨ static imports crawl ended +150ms

System Info

System:
    OS: Windows 10 10.0.22000
    CPU: (8) x64 11th Gen Intel(R) Core(TM) i7-1185G7 @ 3.00GHz
    Memory: 13.06 GB / 31.69 GB
  Binaries:
    Node: 16.18.1 - ~\AppData\Local\Programs\NodeJS\node.EXE
    Yarn: 1.22.19 - ~\AppData\Local\Programs\NodeJS\yarn.CMD
    npm: 8.19.2 - ~\AppData\Local\Programs\NodeJS\npm.CMD
  Browsers:
    Edge: Spartan (44.22000.120.0), Chromium (108.0.1462.42)
    Internet Explorer: 11.0.22000.120
  npmPackages:
    @sveltejs/vite-plugin-svelte: ^1.4.0 => 1.4.0
    svelte: ^3.53.1 => 3.54.0
    vite: ^3.2.5 => 3.2.5

dominikg commented 1 year ago

this isn't really a bug in vite-plugin-svelte. Processing dependencies with vite isn't exactly a feature but merely a side-effect of optimizeDeps.exclude - which we had to set by default for svelte dependencies for a long time because it didn't work.

But recent versions of vite-plugin-svelte enable it by default, so libraries that depend on vite features will not work unless you exclude them from optimizeDeps in your vite config.

Such libraries are a special case, the consumer of these libraries always needs to consume them with vite, otherwise they will break. (eg webpack won't know what import.meta.glob is or even worse, have a different implementation)

So these libraries are better suited for monorepo setups and not actually published to a public registry. If you insist on publishing them, they need to use a peer dependency on vite and accompany documentation on how to set up the required exclusion.

Leaving this open for discussion a bit, but we currently have no plans to support such libraries - and i personally believe you should not build public libraries that only work when consumed with vite.

brunnerh commented 1 year ago

i personally believe you should not build public libraries that only work when consumed with vite

So far these libraries are only published internally. Do you happen to know of a better approach of how asset dependencies could/should be handled in Svelte component libraries? I suspect there is no single solution that would work for all build systems.

bluwy commented 1 year ago

If the libraries are published internally and are meant to be used with Vite only. You can only add them to optimizeDeps.exclude and ssr.noExternal to tell Vite that these libraries should be interpreted as "source code", meaning Vite features will be processed within them.

This is a general issue with libraries that uses Vite-specific features so it's not something vite-plugin-svelte can fix. Ideally Vite's prebundling guide should better cover this, but I haven't got around updating the guide yet. Closing this in favour of https://github.com/vitejs/vite/issues/8735

(PS thanks for the very conclusive reproduction. Not every day we get a well constructed bug report 🙂 )

sveltejs / vite-plugin-svelte