ERROR: No loader is configured for ".wasm" files: node_modules/.pnpm/@dqbd+tiktoken@1.0.7/node_modules/@dqbd/tiktoken/lite/tiktoken_bg.wasm

[happysalada]

First of all, thanks for the package! I've read your readme talking about the no loader error, and I've tried the proposed fixed to no avail. Here is my config (using latest of everything)

        "vite": "^4.3.5",
        "vite-plugin-top-level-await": "^1.3.0",
        "vite-plugin-wasm": "^3.2.2"

in vite.config.js I have

import type { UserConfig } from 'vite';
import { sveltekit } from '@sveltejs/kit/vite';
import { ViteImageOptimizer } from 'vite-plugin-image-optimizer';
import nodePolyfills from 'vite-plugin-node-stdlib-browser'
import { isoImport } from 'vite-plugin-iso-import'
import wasm from 'vite-plugin-wasm';
import topLevelAwait from "vite-plugin-top-level-await";

const config: UserConfig = {
    plugins: [
        wasm(),
        topLevelAwait(),
        isoImport(),
        nodePolyfills(),
        sveltekit(),
        ViteImageOptimizer({})
    ],
    optimizeDeps: {
    exclude: [
      "@dqbd/tiktoken"
    ]
  }
};

export default config;

however I'm still getting the error. I would be happy to try anything if you have ideas

[Menci]

Could you please provide a simple project to reproduce it? Thanks.

[happysalada]

@Menci https://github.com/happysalada/glad-chat let me know if you need anything else.

[Menci]

Thanks. I'll look into this later.

[mrhoodz]

Im also bumping into a similar problem 😥 https://github.com/mrhoodz/satori-attempt

[happysalada]

I'll be making commits to the repo, so just to say that this is the commit where the problem can be reproduced https://github.com/happysalada/glad-chat/tree/68d6d2bfa1895a73881ee8f292f2565a3be58c4b (of course let me know if anything).

[Menci]

@happysalada The error message I get is:

> glad-chat@0.0.1 build /Users/Menci/glad-chat
> vite build

vite v4.3.9 building SSR bundle for production...
transforming (11) node_modules/.pnpm/@sveltejs+kit@1.20.1_svelte@3.59.1+vite@4.3.9/node_modules/@sveltejs/kit/src/runtime/server/endpoint.js3:27:57 PM [vite-plugin-svelte] /src/routes/signin/+page.svelte:73:3 A11y: '#' is not a valid href attribute
71:   >
72:     <a
73:       href="#"
          ^
74:       class="flex items-center mb-6 text-2xl font-semibold text-white"
75:     >
3:27:57 PM [vite-plugin-svelte] /src/routes/signin/+page.svelte:341:9 A11y: '#' is not a valid href attribute
339:                 >I accept the <a
340:                   class="font-medium text-primary-600 hover:underline dark:text-primary-500"
341:                   href="#">Terms and Conditions</a
                       ^
342:                 ></label
343:               >
✓ 91 modules transformed.
✓ built in 1.16s
"default" is not exported by "node_modules/.pnpm/@dqbd+tiktoken@1.0.7/node_modules/@dqbd/tiktoken/lite/tiktoken_bg.wasm", imported by "src/routes/+page.server.ts".
file: /Users/Menci/glad-chat/src/routes/+page.server.ts:6:7
4: } from "openai";
5: import { init, Tiktoken } from "@dqbd/tiktoken/lite/init";
6: import wasm from "@dqbd/tiktoken/lite/tiktoken_bg.wasm";
          ^
7: import model from "@dqbd/tiktoken/encoders/cl100k_base.json";
error during build:
RollupError: "default" is not exported by "node_modules/.pnpm/@dqbd+tiktoken@1.0.7/node_modules/@dqbd/tiktoken/lite/tiktoken_bg.wasm", imported by "src/routes/+page.server.ts".
    at error (file:///Users/Menci/glad-chat/node_modules/.pnpm/rollup@3.23.0/node_modules/rollup/dist/es/shared/node-entry.js:2124:30)
    at Module.error (file:///Users/Menci/glad-chat/node_modules/.pnpm/rollup@3.23.0/node_modules/rollup/dist/es/shared/node-entry.js:13463:16)
    at Module.traceVariable (file:///Users/Menci/glad-chat/node_modules/.pnpm/rollup@3.23.0/node_modules/rollup/dist/es/shared/node-entry.js:13884:29)
    at ModuleScope.findVariable (file:///Users/Menci/glad-chat/node_modules/.pnpm/rollup@3.23.0/node_modules/rollup/dist/es/shared/node-entry.js:12429:39)
    at ReturnValueScope.findVariable (file:///Users/Menci/glad-chat/node_modules/.pnpm/rollup@3.23.0/node_modules/rollup/dist/es/shared/node-entry.js:6965:38)
    at ChildScope.findVariable (file:///Users/Menci/glad-chat/node_modules/.pnpm/rollup@3.23.0/node_modules/rollup/dist/es/shared/node-entry.js:6965:38)
    at BlockScope.findVariable (file:///Users/Menci/glad-chat/node_modules/.pnpm/rollup@3.23.0/node_modules/rollup/dist/es/shared/node-entry.js:6965:38)
    at ReturnValueScope.findVariable (file:///Users/Menci/glad-chat/node_modules/.pnpm/rollup@3.23.0/node_modules/rollup/dist/es/shared/node-entry.js:6965:38)
    at Identifier.bind (file:///Users/Menci/glad-chat/node_modules/.pnpm/rollup@3.23.0/node_modules/rollup/dist/es/shared/node-entry.js:8127:40)
    at CallExpression.bind (file:///Users/Menci/glad-chat/node_modules/.pnpm/rollup@3.23.0/node_modules/rollup/dist/es/shared/node-entry.js:5734:28)
 ELIFECYCLE  Command failed with exit code 1.

You shouldn't import default from a WASM file since it doesn't have an default import. And you shouldn't use this plugin with manually instantiating. Instead you should import the package directly.

diff --git a/src/routes/+page.server.ts b/src/routes/+page.server.ts
index 854209d..968a868 100644
--- a/src/routes/+page.server.ts
+++ b/src/routes/+page.server.ts
@@ -2,8 +2,8 @@ import type { Actions } from './$types';
 import {
        ChatCompletionRequestMessageRoleEnum,
 } from "openai";
-import { init, Tiktoken } from "@dqbd/tiktoken/lite/init";
-import wasm from "@dqbd/tiktoken/lite/tiktoken_bg.wasm";
+import { Tiktoken } from "@dqbd/tiktoken";
+// import wasm from "@dqbd/tiktoken/lite/tiktoken_bg.wasm";
 import model from "@dqbd/tiktoken/encoders/cl100k_base.json";

 const mergeMaxTokens = (list: string[], encoder: Tiktoken, max: number): string[] => {
@@ -120,7 +120,7 @@ export const actions = {
                                }
                        );
                        const { result: qdrantPayloads } = await qdrantResponse.json();
-                       await init((imports) => WebAssembly.instantiate(wasm, imports));
+                       // await init((imports) => WebAssembly.instantiate(wasm, imports));
                        const encoder = new Tiktoken(
                                model.bpe_ranks,
                                model.special_tokens,

[Menci]

@mrhoodz You're using @vercel/og which is NOT WebAssembly ESM. It supports Node.js and Vercel's Edge Runtime but doesn't support frontend bundler. The plugin can't help you use that in Vite.

[happysalada]

Thanks for taking a look at this. After doing what you recommended, I'm still getting an error "TypeError: malloc is not a function" which seems wasm related. I've opened an issue in the original repo though. Thanks again!

[Menci]

To check if an issue is this plugin related or not, you can build your project with Webpack 5 (asyncWebAssembly). This plugin's behavior should be the same as that.