Error: Missing tiktoken_bg.wasm

[rasoulMO]

for last few days I have been trying to solve this issue... I'm using create-llama as starter.

package.json ` { "version": "0.1.0", "scripts": { "format": "prettier --ignore-unknown --cache --check .", "format:write": "prettier --ignore-unknown --write .", "dev": "next dev", "build": "next build", "start": "next start", "lint": "next lint", "generate": "tsx app/api/chat/engine/generate.ts" }, "dependencies": { "@apidevtools/swagger-parser": "^10.1.0", "@e2b/code-interpreter": "^0.0.5", "@llamaindex/pdf-viewer": "^1.1.1", "@radix-ui/react-collapsible": "^1.0.3", "@radix-ui/react-hover-card": "^1.0.7", "@radix-ui/react-slot": "^1.0.2", "ai": "^3.0.21", "ajv": "^8.12.0", "class-variance-authority": "^0.7.0", "clsx": "^2.1.1", "dotenv": "^16.3.1", "duck-duck-scrape": "^2.2.5", "formdata-node": "^6.0.3", "got": "^14.4.1", "llamaindex": "0.4.6", "lucide-react": "^0.294.0", "next": "^14.2.4", "pdf2json": "3.0.5", "react": "^18.2.0", "react-dom": "^18.2.0", "react-markdown": "^8.0.7", "react-syntax-highlighter": "^15.5.0", "rehype-katex": "^7.0.0", "remark": "^14.0.3", "remark-code-import": "^1.2.0", "remark-gfm": "^3.0.1", "remark-math": "^5.1.1", "supports-color": "^8.1.1", "tailwind-merge": "^2.1.0", "tiktoken": "^1.0.15", "uuid": "^9.0.1", "vaul": "^0.9.1" }, "devDependencies": { "@types/node": "^20.10.3", "@types/react": "^18.2.42", "@types/react-dom": "^18.2.17", "@types/react-syntax-highlighter": "^15.5.11", "@types/uuid": "^9.0.8", "autoprefixer": "^10.4.16", "cross-env": "^7.0.3", "eslint": "^8.55.0", "eslint-config-next": "^14.2.4", "eslint-config-prettier": "^8.10.0", "postcss": "^8.4.32", "prettier": "^3.2.5", "prettier-plugin-organize-imports": "^3.2.4", "tailwindcss": "^3.3.6", "tsx": "^4.7.2", "typescript": "^5.3.2" } }

`

app/chat/route `import { initObservability } from "@/app/observability"; import { Message, StreamData, StreamingTextResponse } from "ai"; import { ChatMessage, Settings } from "llamaindex"; import { NextRequest, NextResponse } from "next/server"; import { createChatEngine } from "./engine/chat"; import { initSettings } from "./engine/settings"; import { LlamaIndexStream, convertMessageContent } from "./llamaindex-stream"; import { createCallbackManager, createStreamTimeout } from "./stream-helper";

initObservability(); initSettings();

export const runtime = "nodejs"; export const dynamic = "force-dynamic";

export async function POST(request: NextRequest) { // Init Vercel AI StreamData and timeout const vercelStreamData = new StreamData(); const streamTimeout = createStreamTimeout(vercelStreamData);

try { const body = await request.json(); const { messages }: { messages: Message[] } = body; const userMessage = messages.pop(); if (!messages || !userMessage || userMessage.role !== "user") { return NextResponse.json( { error: "messages are required in the request body and the last message must be from the user", }, { status: 400 }, ); }

const chatEngine = await createChatEngine();

let annotations = userMessage.annotations;
if (!annotations) {
  // the user didn't send any new annotations with the last message
  // so use the annotations from the last user message that has annotations
  // REASON: GPT4 doesn't consider MessageContentDetail from previous messages, only strings
  annotations = messages
    .slice()
    .reverse()
    .find(
      (message) => message.role === "user" && message.annotations,
    )?.annotations;
}

// Convert message content from Vercel/AI format to LlamaIndex/OpenAI format
const userMessageContent = convertMessageContent(
  userMessage.content,
  annotations,
);

// Setup callbacks
const callbackManager = createCallbackManager(vercelStreamData);

// Calling LlamaIndex's ChatEngine to get a streamed response
const response = await Settings.withCallbackManager(callbackManager, () => {
  return chatEngine.chat({
    message: userMessageContent,
    chatHistory: messages as ChatMessage[],
    stream: true,
  });
});

// Transform LlamaIndex stream to Vercel/AI format
const stream = LlamaIndexStream(response, vercelStreamData);

// Return a StreamingTextResponse, which can be consumed by the Vercel/AI client
return new StreamingTextResponse(stream, {}, vercelStreamData);

} catch (error) { console.error("[LlamaIndex]", error); return NextResponse.json( { detail: (error as Error).message, }, { status: 500, }, ); } finally { clearTimeout(streamTimeout); } } `

[marcusschiesser]

That's a bundling issue (not including the tiktoken's WASM code). What are you exactly doing? Just generating a NextJS only app with npx create-llama@0.1.19 and deploying it to Vercel?

[rasoulMO]

yes that's exactly what I'm doing... just deploying the app as it is to Vercel. I don't have this issue locally (with local build).

[marcusschiesser]

@rasoulMO do you run pnpm build or pnpm dev locally?

I just tried:

npx create-llama@latest
Need to install the following packages:
create-llama@0.1.20
Ok to proceed? (y) y

✔ What is your project named? … my-app
✔ Which template would you like to use? › Agentic RAG (single agent)
✔ Which framework would you like to use? › NextJS
✔ Would you like to set up observability? › No
✔ Please provide your OpenAI API key (leave blank to skip): …
✔ Which data source would you like to use? › Use an example PDF
✔ Would you like to add another data source? › No
✔ Would you like to use LlamaParse (improved parser for RAG - requires API key)? … no / yes
✔ Would you like to use a vector database? › No, just store the data in the file system
✔ Would you like to build an agent using tools? If so, select the tools here, otherwise just press enter ›
✔ How would you like to proceed? › Generate code and install dependencies (~2 min)

and then

pnpm build

and that worked

[rasoulMO]

@marcusschiesser Locally working fine. the issue is when deploying to Vercel.... I did pnpm build & pnpm start. it's working as it should... please try to deploy to Vercel. then you will see the error.

[marcusschiesser]

@thucpn, can you have a look into this together with your llamaindex upgrade

[marcusschiesser]

@rasoulMO can you try again adding this to your webpack.config.mjs file?

export default function webpack(config) {
  config.experiments = {
    asyncWebAssembly: true,
    layers: true,
  };

[rasoulMO]

I did try but still give me same issue

[marcusschiesser]

@rasoulMO can you please check if your package.json has the "tiktoken": "^1.0.15" package?

[wanasim]

I had this issue as well. I had to update my Next config file to automatically trace the dependency for all api routes

nex.config.js experimental: { outputFileTracingIncludes: { "/api/**/*": ["./node_modules/**/*.wasm"], }, }

github repo

[i-Mobyl]

Having the same issue. Responding to @marcusschiesser: Yes, package.json contains as a dependency: "tiktoken": "^1.0.15",

Error in Vercel Log: ⨯ Error: Missing tiktoken_bg.wasm at 9312 (/var/task/.next/server/chunks/713.js:77:137123) at t (/var/task/.next/server/webpack-runtime.js:1:143) at 64802 (/var/task/.next/server/chunks/713.js:85:7953) at t (/var/task/.next/server/webpack-runtime.js:1:143) at 4738 (/var/task/.next/server/chunks/713.js:83:96497) at t (/var/task/.next/server/webpack-runtime.js:1:143) at 15713 (/var/task/.next/server/chunks/713.js:192:22224) at t (/var/task/.next/server/webpack-runtime.js:1:143) at 95059 (/var/task/.next/server/app/api/chat/route.js:2:4024) at t (/var/task/.next/server/webpack-runtime.js:1:143) { page: '/api/chat' }

[marcusschiesser]

Makes a lot of sense that @wanasim 's patch is working - great job! I added a PR to include it: https://github.com/run-llama/create-llama/pull/201 Let me know if it solves your issue, then I'll include it in one of the next releases.

[i-Mobyl]

Fyi, I added @wanasim's patch to the next.config.json file, and things are now working.

[marcusschiesser]

added in latest release

[sahmed007]

Obtaining this same issue. Those suggestions above do not seem to be working for me at all.

[marcusschiesser]

@sahmed007 please send the output of create-llama -V and your configuration options

[sahmed007]

Not able to send output of that since I am not using create-llama, however my package version that I am using is "@llamaindex/cloud": "^0.2.2" following this documentation: https://docs.cloud.llamaindex.ai/llamacloud/guides/framework_integration.

[marcusschiesser]

@sahmed007 Bundling tiktoken is a bit tricky; I'll recommend using the configuration from one of the create-llama templates (NextJS or Express). You're also welcome to post your configuration files here https://github.com/run-llama/LlamaIndexTS/issues/1127

[mrsamirr]

I had this issue as well when i deployed on vercel. I had to update my Next config file to automatically trace the dependency for all api routes model: meta-llama/Meta-Llama-3-8B-Instruct -add this in you next.config.js

experimental: { outputFileTracingIncludes: { "/api/**/*": ["./node_modules/**/*.wasm", "./node_modules/tiktoken/**/*.wasm"], "/*": ["./cache/**/*"], }, serverComponentsExternalPackages: ["sharp", "onnxruntime-node", "tiktoken"], },

& also in package.json

add this scripts: "build": "next build && cp ./node_modules/tiktoken/tiktoken_bg.wasm .next/server/chunks/"

if any case checkout repo

before :

after:

[marcusschiesser]

@mrsamirr should not be necessary to do this using withLlamaIndex in next.config.js

what's the llamaindex version that you're using? please try at least 0.6.2

[mrsamirr]

well @marcusschiesser currently i am using the llamaindex version :

└─┬ @upstash/rag-chat@1.6.4
  └── llamaindex@0.5.24

[marcusschiesser]

ah this is outdated, please use at least 0.6.2