Closed rasoulMO closed 3 weeks ago
That's a bundling issue (not including the tiktoken's WASM code). What are you exactly doing? Just generating a NextJS only app with npx create-llama@0.1.19
and deploying it to Vercel?
yes that's exactly what I'm doing... just deploying the app as it is to Vercel. I don't have this issue locally (with local build).
@rasoulMO do you run pnpm build
or pnpm dev
locally?
I just tried:
npx create-llama@latest
Need to install the following packages:
create-llama@0.1.20
Ok to proceed? (y) y
✔ What is your project named? … my-app
✔ Which template would you like to use? › Agentic RAG (single agent)
✔ Which framework would you like to use? › NextJS
✔ Would you like to set up observability? › No
✔ Please provide your OpenAI API key (leave blank to skip): …
✔ Which data source would you like to use? › Use an example PDF
✔ Would you like to add another data source? › No
✔ Would you like to use LlamaParse (improved parser for RAG - requires API key)? … no / yes
✔ Would you like to use a vector database? › No, just store the data in the file system
✔ Would you like to build an agent using tools? If so, select the tools here, otherwise just press enter ›
✔ How would you like to proceed? › Generate code and install dependencies (~2 min)
and then
pnpm build
and that worked
@marcusschiesser Locally working fine. the issue is when deploying to Vercel.... I did pnpm build & pnpm start. it's working as it should... please try to deploy to Vercel. then you will see the error.
@thucpn, can you have a look into this together with your llamaindex upgrade
@rasoulMO can you try again adding this to your webpack.config.mjs
file?
export default function webpack(config) {
config.experiments = {
asyncWebAssembly: true,
layers: true,
};
I did try but still give me same issue
@rasoulMO can you please check if your package.json
has the "tiktoken": "^1.0.15"
package?
I had this issue as well. I had to update my Next config file to automatically trace the dependency for all api routes
nex.config.js
experimental: { outputFileTracingIncludes: { "/api/**/*": ["./node_modules/**/*.wasm"], }, }
Having the same issue. Responding to @marcusschiesser: Yes, package.json contains as a dependency:
"tiktoken": "^1.0.15",
Error in Vercel Log: ⨯ Error: Missing tiktoken_bg.wasm at 9312 (/var/task/.next/server/chunks/713.js:77:137123) at t (/var/task/.next/server/webpack-runtime.js:1:143) at 64802 (/var/task/.next/server/chunks/713.js:85:7953) at t (/var/task/.next/server/webpack-runtime.js:1:143) at 4738 (/var/task/.next/server/chunks/713.js:83:96497) at t (/var/task/.next/server/webpack-runtime.js:1:143) at 15713 (/var/task/.next/server/chunks/713.js:192:22224) at t (/var/task/.next/server/webpack-runtime.js:1:143) at 95059 (/var/task/.next/server/app/api/chat/route.js:2:4024) at t (/var/task/.next/server/webpack-runtime.js:1:143) { page: '/api/chat' }
Makes a lot of sense that @wanasim 's patch is working - great job! I added a PR to include it: https://github.com/run-llama/create-llama/pull/201 Let me know if it solves your issue, then I'll include it in one of the next releases.
Fyi, I added @wanasim's patch to the next.config.json file, and things are now working.
added in latest release
Obtaining this same issue. Those suggestions above do not seem to be working for me at all.
@sahmed007 please send the output of create-llama -V
and your configuration options
Not able to send output of that since I am not using create-llama
, however my package version that I am using is "@llamaindex/cloud": "^0.2.2"
following this documentation: https://docs.cloud.llamaindex.ai/llamacloud/guides/framework_integration.
@sahmed007 Bundling tiktoken is a bit tricky; I'll recommend using the configuration from one of the create-llama templates (NextJS or Express). You're also welcome to post your configuration files here https://github.com/run-llama/LlamaIndexTS/issues/1127
I had this issue as well when i deployed on vercel. I had to update my Next config file to automatically trace the dependency for all api routes
model: meta-llama/Meta-Llama-3-8B-Instruct
-add this in you next.config.js
experimental: { outputFileTracingIncludes: { "/api/**/*": ["./node_modules/**/*.wasm", "./node_modules/tiktoken/**/*.wasm"], "/*": ["./cache/**/*"], }, serverComponentsExternalPackages: ["sharp", "onnxruntime-node", "tiktoken"], },
& also in package.json
add this scripts: "build": "next build && cp ./node_modules/tiktoken/tiktoken_bg.wasm .next/server/chunks/"
if any case checkout repo
before :
after:
@mrsamirr
should not be necessary to do this using withLlamaIndex
in next.config.js
what's the llamaindex version that you're using? please try at least 0.6.2
well @marcusschiesser currently i am using the llamaindex version :
└─┬ @upstash/rag-chat@1.6.4
└── llamaindex@0.5.24
ah this is outdated, please use at least 0.6.2
for last few days I have been trying to solve this issue... I'm using create-llama as starter.
package.json ` { "version": "0.1.0", "scripts": { "format": "prettier --ignore-unknown --cache --check .", "format:write": "prettier --ignore-unknown --write .", "dev": "next dev", "build": "next build", "start": "next start", "lint": "next lint", "generate": "tsx app/api/chat/engine/generate.ts" }, "dependencies": { "@apidevtools/swagger-parser": "^10.1.0", "@e2b/code-interpreter": "^0.0.5", "@llamaindex/pdf-viewer": "^1.1.1", "@radix-ui/react-collapsible": "^1.0.3", "@radix-ui/react-hover-card": "^1.0.7", "@radix-ui/react-slot": "^1.0.2", "ai": "^3.0.21", "ajv": "^8.12.0", "class-variance-authority": "^0.7.0", "clsx": "^2.1.1", "dotenv": "^16.3.1", "duck-duck-scrape": "^2.2.5", "formdata-node": "^6.0.3", "got": "^14.4.1", "llamaindex": "0.4.6", "lucide-react": "^0.294.0", "next": "^14.2.4", "pdf2json": "3.0.5", "react": "^18.2.0", "react-dom": "^18.2.0", "react-markdown": "^8.0.7", "react-syntax-highlighter": "^15.5.0", "rehype-katex": "^7.0.0", "remark": "^14.0.3", "remark-code-import": "^1.2.0", "remark-gfm": "^3.0.1", "remark-math": "^5.1.1", "supports-color": "^8.1.1", "tailwind-merge": "^2.1.0", "tiktoken": "^1.0.15", "uuid": "^9.0.1", "vaul": "^0.9.1" }, "devDependencies": { "@types/node": "^20.10.3", "@types/react": "^18.2.42", "@types/react-dom": "^18.2.17", "@types/react-syntax-highlighter": "^15.5.11", "@types/uuid": "^9.0.8", "autoprefixer": "^10.4.16", "cross-env": "^7.0.3", "eslint": "^8.55.0", "eslint-config-next": "^14.2.4", "eslint-config-prettier": "^8.10.0", "postcss": "^8.4.32", "prettier": "^3.2.5", "prettier-plugin-organize-imports": "^3.2.4", "tailwindcss": "^3.3.6", "tsx": "^4.7.2", "typescript": "^5.3.2" } }
`
app/chat/route `import { initObservability } from "@/app/observability"; import { Message, StreamData, StreamingTextResponse } from "ai"; import { ChatMessage, Settings } from "llamaindex"; import { NextRequest, NextResponse } from "next/server"; import { createChatEngine } from "./engine/chat"; import { initSettings } from "./engine/settings"; import { LlamaIndexStream, convertMessageContent } from "./llamaindex-stream"; import { createCallbackManager, createStreamTimeout } from "./stream-helper";
initObservability(); initSettings();
export const runtime = "nodejs"; export const dynamic = "force-dynamic";
export async function POST(request: NextRequest) { // Init Vercel AI StreamData and timeout const vercelStreamData = new StreamData(); const streamTimeout = createStreamTimeout(vercelStreamData);
try { const body = await request.json(); const { messages }: { messages: Message[] } = body; const userMessage = messages.pop(); if (!messages || !userMessage || userMessage.role !== "user") { return NextResponse.json( { error: "messages are required in the request body and the last message must be from the user", }, { status: 400 }, ); }
} catch (error) { console.error("[LlamaIndex]", error); return NextResponse.json( { detail: (error as Error).message, }, { status: 500, }, ); } finally { clearTimeout(streamTimeout); } } `