Cannot find module './dist/tiktoken-node.linux-x64-gnu.node' when deployed on Vercel - Nuxt 3

ceifa / tiktoken-node

OpenAI's tiktoken but with node bindings

103 stars 10 forks source link

Cannot find module './dist/tiktoken-node.linux-x64-gnu.node' when deployed on Vercel - Nuxt 3 #10

Open balsimpson opened 1 year ago

balsimpson commented 1 year ago

[nuxt] [request error] [unhandled] [500] Cannot find module './dist/tiktoken-node.linux-x64-gnu.node' Require stack:

/var/task/node_modules/tiktoken-node/index.mjs at Module._resolveFilename (node:internal/modules/cjs/loader:1039:15)
at Module._load (node:internal/modules/cjs/loader:885:27)
at Module.require (node:internal/modules/cjs/loader:1105:19)
at require (node:internal/modules/cjs/helpers:103:18)
at ./node_modules/tiktoken-node/index.mjs:11:21
at ModuleJob.run (node:internal/modules/esm/module_job:194:25)

It works perfectly fine in development mode in my local machine, but the deployed code gives me this error on the server.

ceifa commented 1 year ago

This is a problem on your publishing. Check if the .node file it's on the correct location.

balsimpson commented 1 year ago

I don't have enough understanding of that to correct it. My project is just standard Nuxt 3.

mydaoyuan commented 1 year ago

node:internal/modules/cjs/loader:1338 return process.dlopen(module, path.toNamespacedPath(filename)); ^ Error: Error loading shared library ld-linux-x86-64.so.2: No such file or directory (needed by /app/node_modules/.pnpm/tiktoken-node@0.0.5/node_modules/tiktoken-node/dist/tiktoken-node.linux-x64-gnu.node)

Same as me, I use Docker

JanRuettinger commented 1 year ago

@balsimpson Did you solve your issue? I run into the same issue.

JanRuettinger commented 1 year ago

This is a problem on your publishing. Check if the .node file it's on the correct location.

What .node file do you mean?

balsimpson commented 1 year ago

@balsimpson Did you solve your issue? I run into the same issue.

nope. just abandoned it ;-)

balsimpson commented 1 year ago

same issue. What a nightmare. What library you ended up using?

None. couldn't find a good one.

ceifa commented 1 year ago

@mydaoyuan actually your problem is related on #8. Do you have a minimal repro to it?

ceifa commented 1 year ago

Fix probably located here: https://github.com/napi-rs/package-template/blob/main/.github/workflows/CI.yml

AaronFriel commented 11 months ago

@mydaoyuan's issue is likely that they are using Alpine Linux or another MUSL. The gcompat package is needed to add the dependency. Adding that package before npm install does fix that specific error for me.

FROM node:lts-alpine
...
RUN apk add --no-cache libc6-compat gcompat
RUN apk update

Unfortunately, as a musl libc distribution I ran into another linker issue:

Error relocating /app/node_modules/tiktoken-node/dist/tiktoken-node.linux-x64-gnu.node: fwrite: initial-exec TLS resolves to dynamic definition in /app/node_modules/tiktoken-node/dist/tiktoken-node.linux-x64-gnu.node