search

dqbd / tiktoken

JS port and JS/WASM bindings for openai/tiktoken
MIT License
652 stars 48 forks source link

When I build my application with esbuild and docker I get: file not found _tiktoken_bg.wasm #6

Closed NoodleOfDeath closed 1 year ago

NoodleOfDeath commented 1 year ago

what is _tiktoken_bg.wasm? and why is my docker image trying to open this? how do i get this to work?

NoodleOfDeath commented 1 year ago

Currently, I am trying a work around in my Docker file touch dist/_tiktoken.wasm, will post an update soon...

NoodleOfDeath commented 1 year ago

nope... still broken what the heck

NoodleOfDeath commented 1 year ago

when did this start becoming an issue? this was never a problem before?

NoodleOfDeath commented 1 year ago

I had to add it to my Dockerfile after esbuild. Seems like a bundler issue? The weird thing is, this was not an issue 18 hours ago. Nothing really changed in my Dockerfile and I've been using esbuild for a week now. Hope this helps you reproduce

// rimraf dist && esbuild ./src --bundle --platform=node --outfile=./dist/index.cjs
RUN pnpm build
COPY _tiktoken_bg.wasm ./dist
// node ./dist/index.cjs
CMD pnpm start
dqbd commented 1 year ago

Thank you for your report. Will look into it.

dqbd commented 1 year ago

It seems like the issue has been resolved by upgrading to 4.0 version, where bundler issues were tackled. Closing this issue for now.

NoodleOfDeath commented 1 year ago

@dqbd You da man!