search

dqbd / tiktoken

JS port and JS/WASM bindings for openai/tiktoken
MIT License
649 stars 49 forks source link

@sveltejs/adapter-cloudflare: cloudflare worker (pages) with vite: "default" is not exported by module #37

Closed happysalada closed 11 months ago

happysalada commented 1 year ago

hi, I've tried the full vite and cloudflare worker setup, however vite seems to have a problem on the wasm file

error during build:
RollupError: "default" is not exported by "node_modules/.pnpm/@dqbd+tiktoken@1.0.7/node_modules/@dqbd/tiktoken/lite/tiktoken_bg.wasm", imported by "src/routes/+page.server.ts".
    at error (file:///Users/raphael/dev/glad-chat/node_modules/.pnpm/rollup@3.21.5/node_modules/rollup/dist/es/shared/node-entry.js:2125:30)
    at Module.error (file:///Users/raphael/dev/glad-chat/node_modules/.pnpm/rollup@3.21.5/node_modules/rollup/dist/es/shared/node-entry.js:13452:16)
    at Module.traceVariable (file:///Users/raphael/dev/glad-chat/node_modules/.pnpm/rollup@3.21.5/node_modules/rollup/dist/es/shared/node-entry.js:13863:29)
    at ModuleScope.findVariable (file:///Users/raphael/dev/glad-chat/node_modules/.pnpm/rollup@3.21.5/node_modules/rollup/dist/es/shared/node-entry.js:12418:39)
    at ReturnValueScope.findVariable (file:///Users/raphael/dev/glad-chat/node_modules/.pnpm/rollup@3.21.5/node_modules/rollup/dist/es/shared/node-entry.js:6966:38)
    at ChildScope.findVariable (file:///Users/raphael/dev/glad-chat/node_modules/.pnpm/rollup@3.21.5/node_modules/rollup/dist/es/shared/node-entry.js:6966:38)
    at BlockScope.findVariable (file:///Users/raphael/dev/glad-chat/node_modules/.pnpm/rollup@3.21.5/node_modules/rollup/dist/es/shared/node-entry.js:6966:38)
    at ReturnValueScope.findVariable (file:///Users/raphael/dev/glad-chat/node_modules/.pnpm/rollup@3.21.5/node_modules/rollup/dist/es/shared/node-entry.js:6966:38)
    at Identifier.bind (file:///Users/raphael/dev/glad-chat/node_modules/.pnpm/rollup@3.21.5/node_modules/rollup/dist/es/shared/node-entry.js:8116:40)
    at CallExpression.bind (file:///Users/raphael/dev/glad-chat/node_modules/.pnpm/rollup@3.21.5/node_modules/rollup/dist/es/shared/node-entry.js:5735:28)
 ELIFECYCLE  Command failed with exit code 1.

just to make sure, I've got both

        wasm(),
        topLevelAwait(),

in my vite config and I have got the following in my server page

import { Tiktoken, init } from "@dqbd/tiktoken/lite/init";
import wasm from "@dqbd/tiktoken/lite/tiktoken_bg.wasm";
import model from "@dqbd/tiktoken/encoders/cl100k_base.json";

await init((imports) => WebAssembly.instantiate(wasm, imports));
      const encoder = new Tiktoken(
          model.bpe_ranks,
          model.special_tokens,
          model.pat_str
        );

not having the init line actually creates an error that the wasm wasn't initialized properly.

(I'm using sveltekit and deploying on cloudflare pages, just for context).

matthewrobb commented 1 year ago

I too am having this issue with the exact same situation

happysalada commented 1 year ago

@matthewrobb actually after more digging it seems to be a problem with the vite wasm plugin, ive opened an issue there if you want to check it out.

happysalada commented 1 year ago

The documentation on this repo is just wrong

dqbd commented 1 year ago

Hi! Sorry for the delay.

The combination of SvelteKit + Cloudflare Pages / Workers has not been properly tested yet. Will reflect that in README.md.

@happysalada, @matthewrobb could you please share a repository with a reproducible case? Even small things such as version of the packages may be the root of the issue.

If that is not possible, consider using js-tiktoken instead, which is a pure JS port instead of WASM bindings. This does come at a certain performance penalty (at the moment around 5x slower, future PRs will reduce the gap to 2.5x), but will work just fine for these untested environments.

Here are some preliminary notes when finding the cause of the issue.

SvelteKit + Cloudflare is a tricky combination. When using tiktoken/lite/init + tiktoken/lite/tiktoken_bg.wasm with vite-plugin-wasm, Vite inlines the WASM file and instantiates it directly AND with missing properties (memory etc.). This does go against current init() function, which assumes a WASM file to be a WebAssembly.Module instead.

On the other way, when using tiktoken/lite directly and importing WASM file is done by the package itself, the WASM import results into a WebAssembly.Module is found in default key, requiring calling init() inside the package.

happysalada commented 1 year ago

here is the repo https://github.com/happysalada/glad-chat I've updated the dependencies to latest. running pnpm run build should get you the error.

I've tried import * as wasm from ... but that seems to mess the worker such that it returns a 405 on any request.

I've tried using titoken/lite directly and in that case the project builds fine, but once deployed it errors out with (log) TypeError: malloc is not a function (my test for that is at https://github.com/happysalada/glad-chat/commit/cec73b1987a029607bf67443a3ebdc33508b0606)

happysalada commented 1 year ago

@dqbd did you have a chance to have a look ?

happysalada commented 1 year ago

small update that since I'll be making commits to the repo here is the commit where the problem can be reproduced https://github.com/happysalada/glad-chat/tree/68d6d2bfa1895a73881ee8f292f2565a3be58c4b

dqbd commented 1 year ago

Hello! Can reproduce the issue, but still need more time to see whether it is something I can do on the package side. Meanwhile updated the README.md to reflect the incompatibility issue.

happysalada commented 1 year ago

After reporting on the vite wasm plugin https://github.com/Menci/vite-plugin-wasm/issues/33#issuecomment-1574749234 The feedback I had was that it should be a simple import of Tiktoken since wasm files don't have a default import. I tried it on this branch https://github.com/happysalada/glad-chat/blob/wasm/src/routes/%2Bpage.server.ts#L7 and the error I'm getting is "TypeError: malloc is not a function" hopefully that's helpful.

SunnyGPT commented 1 year ago

Hello! Can reproduce the issue, but still need more time to see whether it is something I can do on the package side. Meanwhile updated the README.md to reflect the incompatibility issue.

Can I pay you to integrate it with cloudflare worker? Because I don't know programming, it is too difficult to deploy, so I look forward to your reply.

dqbd commented 11 months ago

Hi! For now, consider using js-tiktoken instead, which makes more sense in size-constrainted edge environments. Feel free to reopen if needed