search

mlc-ai / tokenizers-cpp

Universal cross-platform tokenizers binding to HF and sentencepiece
Apache License 2.0
211 stars 47 forks source link

Fail to import in Node.js #32

Open zcbenz opened 2 months ago

zcbenz commented 2 months ago

It seems that the npm package can not be imported in a Node.js:

import {Tokenizer} from "@mlc-ai/web-tokenizers";
import {Tokenizer} from "@mlc-ai/web-tokenizers";
        ^^^^^^^^^
SyntaxError: The requested module '@mlc-ai/web-tokenizers' does not provide an export named 'Tokenizer'
import Tokenizer from "@mlc-ai/web-tokenizers";
import Tokenizer from "@mlc-ai/web-tokenizers";
       ^^^^^^^^^
SyntaxError: The requested module '@mlc-ai/web-tokenizers' does not provide an export named 'default'

I have no experience with rollup so I can't quite see where went wrong in packaging.