Update build.sh to make web tokenizer work with chrome extension

mlc-ai / tokenizers-cpp

Universal cross-platform tokenizers binding to HF and sentencepiece

Apache License 2.0

253 stars 58 forks source link

Update build.sh to make web tokenizer work with chrome extension #20

Closed manuongithub closed 10 months ago

manuongithub commented 10 months ago

web-llm's chrome extension not working with manifest version 3 and Google will be stopping manifest version 2 in January, 2024

The chrome extension isn't working because of use of evals in code. using -s NO_DYNAMIC_EXECUTION=1 removes eval() and new Function() from generated code

refer: discussions in https://github.com/mlc-ai/web-llm/issues/214

tqchen commented 10 months ago

cc @CharlieFRuan @sudeepag do you mind help release a new version in npm

CharlieFRuan commented 10 months ago

@tqchen @manuongithub Just published 0.1.2 https://www.npmjs.com/package/@mlc-ai/web-tokenizers