search

niieani / gpt-tokenizer

The fastest JavaScript BPE Tokenizer Encoder Decoder for OpenAI's GPT-2 / GPT-3 / GPT-4 / GPT-4o. Port of OpenAI's tiktoken with additional features.
https://gpt-tokenizer.dev
MIT License
432 stars 35 forks source link

ReferenceError: Property 'TextDecoder' doesn't exist, js engine: hermes #28

Closed YunySun closed 2 months ago

YunySun commented 1 year ago

hello, i use it in rn project, it took a mistake 

ReferenceError: Property 'TextDecoder' doesn't exist, js engine: hermes
    at SharePage (http://localhost:8081/src\pages\share\index.bundle?platform=android&app=com.awesomeproject&modulesOnly=true&dev=true&minify=false&runModule=true&shallow=true:12:26)
    at RNGestureHandlerRootView
    at GestureHandlerRootView
    at App
    at RCTView
niieani commented 2 months ago

If you want to use it with hermes, you need to make sure that TextDecoder global is present / polyfilled first.