daulet / tokenizers

Go bindings for HuggingFace Tokenizer
MIT License
89 stars 23 forks source link

panic: invalid argument #17

Closed wuchaowei2012 closed 5 months ago

wuchaowei2012 commented 6 months ago

The issue is not resolved, so I create it one more time. I tried docker, make build, but no matter what I do, the issue is still not resolved.

i finish to make build, but when i test example/main.go, the following error was found:

➜ tokenizers git:(main) ✗ go run example/main.go --- invalid argument panic: invalid argument

goroutine 1 [running]: main.main() /home/zhaopanpan/aigc/tokenizers/example/main.go:13 +0x396 exit status 2

daulet commented 5 months ago

are you synced up to main branch? do you have local changes? line 13 does not panic.