search

xyzhang626 / embeddings.cpp

ggml implementation of embedding models including SentenceTransformer and BGE
MIT License
52 stars 3 forks source link

tokenizer.json seems ignored in the model conversion #3

Open WayneCao opened 9 months ago

WayneCao commented 9 months ago

In convert-to-ggml.py line 34, tokenizer.json seems ignored when convert model to ggml