openvinotoolkit / model_server

A scalable inference server for models optimized with OpenVINOâ„¢
https://docs.openvino.ai/2024/ovms_what_is_openvino_model_server.html
Apache License 2.0
679 stars 212 forks source link

change llm tokenizer to drop extra special tokens #2806

Closed dtrawins closed 3 weeks ago

dtrawins commented 3 weeks ago

🛠 Summary

CVS-156933 detokenizer is now adding special tokens by default hence additional flag is needed in export command

🧪 Checklist