search

openvinotoolkit / openvino.genai

Run Generative AI models using native OpenVINO C++ API
Apache License 2.0
113 stars 151 forks source link

Reuse Tokenizer #558

Closed Wovchena closed 3 months ago

Wovchena commented 3 months ago

It's now required to build the whole project to get continuous batching. Continuous batching is going to be merged into the main library over time.