Update rt_info - Githubissues

openvinotoolkit / openvino_tokenizers

OpenVINO Tokenizers extension

Apache License 2.0

17 stars 13 forks source link

Update rt_info #191

Closed apaniukov closed 1 week ago

apaniukov commented 1 week ago

Add BOS and PAD tokens ids
Add chat template
Add original tokenizer class name

CVS-144174