issues
search
openvinotoolkit
/
openvino_tokenizers
OpenVINO Tokenizers extension
Apache License 2.0
15
stars
12
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Update rt_info
#191
apaniukov
opened
34 minutes ago
0
Update Remaining Inputs Detection During Model Connection
#190
apaniukov
closed
7 hours ago
0
[GHA] Set permissions read-all
#189
mryzhov
closed
3 days ago
0
Error in running code under "Connect Tokenizer to a Model"
#188
gnool
closed
7 hours ago
4
Fix Decoding For Long Tokens
#187
apaniukov
closed
1 week ago
0
Sentencepiece Left Padding
#186
apaniukov
closed
3 days ago
0
Update Sentencepiece Parsing
#185
apaniukov
closed
1 week ago
0
Patch icu external project
#184
mryzhov
closed
1 week ago
0
[CI] Build from OV wheel
#183
mryzhov
closed
3 days ago
0
Revert "Support build from OpenVINO wheel only"
#182
mryzhov
closed
2 weeks ago
0
BPE Fallback for Sentencepiece
#181
apaniukov
closed
2 weeks ago
0
Configure cmake similar to GenAI (#175)
#180
Wovchena
closed
1 week ago
0
[CI] Use OV 2024.2.0 tarballs
#179
mryzhov
closed
2 weeks ago
0
Support build from OpenVINO wheel only
#178
mryzhov
closed
2 weeks ago
0
Runtime Error：Error loading libopenvino_tokenizers.so
#177
bsq1989
opened
2 weeks ago
1
Sentencepiece Tokenization Improvements
#176
apaniukov
closed
2 weeks ago
0
Configure cmake similar to GenAI
#175
Wovchena
closed
2 weeks ago
2
Cherry Pick For Release
#174
apaniukov
closed
3 weeks ago
0
Fix Tokenization of Special Tokens in Sentencepiece
#173
apaniukov
closed
3 weeks ago
0
[CI] Switch 2024.2 branch to pre-release OV tarballs
#172
mryzhov
closed
2 weeks ago
0
Cannot build OpenVINO Tokenizers with `BUILD_FAST_TOKENIZERS` in Clang
#171
zhxie
closed
1 week ago
2
[DNM] test commit
#170
mryzhov
closed
2 weeks ago
0
[CI] Pin ov commit
#169
mryzhov
closed
2 weeks ago
0
Take OV from latest nightly package on master
#168
akladiev
opened
1 month ago
0
Package into correct dirs (#155)
#167
Wovchena
closed
1 month ago
0
New python build scheme
#166
ilya-lavrenov
closed
3 weeks ago
1
Fixed conda-forge on Windows (#164)
#165
ilya-lavrenov
closed
1 month ago
0
Fixed conda-forge on Windows
#164
ilya-lavrenov
closed
1 month ago
0
Build release by default (#162)
#163
ilya-lavrenov
closed
1 month ago
0
Build release by default
#162
ilya-lavrenov
closed
1 month ago
0
Switch default skip tokens flag behavior
#161
slyalin
closed
1 month ago
0
Switch default skip tokens flag behavior
#160
slyalin
closed
1 month ago
2
Fix usage of protobuf_MODULE_COMPATIBLE
#159
ilya-lavrenov
closed
1 month ago
0
Fix usage of protobuf_MODULE_COMPATIBLE
#158
ilya-lavrenov
closed
1 month ago
0
Set cmake policies
#157
ilya-lavrenov
closed
1 month ago
0
Set cmake policies
#156
ilya-lavrenov
closed
1 month ago
0
Package into correct dirs
#155
Wovchena
closed
1 month ago
0
[CI] Changed openvino version 2024.3
#154
akladiev
closed
1 month ago
0
[CI] Temporary removed dependency version of OpenVINO
#153
akladiev
closed
1 month ago
0
Add Left Padding and Padding to Max Length
#152
apaniukov
closed
3 weeks ago
3
Propagate _GLIBCXX_USE_CXX11_ABI
#151
Wovchena
closed
1 month ago
1
Ability to use sentensepiece from conda-forge
#150
ilya-lavrenov
closed
1 month ago
0
trigger
#149
Wovchena
closed
1 month ago
0
Package into correct dirs
#148
Wovchena
closed
1 month ago
5
Revert "revert added tokens support"
#147
apaniukov
closed
1 month ago
0
Revert "Added Tokens Support for BPE Tokenizers"
#146
rkazants
closed
1 month ago
0
Support Punctuation Split
#145
apaniukov
closed
1 month ago
0
Added Tokens Support for BPE Tokenizers
#144
apaniukov
closed
1 month ago
0
Update Version To 2024.1.0.2
#143
apaniukov
closed
1 month ago
0
Fixed tokenizers on Windows
#142
ilya-lavrenov
closed
1 month ago
0
Next