issues
search
openvinotoolkit
/
openvino_tokenizers
OpenVINO Tokenizers extension
Apache License 2.0
18
stars
16
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Port from Releases/2024/4 to Master update
#251
apaniukov
opened
16 hours ago
0
Port Build Info From Master
#250
apaniukov
closed
1 day ago
0
Port from Releases/2024/4 to master
#249
apaniukov
closed
16 hours ago
1
Fixed BUILD_FAST_TOKENIZERS condition
#248
mryzhov
closed
19 hours ago
0
Building Without FastTokenizer Info
#247
apaniukov
closed
4 days ago
0
Port changes from the Releases/2024/4
#246
andrei-kochin
closed
1 day ago
1
Checked Android RISCV64 and adjusted cmake messages
#245
ilya-lavrenov
closed
4 days ago
0
[2024.4] OV RC1
#244
mryzhov
opened
4 days ago
1
[2024.4] OV RC1
#243
mryzhov
closed
4 days ago
0
Add UTF-8 validation
#242
pavel-esir
opened
4 days ago
0
[GHA] Update OV refs to 2024.4
#241
akladiev
closed
4 days ago
1
Fixed errors during Android build
#240
ilya-lavrenov
closed
4 days ago
0
[CMAKE] Don't theow on version check if tokenizers is submodule
#239
ilya-lavrenov
closed
4 days ago
0
Bump product version to 2024.5
#238
akladiev
closed
5 days ago
0
Enabled build w/o FastTokenizers
#237
ilya-lavrenov
closed
6 days ago
0
[Cmake] Usage of explicit openvino::threading
#236
ilya-lavrenov
closed
6 days ago
0
Switch To BPE Backend
#235
apaniukov
closed
4 days ago
0
cmake 3.14
#234
mryzhov
opened
1 week ago
0
Fix infinite loop in split for empty split_pattern
#233
pavel-esir
closed
1 week ago
0
[TF FE][Tokenizers] Optimize TF FE extensions
#232
rkazants
closed
1 week ago
0
Fix Segfault on Mac
#231
pavel-esir
closed
1 week ago
1
Use OV threading backend instead of direct TBB
#230
ilya-lavrenov
closed
1 week ago
0
Ensure that oneTBB is found
#229
ilya-lavrenov
closed
1 week ago
0
[Do not merge] Fix MacOS build
#228
rkazants
closed
2 weeks ago
0
[TF FE][Tokenizers] Avoid dependency from TF FE in tokenizers
#227
rkazants
closed
1 week ago
0
Revert "Revert "Ops Optimization (#219)" (#222)"
#226
apaniukov
closed
2 weeks ago
0
Add Truncation To Sentencepiece
#225
apaniukov
closed
1 week ago
0
Is it necessary to include openvino_tokenizers in the production environment?
#224
LNTH
closed
2 weeks ago
3
Fix segfault in PCRE2
#223
pavel-esir
closed
2 weeks ago
0
Revert "Ops Optimization"
#222
akladiev
closed
2 weeks ago
1
Use setupvars.ps1 on Windows
#221
ilya-lavrenov
closed
3 weeks ago
0
reimplement BPE tokenizer
#220
pavel-esir
closed
1 week ago
0
Ops Optimization
#219
apaniukov
closed
2 weeks ago
0
Win debug build
#218
mryzhov
closed
4 days ago
1
Conda forge fixes for 2024.3 (#213)
#217
ilya-lavrenov
closed
4 weeks ago
0
[CI] Use ov nightly builds
#216
mryzhov
opened
4 weeks ago
0
Update Data Type For Windows
#215
apaniukov
closed
3 weeks ago
4
[2024.3.0] Release conda fix
#214
mryzhov
closed
4 weeks ago
1
Conda forge fixes for 2024.3
#213
ilya-lavrenov
closed
4 weeks ago
0
Set minimal supported OpenVINO version
#212
ilya-lavrenov
closed
4 weeks ago
0
Avoid installation of PRCE2 artifacts together with OpenVINO Tokenizers
#211
ilya-lavrenov
closed
1 month ago
0
Update gitignore
#210
as-suvorov
closed
4 weeks ago
0
ICU debug build
#209
mryzhov
closed
1 day ago
0
Split implementation without FastTokenizer (to master)
#208
pavel-esir
closed
1 month ago
1
Preserve FETCHCONTENT_BASE_DIR
#207
Wovchena
closed
4 weeks ago
1
Split implementation without FastTokenizer
#206
pavel-esir
closed
1 month ago
1
Align Sentencepiece Model Vocab
#205
apaniukov
closed
1 month ago
0
Port #195
#204
Wovchena
closed
1 month ago
2
port 'use PCRE2 fallback for RegexNormalization' to release
#203
pavel-esir
closed
1 month ago
0
Support GLM-4 Tokenizer
#202
apaniukov
closed
1 month ago
0
Next