issues
search
google
/
sentencepiece
Unsupervised text tokenizer for Neural Network-based text generation.
Apache License 2.0
10.07k
stars
1.16k
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Add missing output formats to spm_encode flag documentation
#1002
mcognetta
closed
4 months ago
0
Tokenize at the word level without spacers nor joiners
#1001
HURIMOZ
closed
4 months ago
2
No make file found while build and install the Python wrapper
#1000
NickStrain
closed
4 months ago
2
Treat Hawaiian Glottal stop as consonant, not punctuation
#999
HURIMOZ
closed
4 months ago
4
Bump idna from 3.6 to 3.7 in /.github/workflows/requirements
#998
dependabot[bot]
closed
5 months ago
0
Is GGUF supported?
#997
micheledellaguardia
closed
5 months ago
1
Bump the github-actions group with 6 updates
#996
dependabot[bot]
closed
4 months ago
1
Bump the build-time-deps group in /.github/workflows/requirements with 3 updates
#995
dependabot[bot]
closed
5 months ago
0
Support for Windows Python 3.12.2
#994
Nick-
closed
5 months ago
0
Error when running this command: pip install 'transformers[tf-cpu]' on mac
#993
ambadumbuya
closed
5 months ago
1
Inconsistent result between py and cpp
#992
Lewis-Lu
closed
5 months ago
1
Any api for setting user defined symbols?
#991
zhangyuhanjc
closed
5 months ago
1
Windows pip Dependancy Installation Error
#990
Nick-
closed
5 months ago
2
pip subprocess to install build dependencies did not run successfully. │ exit code: 1
#989
Anubiiss
closed
1 month ago
2
Only Pretokenization
#988
SeverinoDaDalt
closed
5 months ago
3
coredump when build with CXXFLAGS `-Wp,-D_GLIBCXX_ASSERTIONS`
#987
Henry-ZHR
opened
5 months ago
0
Bump the github-actions group with 4 updates
#986
dependabot[bot]
closed
5 months ago
1
Bump the build-time-deps group in /.github/workflows/requirements with 3 updates
#985
dependabot[bot]
closed
6 months ago
0
Allow whitespace-only pieces
#984
bauwenst
opened
6 months ago
0
TSV for NFC normalization
#983
JaumePrats
closed
6 months ago
1
Sequence of byte '<0x09>' as token
#982
SeverinoDaDalt
closed
6 months ago
1
Bump cryptography from 42.0.2 to 42.0.4 in /.github/workflows/requirements
#981
dependabot[bot]
closed
6 months ago
0
HELP NEEDED Mask Token in SentencePiece tokenizer HELP NEEDED
#980
debrupf2946
closed
6 months ago
1
move setting of default CMAKE_INSTALL_{BIN,INCLUDE,LIB}DIR before first use
#979
h-vetinari
closed
6 months ago
0
entry points return non-zero exit code (at least for `--help`)
#978
h-vetinari
closed
6 months ago
2
Many tests fail
#977
yurivict
closed
6 months ago
2
error while installing sentencepiece python 3.12.2
#976
mistrytejasm
closed
6 months ago
2
Bump cryptography from 42.0.0 to 42.0.2 in /.github/workflows/requirements
#975
dependabot[bot]
closed
6 months ago
0
Fix a typo in api.md
#974
xunkai55
closed
7 months ago
1
Not found google.protobuf packages
#973
CharlinChen
closed
7 months ago
1
Bump cryptography from 41.0.7 to 42.0.0 in /.github/workflows/requirements
#972
dependabot[bot]
closed
7 months ago
0
Getting requirements to build wheel did not run successfully.
#971
sapatmohit
closed
7 months ago
7
Bump the build-time-deps group in /.github/workflows/requirements with 1 update
#970
dependabot[bot]
closed
7 months ago
0
Bump the github-actions group with 3 updates
#969
dependabot[bot]
closed
6 months ago
1
Error while installing the library "sentence-transformers" which has dependency on "sentencepiece"
#968
AnkitBaliyan1
closed
6 months ago
11
High frequency token segmented into letter sequence when input is a tsv file
#967
TingxunShi
opened
7 months ago
3
coredump when build with CXXFLAG `-Wp,-D_GLIBCXX_ASSERTIONS`
#966
samchugit
closed
6 months ago
4
RuntimeError
#965
fkurushin
closed
7 months ago
1
Merging tokenizers issue
#964
gordicaleksa
closed
7 months ago
4
Official support for Android compilation in Release/Assets
#963
JordiFB
closed
7 months ago
1
Additional external absl fixes
#962
Halmoni100
closed
8 months ago
0
Evaluate Profile-Guided Optimization (PGO)
#961
zamazan4ik
opened
8 months ago
0
Same oov count while using different vocab size
#960
shreyasinghal-17
closed
8 months ago
2
Revert "Bump the github-actions group with 2 updates"
#959
taku910
closed
8 months ago
0
Extract & modify the merge rules from the .model file of a SentencePiece BPE model
#958
kitkhai
closed
8 months ago
1
Bump the github-actions group with 2 updates
#957
dependabot[bot]
closed
8 months ago
0
How to safely extend vocabulary?
#956
ekurtulus
closed
8 months ago
3
Hash-pin Python dependencies in CI/CD release workflows
#955
pnacht
closed
8 months ago
0
Segmentation fault (core dumped)
#954
ivankrylatskoe
closed
6 months ago
2
c++ API compilation problem
#953
wangning7149
closed
8 months ago
1
Previous
Next