issues
search
google
/
sentencepiece
Unsupervised text tokenizer for Neural Network-based text generation.
Apache License 2.0
10.32k
stars
1.18k
forks
source link
issues
Newest
Newest
Most commented
Recently updated
Oldest
Least commented
Least recently updated
Bump the github-actions group across 1 directory with 6 updates
#1019
dependabot[bot]
closed
5 months ago
1
resume/restart training of tokenizer
#1018
ganeshkrishnan1
closed
5 months ago
3
I want to obtain a model file using my vocab!
#1017
scj0709
closed
6 months ago
1
Convert SentencePiece .vocab format to OpenNMT-py .onmt_vocab format
#1016
HURIMOZ
closed
6 months ago
1
fixing minor typos in the API.md
#1015
Cassini-chris
closed
6 months ago
0
debloat the cmakelists.txt and add a bunch of customization for building
#1014
alexlnkp
opened
6 months ago
0
bump CMake minimum required version to avoid warnings
#1013
alexlnkp
closed
6 months ago
2
adding vocab_size consistency
#1012
Cassini-chris
closed
6 months ago
0
Bump requests from 2.31.0 to 2.32.0 in /.github/workflows/requirements in the pip group
#1011
dependabot[bot]
closed
6 months ago
0
Runtime error on iOS
#1010
l3utterfly
closed
3 months ago
11
Tokenization for phonetic languages
#1009
divyeshrajpura4114
closed
6 months ago
3
make it more friendly for mingw enviroments
#1008
Kreijstal
closed
6 months ago
1
Fix typo
#1007
xu-song
closed
6 months ago
0
Build sentencepiece with mingw
#1006
Kreijstal
closed
6 months ago
1
Fixing issues with the normalizer.cc (typo, type safety, cast fucn)
#1005
Cassini-chris
closed
6 months ago
0
Bump setuptools from 69.2.0 to 69.5.1 in /.github/workflows/requirements in the build-time-deps group
#1004
dependabot[bot]
closed
6 months ago
0
Bump the github-actions group with 6 updates
#1003
dependabot[bot]
closed
6 months ago
1
Add missing output formats to spm_encode flag documentation
#1002
mcognetta
closed
6 months ago
0
Tokenize at the word level without spacers nor joiners
#1001
HURIMOZ
closed
6 months ago
2
No make file found while build and install the Python wrapper
#1000
NickStrain
closed
7 months ago
2
Treat Hawaiian Glottal stop as consonant, not punctuation
#999
HURIMOZ
closed
7 months ago
4
Bump idna from 3.6 to 3.7 in /.github/workflows/requirements
#998
dependabot[bot]
closed
7 months ago
0
Is GGUF supported?
#997
micheledellaguardia
closed
7 months ago
1
Bump the github-actions group with 6 updates
#996
dependabot[bot]
closed
7 months ago
1
Bump the build-time-deps group in /.github/workflows/requirements with 3 updates
#995
dependabot[bot]
closed
7 months ago
0
Support for Windows Python 3.12.2
#994
Nick-
closed
8 months ago
0
Error when running this command: pip install 'transformers[tf-cpu]' on mac
#993
ambadumbuya
closed
8 months ago
1
Inconsistent result between py and cpp
#992
Lewis-Lu
closed
8 months ago
1
Any api for setting user defined symbols?
#991
zhangyuhanjc
closed
8 months ago
1
Windows pip Dependancy Installation Error
#990
Nick-
closed
8 months ago
2
pip subprocess to install build dependencies did not run successfully. │ exit code: 1
#989
Anubiiss
closed
3 months ago
2
Only Pretokenization
#988
SeverinoDaDalt
closed
8 months ago
3
coredump when build with CXXFLAGS `-Wp,-D_GLIBCXX_ASSERTIONS`
#987
Henry-ZHR
opened
8 months ago
0
Bump the github-actions group with 4 updates
#986
dependabot[bot]
closed
8 months ago
1
Bump the build-time-deps group in /.github/workflows/requirements with 3 updates
#985
dependabot[bot]
closed
9 months ago
0
Allow whitespace-only pieces
#984
bauwenst
opened
9 months ago
0
TSV for NFC normalization
#983
JaumePrats
closed
9 months ago
1
Sequence of byte '<0x09>' as token
#982
SeverinoDaDalt
closed
9 months ago
1
Bump cryptography from 42.0.2 to 42.0.4 in /.github/workflows/requirements
#981
dependabot[bot]
closed
9 months ago
0
HELP NEEDED Mask Token in SentencePiece tokenizer HELP NEEDED
#980
debrupf2946
closed
9 months ago
1
move setting of default CMAKE_INSTALL_{BIN,INCLUDE,LIB}DIR before first use
#979
h-vetinari
closed
9 months ago
0
entry points return non-zero exit code (at least for `--help`)
#978
h-vetinari
closed
9 months ago
2
Many tests fail
#977
yurivict
closed
9 months ago
2
error while installing sentencepiece python 3.12.2
#976
mistrytejasm
closed
9 months ago
2
Bump cryptography from 42.0.0 to 42.0.2 in /.github/workflows/requirements
#975
dependabot[bot]
closed
9 months ago
0
Fix a typo in api.md
#974
xunkai55
closed
9 months ago
1
Not found google.protobuf packages
#973
CharlinChen
closed
9 months ago
1
Bump cryptography from 41.0.7 to 42.0.0 in /.github/workflows/requirements
#972
dependabot[bot]
closed
9 months ago
0
Getting requirements to build wheel did not run successfully.
#971
sapatmohit
closed
9 months ago
9
Bump the build-time-deps group in /.github/workflows/requirements with 1 update
#970
dependabot[bot]
closed
9 months ago
0
Previous
Next