-
Traceback (most recent call last):
File "D:\sentencepiece_chinese_bpe-main\chinese_bpe.py", line 23, in
tokenizer = ChineseTokenizer(vocab_file=output_dir + 'chinese.model')
File "D:\sente…
-
Descriptors cannot be created directly. If this call came from a pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0. If you cannot immediately regenerate you…
-
Run-classifier.py was running fine. Till I pretrained model and tried to use a checkpoint for fine tuning.
python3 run_classifier.py --use_tpu=True --tpu=$TPU_NAME --do_train=False --do_e…
-
Sentencepiece has no typing information, which makes it hard to work with. For example:
-
git clone https://github.com/baidu/Senta.git
cd Senta
python -m pip install .
报错的是:
ERROR: Command errored out with exit status 1:
command: /Users/cooper/opt/anaconda3/bin/python -c 'import …
-
feature request:
Sentencepiece is the tokenizer used in XLNet.
I think if `Language` tokenize text with `sentencepiece`, the alignment process can be skipped and it make model efficient.
-
### Bug description
Possibly this is a feature and not a bug.
Sometimes there's a conflict in
- `--dim-vocabs` and
- `--sentencepiece-options "--character_coverage=1.0" `
When the sent…
-
I expect people will use the newest [Python 3.13](https://docs.python.org/3/whatsnew/3.13.html) and try to install mflux and fail. Sharing the latest compat info that I gathered.
# Known blockers
…
-
Encountered this error while running this cell
from transformers import DonutProcessor, VisionEncoderDecoderModel, BartConfig
processor = DonutProcessor.from_pretrained("naver-clova-ix/donut-bas…
-
**_tl;dr_** - espnet isn't made to be built on python 3.11 which is what standalone ComfyUI is packaged with, which causes installation of the plugin to fail.
Hey there, I realize you're probably s…