-
I am getting maximum recursion depth error after running this following command:
python qlora.py --model_name_or_path decapoda-research/llama-7b-hf
And this is the error I got:
File "/home/at…
-
unable load Tokenizer using AutoTokenizer.from_pretrained()
errors:
tokenizer = AutoTokenizer.from_pretrained(model_id)
File "/home/ubuntu/venv/lib/python3.10/site-packages/transformers/mod…
-
``` haskell
>>> Tokens.toWords "I am the very model of a modern major general.."
["I", "am", "the", "very", "model", "of", "a", "modern", "major", "general", ".."]
```
The following text, as mentione…
-
![image](https://github.com/yuanzhoulvpi2017/zero_nlp/assets/91042213/4e681244-224e-4473-8be7-61bc9f995f81)
在将模型保存下来后然后重新读取的时候,读取预处理器会报错。下边是错误内容,transformers版本为4.40.1, torch版本为2.0.1+cuda11.8, 设备为移动端R…
-
### What happened?
The llama.cpp tokenizer for Phi-3 has odd behavior, where re-tokenizing the same text over and over keeps adding whitespaces to the first non-BOS token. This has several issues:
…
-
### Describe the bug
Mapped tokenization slows down substantially towards end of dataset.
train set started off very slow, caught up to 20k then tapered off til the end.
what's particularly s…
-
When I run this script-scripts/llama3/train/stage_2_full_v8b_672_hr_1536.sh, I encounter this error- WARNING: tokenization mismatch: 156 vs. 161. (ignored)
-
This test failed!
To configure my behavior, see [the Flaky Bot documentation](https://github.com/googleapis/repo-automation-bots/tree/main/packages/flakybot).
If I'm commenting on this issue too oft…
-
1. 我已搜索相关问题,但无法获得预期的帮助。
**描述错误**
运行默认工作流,弹出如下错误:
相关目录已经有相关的文件但是任然提示找不到。
!!! Exception during processing!!! Unable to load vocabulary from file. Please check that the provided vocabulary is…
-
Hi
can you please review my solution for this bug:
https://sourceforge.net/p/snuggletex/bugs/9/
It's done in this commit in `LaTeXTokenizer`:
https://github.com/axkr/symja_android_library/com…
axkr updated
11 months ago