-
`llama_model_load: error loading model: error loading model vocabulary: unknown pre-tokenizer type: 'qwen2' llama_load_model_from_file: failed to load model
`
I tried to load this model using llam…
-
The tokenization structure is different from languages like English or other languages that start from left to right, and those language like Arabic and langues that start from right to left. Addeding…
-
### System Info
peft=0.11.1
python=3.10
### Who can help?
When I run this script, there is no problem with a single GPU. When I try to run 2 GPUs, the system resources show that the utilizat…
-
I have a couple of suggestions for the tokenizer API -- things that I have needed to work around here: https://github.com/ml-explore/mlx-swift-examples/blob/main/Libraries/LLM/Tokenizer.swift
- add…
-
msh> < a
Error: Core dump in child
-
As mentioned in title, running the code under the section "Connect Tokenizer to a Model" gives me the error
Can't set the input tensor with index: 0, because the model input (shape=[?]) and the tens…
gnool updated
5 hours ago
-
When I'm trying to add some tokens in vocab, there's 3 issue in `Fast` type tokenizers; there's no problem in python tokenizer, though.
- Spacebar before additional token would be deleted if added …
-
Hello,
`TSQLStatementReader.ParseStatements` throws ArgumentOutOfRangeException exception when there is a string literal immediately after SELECT keyword like `SELECT' '+item`
Error Details:
…
-
while using the load_model function, I encountered an issue exporting the operator 'FileNotFoundError: [Errno 2] No such file or directory: 'C:\\Users\\v_ztfang\\.cache\\huggingface\\hub\\models--bert…
-
If I use distilbert-base-uncased model
trans_version 4.40
It will have max_length 1000000000000000019884624838656
in the utils.py line 216
DistilBertTokenizer(name_or_path='distilbert-base-uncas…