-
Thank you for developing this!
## Context
Due to lenghty computation time and in order to speed things up, I thought about using the `flash_attention_2` and smaller floating points `torch.float16`…
-
Trace -
` File "/home/ec2-user/SageMaker/mistral-finetune-unsloth/multi-run-compare/run_model_qwen.py", line 1, in
from unsloth import FastLanguageModel
File "/home/ec2-user/anaconda3/en…
-
**The bug**
It seems that many models loaded with `models.Transformers()` error out with:
`AssertionError: The passed tokenizer does have a byte_decoder property and using a standard gpt2 byte_d…
-
### System Info
transformers==4.45.2
### Who can help?
@ArthurZucker
### Information
- [ ] The official example scripts
- [X] My own modified scripts
### Tasks
- [ ] An officially supported tas…
-
### Checklist
- [ ] The issue exists after disabling all extensions
- [X] The issue exists on a clean installation of webui
- [ ] The issue is caused by an extension, but I believe it is caused by a …
-
The code from https://huggingface.co/Salesforce/codegen25-7b-multi_P#causal-sampling-code-autocompletion and https://github.com/salesforce/CodeGen/tree/main/codegen25#sampling does not work currently.…
-
I get this error:
```
Traceback (most recent call last):
File "/home/denis/Documents/ai/unsloth/llama3-chat-template.py", line 20, in
model, tokenizer = FastLanguageModel.from_pretrained(…
-
### System Info
ubuntu 22.04
torch 2.5.0
cuda 12.4
running on a single gpu with CUDA_VISIBLE_DEVICES=1
…
-
### System Info
not relevant here
### Who can help?
@stevhliu
### Information
- [ ] The official example scripts
- [ ] My own modified scripts
### Tasks
- [ ] An officially suppo…
-
hellow, I failed to covert trt-llm Llama3.2 3B when I tried to run convert_checkpoint.py.
(like this link - https://github.com/NVIDIA/TensorRT-LLM/issues/2339)
I want to know if Llama3.2 3B model con…