-
I looked into the source code and found that the class `Sampler` discards the prefix in `last_hidden_states`.
```Python
class Sampler(nn.Module):
"""Samples the next tokens from the model's o…
-
Hello,
I've successfully finetuned Llama-3 8B with QDoRA and am now looking to perform inference using vLLM. Could you provide guidance or scripts on how to merge the QDoRA adapters with the origin…
-
```
models = [
enc_dec_models[0], # bart-large-cnn
dec_models[0], # Mistral-7B-Instruct-v0.3
gpts[0] # gpt-4o-mini
]
```
-
### Reminder
- [X] I have read the README and searched the existing issues.
### System Info
## QWEN2-1.5B(0.5B)
正常
## QWEN2-7B(MoE)
需要使用bf16 #4278
正常
## QWEN2-72B
正常,有一点点问题,只能在8卡上启动(s…
-
### 🚀 The feature, motivation and pitch
Hi all, I was wondering if it's possible to do precise model device placement. For example, I would like to place the vLLM model on GPU 1 and let GPU 0 do othe…
-
-
### System Info
- `transformers` version: 4.45.0.dev0
- Platform: Linux-5.15.0-78-generic-x86_64-with-glibc2.35
- Python version: 3.11.9
- Huggingface_hub version: 0.24.6
- Safetensors version: 0…
-
I did a DPO fine-tuning using the default MP command provided [here](https://github.com/modelscope/ms-swift/blob/main/docs/source_en/Multi-Modal/human-preference-alignment-training-documentation.md#dp…
-
Anh ơi em đọc readme.md thì hơi rối ạ aanh có thể hướng dẫn chi tiết cách cài llma cho python được không ạ?
-
Hi Team,
I have successfully finetuned a QLoRA adapter on a custom dataset. When I try to load it in full precision, it gets loaded and works well
But this takes too much time and GPU memory to …