-
### Your current environment
**When I send the cl:**
```
python -m vllm.entrypoints.openai.api_server --model Qwen/Qwen2-1.5B-Instruct --served-model-name Qwen2-7B-Instruct-lora --max-model-len=204…
-
### System Info
vllm version: 0.4.1
### Who can help?
_No response_
### Information
- [ ] The official example scripts
- [ ] My own modified scripts
### Tasks
- [ ] An officially supported task…
-
**Describe the bug**
I am trying to finetune Mistral Instruct V0.2 and I keep getting this error.
`TypeError: LoraConfig.__init__() got an unexpected keyword argument 'use_rslora`
`
**To Reproduc…
-
### Question Validation
- [X] I have searched both the documentation and discord for an answer.
### Question
my gpu is 24G memory,so, its possiable to spread models to 2 GPUs。
torch.cuda.OutOf…
OKC13 updated
4 months ago
-
## 📚 Documentation
### Suggestion
### Bug
- Link to the buggy documentation/tutorial:
- Description of the bug:
-
### System Info
```
- `transformers` version: 4.29.2
- Platform: macOS-13.4-x86_64-i386-64bit
- Python version: 3.10.2
- Huggingface_hub version: 0.14.1
- Safetensors version: not installed
- P…
-
The example (https://github.com/Lightning-AI/litgpt?tab=readme-ov-file#continue-pretraining-an-llm) works fine on my machine but as soon as i replace with custom text files that each just contain one …
-
### Your current environment
```
Collecting environment information...
PyTorch version: 2.0.1+cu117
Is debug build: False
CUDA used to build PyTorch: 11.7
ROCM used to build PyTorch: N/A
OS…
-
### System Info
```shell
Platform:
- Platform: Linux-5.15.0-1056-aws-x86_64-with-glibc2.29
- Python version: 3.8.10
Python packages:
- `optimum-neuron` version: 0.0.23
- `neuron-sdk` …
-
### System Info
`transformers` version: 4.32.0
- Platform: Linux-5.19.0-38-generic-x86_64-with-glibc2.35
- Python version: 3.10.9
- Huggingface_hub version: 0.16.4
- Safetensors version: 0.3.1
…