-
### Is there an existing issue for this?
- [X] I have searched the existing issues and checked the recent builds/commits
### What happened?
Had to reinstall webui cause git pull was not updating ev…
-
Hello, could you share information about the hardware you are using?I couldn't find it in the paper.
-
```python
import torch
from lightning_attn.ops import lightning_attn_func
from lightning_attn.utils import _build_slope_tensor
dtype = torch.bfloat16
device = torch.device("cuda")
b, h, …
-
Recently made public:
https://openai.com/blog/whisper/
https://github.com/openai/whisper
Interesting, they have some multilingual models that can be used for multiple languages without fine tunin…
-
rtx-4090多卡推理(模型为qlora微调后qwen72b)是否支持?通过FSDP+QLoRA,可以正常对qwen-72b的模型进行微调,想问一下,如何使用rxt-4090对其进行推理部署呢?
我尝试使用如下的脚本进行多卡推理:
```
CUDA_VISIBLE_DEVICES=4,5,6,7 accelerate launch --config_file fsdp_config.y…
-
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 192.00 MiB (GPU 0; 8.00 GiB total capacity; 14.21 GiB already allocated; 0 bytes free; 14.53 GiB reserved in total by PyTorch) If res…
-
I followed the conda installation instructions in the README:
```
conda create --name unsloth_env python=3.10
conda activate unsloth_env
conda install pytorch cudatoolkit torchvision torchaudio …
-
## 🐛 Bug
mlc_ai_nightly_cu121-0.15.dev315-cp310-cp310-manylinux_2_28_x86_64.whl
mlc_llm_nightly_cu121-0.1.dev1166-cp310-cp310-manylinux_2_28_x86_64.whl
benchmark with llama3 model, got foll…
-
### What Happened?
**Description of the problem:**
I have successfully setup and launched minikube, however I cannot detect the GPU in my setup.
From the official Minikube tutorials page [here…
-
Hello, can your code run on an RTX 3090?