-
### System Info
ghcr.io/huggingface/text-generation-inference 2.0.4
platform windows10
Docker version 27.0.3
llm model:lllyasviel/omost-llama-3-8b-4bits
cuda 12.3
gpu nvidia rtx A6000
### Inf…
-
### Your current environment
```text
PyTorch version: 2.3.1+cu121
Is debug build: False
CUDA used to build PyTorch: 12.1
ROCM used to build PyTorch: N/A
OS: Ubuntu 20.04.6 LTS (x86_64)
GCC ve…
-
### Jan version
0.5.3
### Describe the Bug
I imported many models and for some of them, they are failing to load if I selected my both graphic cards (RTX 3060 12Go).
If I unselect one of them, the…
-
## Description
While following https://slideflow.dev/installation/#run-a-docker-container for the torch backend container I encountered a problem with starting the container.
## To Reproduce
`doc…
-
I have a Lenovo Legion Y540 with i7-9750H and GTX1660 Ti. Running latest Arch Linux with Gnome using modified gdm-prime. In BIOS/UEFI I can choose "Switchable Graphcis" or "Discrete Grahpics" (this ma…
-
Hi, I'm trying to setup MPS partitioning on GKE, but I can't get the k8s-device-plugin to work. The plugin gets installed correctly, but it never starts any driver pods.
Cluster data:
- K8s Rev…
-
### Describe the bug
![image](https://github.com/oobabooga/text-generation-webui/assets/96732179/3b7f46e8-d59b-4d56-bfb3-95c7ecd73887)
![image](https://github.com/oobabooga/text-generation-webui/ass…
-
```txt
[03:29:04] WARN - NVML init failed.
[03:29:04] WARN - Failed to initialize NVML. Nvidia GPUs' health information will not display.
```
NBMiner Version: 39.5
OS: Linux - CentOS 7
Driver …
-
### What is the problem?
Today we discovered an issue with a ray deployment on a DGX A100. NVIDIAs new Ampere cards support MIG (multi instance GPUs) where a physical GPU is split into multiple vir…
-
For EESSI, we implemented GPU support for the stack and to access the drivers, it basically requires that someone runs the script https://github.com/EESSI/software-layer/blob/2023.06-software.eessi.io…