-
### Your current environment
Command line:
```
cd vllm-fork/benchmarks
python benchmark_latency.py \
--model meta-llama/Meta-Llama-3-8B \
--dtype bfloat16 \
--output-len 128 \
…
-
### System Info
Hi there, I met a bug that when using TGI Gaudi 2.0.5 with both meta-llama/Meta-Llama-3-8B-Instruct and Intel/neural-chat-7b-v3-3. When I set the default frequency/repetition/presen…
-
### System Info
Optimum Habana: 1.10.4
Synapse: 1.14.0
Dockerfile:
```
FROM vault.habana.ai/gaudi-docker/1.14.0/ubuntu22.04/habanalabs/pytorch-installer-2.1.1:latest
# Installs pdsh and upg…
-
### System Info
```shell
Image: vault.habana.ai/gaudi-docker/1.17.1/ubuntu22.04/habanalabs/pytorch-installer-2.3.1:latest
harware: Habana Labs Gaudi HL205 Mezzanine Card with HL-2000 AI Training …
-
Fromager uses variants to build wheels for multiple GPU architectures. The large majority of Python packages are platform antagonistic (`py3-none-any` wheels) or just CPU, OS, and Python version speci…
tiran updated
2 months ago
-
### System Info
```shell
HL-SMI Version: hl-1.17.0-fw-51.3.0
Driver Version: 1.17.0-28a11ca
Docker image: vault.habana.ai/gaudi-docker/1.17.0/ubuntu22.04/habanalabs/pytorch-installer-2.3.…
-
The container file https://github.com/containers/ai-lab-recipes/blob/main/training/intel-bootc/Containerfile does not contain necessary bits and pieces to setup InfiniBand Intel Gaudi devices. Without…
tiran updated
4 months ago
-
### System Info
```shell
Transformers fails with the following error, when trying to use AWQ with TGI / neural compression enginer, or optimum habana
ValueError: AWQ is only available on GPU
```
#…
-
Xeon and Gaudi setup instructions are scattered throughout the guides. We need to keep the project and the product part separate. This will create space for other hardware vendors to add their product…
-
### System Info
Which image should I use on Macbook Pro? I can't find arm64 image. Please see the below error I'm having:
```
1 warning found (use docker --debug to expand):
- InvalidBaseImageP…