-
### System Info
```shell
I was reading that Gaudi supports Llama 3.1 -- https://www.intel.com/content/www/us/en/developer/articles/technical/intel-ai-solutions-support-meta-llama-3-1-launch.html
A…
-
- Gaudi reference installation link is just linked and no version description, which result in compatibility issues.
- Considering using operator to provision and manage Gaudi drivers of clus…
-
### System Info
```shell
Optimum Habana version v1.12.1
Synapse 1.16.2
docker vault.habana.ai/gaudi-docker/1.16.2/ubuntu22.04/habanalabs/pytorch-installer-2.2.2:latest
```
### Information
- [ ] …
-
### Your current environment
```
#!/bin/bash
# Copyright (C) 2024 Intel Corporation
# SPDX-License-Identifier: Apache-2.0
# Set default values
default_port=8008
default_model=$LLM_MODEL
defa…
-
I have been staging some updates testing the tgi-gaudi software with llama 405B fp8, i am waiting for habana optimum to approve the PR, and then I will submit a pr for huggingface/tgi_gaudi and will s…
-
Quick question: when is an update to an optimum-habana version which includes https://github.com/huggingface/optimum-habana/issues/1154 (fix for rope_scaling @ llama3.1 family) planned?
-
### System Info
Use the ghcr.io/huggingface/tgi-gaudi:2.0.1 official docker image.
As shown in https://github.com/huggingface/tgi-gaudi/blob/habana-main/docs/source/supported_models.md?plain=1, l…
-
### System Info
Linux gaudi2-wsf-test 5.15.0-92-generic #102-Ubuntu SMP Wed Jan 10 09:33:48 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
docker image: ghcr.io/huggingface/tgi-gaudi:2.0.0
###…
-
**Is your feature request related to a problem? Please describe.**
I run GEOPM on a Gaudi system but there are telemetry metrics missing.
**Describe the solution you'd like**
Support for gatherin…
-
Hi everyone,
I can see there has been a recent effort to add more documentation on TGI, and I appreciate it. However, there are some sections that are misleading, for example:
**docs/source/concep…