-
-
### Priority
Undecided
### OS type
Ubuntu
### Hardware type
Gaudi2
### Installation method
- [X] Pull docker images from hub.docker.com
- [ ] Build docker images from source
### Deploy method
…
-
Currently the TGI have [tgi-gaudi](https://github.com/huggingface/tgi-gaudi) to support the HPU for Gaudi 2 and also VLLM is included their [roadmap in Q1 2024](https://github.com/vllm-project/vllm/is…
-
### Description & Motivation
For below example, model is being compiled, `DDPStrategy` is passed to Trainer, then during fit method `DDPStrategy` is being applied, so forward is compiled but `_pre_…
-
### System Info
```shell
Transformers fails with the following error, when trying to use AWQ with TGI / neural compression enginer, or optimum habana
ValueError: AWQ is only available on GPU
```
#…
-
I have been staging some updates testing the tgi-gaudi software with llama 405B fp8, i am waiting for habana optimum to approve the PR, and then I will submit a pr for huggingface/tgi_gaudi and will s…
-
### Your current environment
```text
PyTorch version: 2.2.0a0+git8964477
Is debug build: False
CUDA used to build PyTorch: None
ROCM used to build PyTorch: N/A
OS: Ubuntu 22.04.4 LTS (x86_64)
…
-
### 🐛 Describe the bug
When using compile mode, i am getting graph breaks due with following errors
"[rank0]:I0702 10:23:29.307000 139872906987520 torch/_dynamo/variables/higher_order_ops.py:468] …
-
### Description
The example pages under each library may contain examples contributed by the Ray Community. We want to demarcate them clearly so that readers understand they are not maintained by the…
-
T2V is planned to enable inferencing LLMs like Stable Diffusion on CPU/GPU, and training on Habana Gaudi/DG2, as well as improving the Generated Video quality, like more realistic frame, and coherency…