-
### Priority
P3-Medium
### OS type
Ubuntu
### Hardware type
Xeon-ICX
### Installation method
- [ ] Pull docker images from hub.docker.com
- [ ] Build docker images from source
…
-
### System Info
```shell
Google Colab (CPU runtime)
```
### Information
- [X] The official example scripts
- [ ] My own modified scripts
### Tasks
- [X] An officially supported task in the `exam…
-
```
cd /root/workspace/github/optimum-habana/examples/text-generation/
python run_generation.py \
--model_name_or_path /root/workspace/model/meta-llama/Llama-3.1-8B/ \
--use_hpu_graphs \
--use_kv…
-
### 🚀 The feature, motivation and pitch
```
INFO 08-26 07:31:47 habana_model_runner.py:1192] [Warmup][Prompt][1/56] batch_size:64 seq_len:1024 free_mem:13.93 GiB
INFO 08-26 07:32:25 habana_model_…
-
### System Info
```shell
Image: vault.habana.ai/gaudi-docker/1.17.1/ubuntu22.04/habanalabs/pytorch-installer-2.3.1:latest
harware: Habana Labs Gaudi HL205 Mezzanine Card with HL-2000 AI Training …
-
### Your current environment
### Environment Details
Running in a Kubernetes environment with Habana Gaudi2 accelerators:
- **Hardware**: Habana Gaudi2 accelerators
- **Deployment**: Kubernetes …
-
### Your current environment
The offline inference of Llama-3-8B with benchmark_latency.py sweeping on 1, 2, 4 cards results:
And the optimum-habana results:
The results show that on 1 card…
-
### System Info
Optimum Habana: 1.10.4
Synapse: 1.14.0
Dockerfile:
```
FROM vault.habana.ai/gaudi-docker/1.14.0/ubuntu22.04/habanalabs/pytorch-installer-2.1.1:latest
# Installs pdsh and upg…
-
### System Info
```shell
HL-SMI Version: hl-1.17.0-fw-51.3.0
Driver Version: 1.17.0-28a11ca
Docker image: vault.habana.ai/gaudi-docker/1.17.0/ubuntu22.04/habanalabs/pytorch-installer-2.3.…
-
### System Info
```shell
optimum 1.21.4
optimum-habana 1.14.0.dev0
transformers 4.45.2
+------------------------------------------------------------------…