-
Integrate intel/Habana HPU kernel support for uint4 inference on Habana HPU. This was merged into AutoGPTQ https://github.com/AutoGPTQ/AutoGPTQ/pull/689/files but there are no ci tests and we have no …
-
-
### System Info
```shell
optimum-habana==1.11.1
Gaudi 2 on Intel Developer Cloud
image: vault.habana.ai/gaudi-docker/1.15.1/ubuntu22.04/habanalabs/pytorch-installer-2.2.0:latest
```
### Informati…
-
Model: [mistralai/Mixtral-8x7B-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1)
Deployed with single card, it will report OOM error:
> (ServeController pid=207518) F…
-
follow the instructions on
https://github.com/HabanaAI/Model-References/tree/master/MLPERF3.1/Training/benchmarks
to execute comamnd:
`python3 pack_pretraining_data_pytorch.py --input_dir=$PYT…
-
-
InstructLab 0.13 supports hardware acceleration for Apple Silicon (via `mlx`) and CUDA-like GPUs (NVIDIA CUDA and AMD ROCm via `torch.cuda`). I would like to add support for Intel Gaudi 2 hardware and…
tiran updated
2 months ago
-
https://github.com/huggingface/optimum-habana/blob/eaac913c56617a4deefa99548d89366722b7397e/optimum/habana/accelerate/accelerator.py#L250
As the src code above, optimum-habana impl a subclass of Ac…
-
### System Info
```shell
optimum-habana v1.11.0.dev0
deepspeed v1.11.0
vault.habana.ai/gaudi-docker/1.14.0/ubuntu22.04/habanalabs/pytorch-installer-2.1.1:latest
| HL-SMI Version: …
-
### Feature request
I see the release version 1.12 has supported fp8, but I didn't see any example code for how to train LLM by using FP8.
How can I use FP8 to train model?
### Motivation
I want t…