search

huggingface / optimum-habana

Easy and lightning fast training of 🤗 Transformers on Habana Gaudi processor (HPU)
Apache License 2.0
148 stars 187 forks source link

Is there example of FP8 train LLM, pre-train or fine-tune #1073

Open harborn opened 3 months ago

harborn commented 3 months ago

Feature request

I see the release version 1.12 has supported fp8, but I didn't see any example code for how to train LLM by using FP8. How can I use FP8 to train model?

Motivation

I want to train model by using FP8.

Your contribution

NO.

regisss commented 3 months ago

v1.12 of Optimum Habana will actually be released in a few days. Fp8 training is in a very early stage at the moment, that's why we have not added an example yet. It will get much better support in the following weeks and months!

For now, if you want to do it, you can check how it is done in this test that is part of our CI: https://github.com/huggingface/optimum-habana/blob/main/tests/test_fp8_examples.py