huggingface / optimum-habana

Easy and lightning fast training of 🤗 Transformers on Habana Gaudi processor (HPU)
Apache License 2.0
154 stars 202 forks source link

Add option to use bf16 in PT sdp (#5) #1514

Open astachowiczhabana opened 4 days ago

astachowiczhabana commented 4 days ago

The new option --sdp_on_bf16 allows pyTorch to use reduced precision in sdp in the math backend. The change affects the following examples:

astachowiczhabana commented 4 days ago

Hi @libinta this commit is also required with next OH release

jiminha commented 19 hours ago

@astachowiczhabana which model and configuration are you running with this flag enabled? We'd like to do the same for the OH CI/PYTEST.

hsubramony commented 18 hours ago

@astachowiczhabana dont we need to update README for all the models with this option ?

ugolowic commented 4 hours ago

@jiminha

Inference - 1 card:

Training - 8 cards: