search

huggingface / optimum-habana

Easy and lightning fast training of 🤗 Transformers on Habana Gaudi processor (HPU)
Apache License 2.0
153 stars 202 forks source link

[SW-209062] Disable default sdpa in Albert (#22) #1517

Open astachowiczhabana opened 11 hours ago

astachowiczhabana commented 11 hours ago

Transformers v4.45 introduced sdpa as the default implementation in Albet. This caused performance drop. Adding Albert to the list of models which don't yet have sdpa implementation in Gaudi and use thus eager attention.

astachowiczhabana commented 11 hours ago

Hi @libinta this commit is also required with next OH release