search

mosaicml / examples

Fast and flexible reference benchmarks
Apache License 2.0
424 stars 121 forks source link

Change bf16 to amp_bf16 #322

Open Srini-98 opened 1 year ago

Srini-98 commented 1 year ago

Hi,

I tried replicating the pretraining bert script and when I ran it with the yaml script I got the following error: Value bf16 is not available in Precision. I traced the code and changed bf16 to amp_bf16 to get it to work. Not sure if I am using the outdated code but flagging it here.

https://github.com/mosaicml/examples/blob/main/examples/bert/yamls/main/hf-bert-base-uncased.yaml - line 82

jacobfulano commented 1 year ago

Thanks for catching this! You are correct in using the amp_bf16 flag. We are updating the yamls accordingly

dakinggg commented 10 months ago

@jacobfulano was this fixed?

Taytay commented 5 months ago

@jacobfulano : I submitted a PR. Just got bitten by this myself.