search

bytedance / neurst

Neural end-to-end Speech Translation Toolkit
Other
298 stars 45 forks source link

Easy to raise 'NaN' #65

Open ghtwht opened 1 year ago

ghtwht commented 1 year ago

Its easy to raise 'nan' error when training the translation model with 'transformer_base', have you ever encounter with this problem and how did you deal with it?

zhaocq-nlp commented 1 year ago

Counld you try with --dtype float32 to disable mix-precision training?

ghtwht commented 1 year ago

dtype is default to be float32