I think I misunderstood the purpose of the "Fallback train data type". This isnt surprising, since a google search for OneTrainer "Fallback train data type" returns NO official documentation on the value.
My tentative understanding now, is that it is usually best to leave it at "float32", even if your main data type is bf16?
If so, it would be really nice to explicitly say in the mouseover, "it is usually best to leave this as float32" Although I'm unclear why it would ever be good to set it otherwise?
What would you like to see as a solution?
Clearer mouse-over docs, or better still explicit documentation in the wiki about it.
Describe your use-case.
I think I misunderstood the purpose of the "Fallback train data type". This isnt surprising, since a google search for
OneTrainer "Fallback train data type"returns NO official documentation on the value.My tentative understanding now, is that it is usually best to leave it at "float32", even if your main data type is bf16?
If so, it would be really nice to explicitly say in the mouseover, "it is usually best to leave this as float32" Although I'm unclear why it would ever be good to set it otherwise?
What would you like to see as a solution?
Clearer mouse-over docs, or better still explicit documentation in the wiki about it.
Have you considered alternatives? List them here.
No response