Mixed precision does not work

ibrahimethemhamamci / CT-CLIP

Developing Generalist Foundation Models from a Multimodal Dataset for 3D Computed Tomography

183 stars 17 forks source link

Mixed precision does not work #18

Open wangxiaoyuwangdayu opened 4 months ago

wangxiaoyuwangdayu commented 4 months ago

I printed the tensor dtype when training, and they were all float32. It seems that mixed precision does not work.

sezginerr commented 4 months ago

Yes, mixed precision does not work with the code. I also have some experiences manually applying it as well, but training diverges. So I would recommend not to use it.