search

buoyancy99 / diffusion-forcing

code for "Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion"
Other
622 stars 29 forks source link

The loss of Robot Imitation Learning is NaN. #21

Closed le-wei closed 2 months ago

le-wei commented 2 months ago

"Hello, thank you very much for your work. I achieved good results when training using the robot_swap dataset, but when I trained with our own collected dataset, the model didn't function correctly, and the loss became NaN. I identified that the problem occurs after executing block1 in the downsampling stage (self.downs) of the Unet, where x turns into NaN. I'm not sure what could be causing this issue. What should I pay attention to when creating the dataset, or how can I verify if my dataset is correct? Thank you for your guidance."

le-wei commented 2 months ago

I found my mistake. Thank you very much!