sail-sg / MDT

Masked Diffusion Transformer is the SOTA for image synthesis. (ICCV 2023)
Apache License 2.0
500 stars 35 forks source link

CUDA error: device-side assert triggered #12

Open Bhavay-2001 opened 1 year ago

Bhavay-2001 commented 1 year ago

Hi authors. I'm trying to merge your model n setup with a different repo here. I'm trying to solve the errors at test time when I run the classifier_sample_known.py file. It runs for some time and then throws this error- RuntimeError: CUDA error: device-side assert triggered CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1. Compile with TORCH_USE_CUDA_DSA to enable device-side assertions

I have also tried with PyTorch version - 2.1.0.dev20230523+cu117 as I saw some solution online but that didn't work for me. Can you please tell what can be the possible solution? Thanks