Closed vahidEtt closed 2 years ago
what GPU are you using? I don't get that on a K80
I am getting the same error as @vahidEtt, and at the same code location as well. I am running it on Titan RTX (24GB), Driver Version: 460.91.03, CUDA Version: 11.2.
Sorry about the issues, we've been using Titan RTX 24GB so it's quite weird.
Yeah reducing the number of threads or adding launch_bounds will fix it this issue most likely. We have to pinpoint which one is causing trouble. However from your screenshot it's not super clear which kernel is at fault; it seems it's not one of the main ones. Maybe it's dilate_kernel
?
What data are you using here?
I just pre-emptively added launch_bounds in a bunch of places, not sure if that will fix it
It's only happened when I ran it on the RTX 2080Ti. It was ok on the GTX 1080Ti.
I used my own data and dataloader.
OK. I will test it. Thanks!
after reinstalling, It fixed the issue! Thanks!
Indeed, adding launch_bounds seems to have fixed the issue. Thanks!
Any idea how this solves this issue? it looks related to the CUDA_NUM_THREADS. Can we set in the build?![image](https://user-images.githubusercontent.com/75507961/145660897-d8208363-4b31-4c27-9c4b-66a209da048b.png)