search

lilanxiao / Rotated_IoU

Differentiable IoU of rotated bounding boxes using Pytorch
MIT License
416 stars 64 forks source link

Sorting vertices is gives wrong result on different gpu #30

Closed alexzhou907 closed 2 years ago

alexzhou907 commented 3 years ago

I have a 4 gpu server and I use the following code to test cal_iou function

    device = 'cuda:0'
    box3 = [-0.3541, -0.5818,  0.6220,  1.2440,  2.1768]
    box4 = [-0.4219, -0.1952,  0.3413,  0.6827,  0.8751]
    tensor1 = torch.FloatTensor(box3).to(device)[None,None] # 2, 4, 2 # test same box
    tensor2 = torch.FloatTensor(box4).to(device)[None,None]

    area = cal_iou(tensor1, tensor2)[0]

    print("CUDA: ")
    print(area)

When device is 'cuda:0' it seems to work fine, but when it is 'cuda:1' or 'cuda:2' or 'cuda:3', area gives 0 always. What is wrong?

lilanxiao commented 3 years ago

Hi, thank you very much for the issue!

I've never tested my code with multi-GPUs. I'll investigate this issue on a server and update this ASAP.

lilanxiao commented 3 years ago

Hi, the CUDA kernel seems to need a guard. I'm still not sure what is going on here, but adding the guard fixed this issue on my server :D

More info: https://discuss.pytorch.org/t/c-cuda-extension-with-multiple-gpus/91241

Please pull the new commit and verify if it works on your machine. Remember to recompile the CUDA extension.