batch issue

[udion]

I think there is something wrong in the forward function of networks as it doesn't supports arbitary batch sizes, it only supports batch size of 1

[crazyleg]

just iterate over images in the batch during the loss function calculation `class Canny(): def repr(self): return "Canny"

def __init__(self):
    self.canny = Net(threshold=1, use_cuda=True)
    self.canny.cuda()
    self.criterion = torch.nn.MSELoss()

def __call__(self, output, target, *args, **kwargs):
    losses = []

    for i in range(output.size()[0]):
        losses.append(self.criterion(self.canny(output[i, :, :,:].unsqueeze(0)),
                                     self.canny(target[i, :, :,:].unsqueeze(0))))
    return sum(losses)/output.size()[0]*0.15

`

[RuiWang1998]

add the following code as the first line of forward() batch_size = img.shape[0] change code in lines 119-128 to

indices = (inidices_negative.view(-1).data * pixel_count + pixel_range.repeat(1, batch_size)).squeeze() channel_select_filtered_negative = all_filtered.view(-1)[indices.long()].view(batch_size, 1, height, width) channel_select_filtered = torch.cat([channel_select_filtered_positive,channel_select_filtered_negative], 1) is_max = channel_select_filtered.min(dim=1)[0] > 0.0 is_max = torch.unsqueeze(is_max, dim=1)

[dekura]

@RuiWang1998 Hi, I change the code as you said. But I got error like this.

Traceback (most recent call last):
  File "train.py", line 51, in <module>
    model.optimize_parameters()   # calculate loss functions, get gradients, update network weights
  File "/research/dept7/glchen/github/pixel2pixel/models/pix2pix_model.py", line 162, in optimize_parameters
    self.forward()                   # compute fake images: G(A)
  File "/research/dept7/glchen/github/pixel2pixel/models/pix2pix_model.py", line 122, in forward
    self.fake_B_CANNY = self.CannyNet(self.fake_B)
  File "/research/dept7/glchen/miniconda3/envs/guojin/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/research/dept7/glchen/miniconda3/envs/guojin/lib/python3.7/site-packages/torch/nn/parallel/data_parallel.py", line 150, in forward
    return self.module(*inputs[0], **kwargs[0])
  File "/research/dept7/glchen/miniconda3/envs/guojin/lib/python3.7/site-packages/torch/nn/modules/module.py", line 547, in __call__
    result = self.forward(*input, **kwargs)
  File "/research/dept7/glchen/github/pixel2pixel/models/canny_net.py", line 127, in forward
    thin_edges[is_max == 0] = 0.0
RuntimeError: copy_if failed to synchronize: device-side assert triggered

I googled this error and know it may be caused by this line of code

    thin_edges[is_max == 0] = 0.0

because ths mask of is_max == 0 will cause the gradient to explode. Am I right?If so, Do you have any solution about this?

I am going to compare the edge of real_input and generated_fake，and feed if to the loss function, so as to increase the performance, But i got this problem now.

[ireneMsm2020]

@dekura Hi, I have the same problem as you . Have you solved it

[fordevoted]

@dekura @ireneMsm2020 Hey guys, I modify the code to support batch size >1, you can refer to https://github.com/fordevoted/CannyEdgePytorch/blob/master/canny.py, if needed. I discover that if use method provided by @RuiWang1998, it will contain different edge detection result for same image, instead different images (images in batch), and get incorrect result.

[poptree]

@dekura @ireneMsm2020 Hey guys, I modify the code to support batch size >1, you can refer to https://github.com/fordevoted/CannyEdgePytorch/blob/master/canny.py, if needed. I discover that if use method provided by @RuiWang1998, it will contain different edge detection result for same image, instead different images (images in batch), and get incorrect result.

This version of code is very buggy, and I think it doesn't really solve the batch size issue.

[fordevoted]

@dekura @ireneMsm2020 Hey guys, I modify the code to support batch size >1, you can refer to https://github.com/fordevoted/CannyEdgePytorch/blob/master/canny.py, if needed. I discover that if use method provided by @RuiWang1998, it will contain different edge detection result for same image, instead different images (images in batch), and get incorrect result.

This version of code is very buggy, and I think it doesn't really solve the batch size issue.

Sorry for forgetting to test input images are gray scale, I have already updated the code. If there are still any bug, please let me know and provide your environment and testing data for debugging.

[HeMuling]

@dekura @ireneMsm2020 Hey guys, I modify the code to support batch size >1, you can refer to https://github.com/fordevoted/CannyEdgePytorch/blob/master/canny.py, if needed. I discover that if use method provided by @RuiWang1998, it will contain different edge detection result for same image, instead different images (images in batch), and get incorrect result.

This version of code is very buggy, and I think it doesn't really solve the batch size issue.

Sorry for forgetting to test input images are gray scale, I have already updated the code. If there are still any bug, please let me know and provide your environment and testing data for debugging.

hi @fordevoted, thanks for providing such useful code. I believe your improved code generally works for multi-batch case, however, there are some bugs:

https://github.com/fordevoted/CannyEdgePytorch/blob/1c455a960e4021663fc173fb8f1364564b764908/net_canny.py#L8 has wrong indent

https://github.com/fordevoted/CannyEdgePytorch/blob/1c455a960e4021663fc173fb8f1364564b764908/net_canny.py#L11 should be super(Net, self).__init__()

also, i think if you want to obtain the final images with multiple batches, the canny.py file should define the function to be:

def canny(raw_img ,use_cuda=False):
    if isinstance(raw_img, np.ndarray):
        img = torch.from_numpy(raw_img.transpose((2, 0, 1)))
    else:
        img = raw_img
    if len(img.shape) == 3:
        batch = img.unsqueeze(0).float()
    else:
        batch = img.float() 

    if img.max() > 1:
        thr = 1000
    else:
        thr = 3

    net = Net(threshold=thr, use_cuda=use_cuda)
    if use_cuda:
        net.cuda()

    net.eval()

    if use_cuda:
        data = Variable(batch).cuda()
    else:
        data = Variable(batch)

    blurred_img, grad_mag, grad_orientation, thin_edges, thresholded, early_threshold = net(data)

    return (thresholded.data.cpu() > 0.0).to(torch.float)

where i would confine input the data to be torch.Tensor with shape [batch_size, 3, H, W] or [batch_size, 1, H, W]

by test, this should successfully work for multi-batch case

[fordevoted]

hi @fordevoted, thanks for providing such useful code. I believe your improved code generally works for multi-batch case, however, there are some bugs:

https://github.com/fordevoted/CannyEdgePytorch/blob/1c455a960e4021663fc173fb8f1364564b764908/net_canny.py#L8 has wrong indent

https://github.com/fordevoted/CannyEdgePytorch/blob/1c455a960e4021663fc173fb8f1364564b764908/net_canny.py#L11 should be super(Net, self).__init__()

also, i think if you want to obtain the final images with multiple batches, the canny.py file should define the function to be:

def canny(raw_img ,use_cuda=False):
    if isinstance(raw_img, np.ndarray):
        img = torch.from_numpy(raw_img.transpose((2, 0, 1)))
    else:
        img = raw_img
    if len(img.shape) == 3:
        batch = img.unsqueeze(0).float()
    else:
        batch = img.float() 

    if img.max() > 1:
        thr = 1000
    else:
        thr = 3

    net = Net(threshold=thr, use_cuda=use_cuda)
    if use_cuda:
        net.cuda()

    net.eval()

    if use_cuda:
        data = Variable(batch).cuda()
    else:
        data = Variable(batch)

    blurred_img, grad_mag, grad_orientation, thin_edges, thresholded, early_threshold = net(data)

    return (thresholded.data.cpu() > 0.0).to(torch.float)

where i would confine input the data to be torch.Tensor with shape [batch_size, 3, H, W] or [batch_size, 1, H, W]

by test, this should successfully work for multi-batch case

Hi HeMuling, thanks for the review, I just update the code and fix the bugs according to your suggestion. It is greatly appreciated.