search

mrlooi / rotated_maskrcnn

Rotated Mask R-CNN: From Bounding Boxes to Rotated Bounding Boxes
MIT License
347 stars 62 forks source link

ROIMaskHead returns empty proposal tensors #13

Open ashnair1 opened 4 years ago

ashnair1 commented 4 years ago

❓ Questions and Help

I was able to train the rotated maskrcnn on my dataset. Note that my object contains small objects. However at some point in the code, it returns mask (selected_mask) as a tensor of shape [0, 1, 28, 28]. This was also the case for features and labels.

Initially I thought it was caused by images with no annotations. But I've checked to see that's not the case. It could be possible that the code is filtering out my objects because they're so small and thus resulting in an empty image.

So I guess my questions are:

  1. Does the code filter out object below a certain size?
  2. Can the code handle images with no annotations?
mrlooi commented 4 years ago
  1. Does the code filter out object below a certain size? Not explicitly, but in general you want to make sure that the object is big enough to at least fit into the receptive field of the conv net. Too small e.g. area of 30 pixels only, will not be possible in many proposal-based conv net architectures
  2. Can the code handle images with no annotations? I'm guessing you mean during training? If there are no annotations in an image, the training script skips it (if I remember correctly)
ashnair1 commented 4 years ago
  1. Does the code filter out object below a certain size? Not explicitly, but in general you want to make sure that the object is big enough to at least fit into the receptive field of the conv net. Too small e.g. area of 30 pixels only, will not be possible in many proposal-based conv net architectures
  2. Can the code handle images with no annotations? I'm guessing you mean during training? If there are no annotations in an image, the training script skips it (if I remember correctly)
  1. Couldn't I use smaller anchors to address that?

I noticed that in the step x = self.feature_extractor(features, proposals), x is an empty tensor. So I looked into its inputs: features and proposals. The features looked fine, but the proposals were empty tensors. Following this I looked into keep_only_positive_boxes. This function filters out proposals with label = 0. But in my current example, all proposals were label = 0. This results in zero proposals. This in turn causes x to be zero which cause the selected_mask to be empty and finally fails in the maskiou_head's forward function.

@mrlooi Do you have any advice on how to deal with this problem?

mrlooi commented 4 years ago

How small or big are your proposals? You can't use smaller anchors if they are already too small to fit into a convnet receptive field. Do read up about receptive fields and what that would mean when designing anchors and neural networks.

ashnair1 commented 4 years ago

Well the dataset I have is of satellite imagery (DOTA dataset) where object areas can vary widely between 3^2 pixels (like vehicles) and 712^2 pixels (like stadiums). Since using the FPN backbone locks the number of anchors that can be used to 5 which should also increase in powers of 2, I set ANCHOR_SIZES = (8, 16, 32, 64, 128).

trungpham2606 commented 4 years ago

@ashnair1 Have u solved this issue ? Iam also having this issue. The weird thing is that the problem comes from this PPC dataset. hmmmmmm