reformulate embedmask based on detectron2

TengFeiHan0 commented 4 years ago

Hello! I am working on rewriting your work based on detectron2, however, I got a problem and hope to receive some cues from you. In your implementation, there is a line as below: targets_masks = [target_im.get_field('masks').convert('mask').instances.masks.to(device=pixel_embed.device) for target_im in targets]

The structures(Boxes, Instances, Masks) in detectron2 are different from maskrcnn-benchmark. here is my reproduced code: targets_masks = [ target_im.gt_masks.to(device=pixel_embed.device) for target_im in targets]

then I got an error like this,

File "/home/fei/adet/adet/modeling/embedmask/losses.py", line 303, in prepare_masks
    n, h, w = mask_t.shape
AttributeError: 'PolygonMasks' object has no attribute 'shape',

I submitted an issue under Detectron2. The maintainer said that this is a bug of their source code, which means creating an object of PolygonMasks wouldn't return a tensor. So many operations, such as mask.shape, mask.permute(), are prohibited.

Any cues or suggestions would be highly appreciated.

yinghdb commented 4 years ago

The operation in my code is used to get the bitmaps of the masks, which should be Tensors. So, in the Detectron2, you can use its function to convert "PolygonMasks" to "BitMasks" first.

TengFeiHan0 commented 4 years ago

@yinghdb Thank you for your quick reply. Actually I did this after reading your code carefully. The new code is targets_masks = [ polygons_to_bitmask(target_im.gt_masks, o_h, o_w ) for target_im in targets] However, The image_size I chosen is incorrect.

File "/home/fei/detectron2/detectron2/structures/masks.py", line 31, in polygons_to_bitmask
    rles = mask_utils.frPyObjects(polygons, height, width)
  File "pycocotools/_mask.pyx", line 307, in pycocotools._mask.frPyObjects
Exception: input type is not supported.

So how to choose the height,width parameters?

yinghdb commented 4 years ago

@TengFeiHan0 They are just the height and width of the gt masks. But isn't it the problem of input type?

TengFeiHan0 commented 4 years ago

I use this command to acquire mask's height and width t_h, t_w = targets[0].image_size,because all target should have same image_size. then the mentioned error still exists. Anyway, I will try my best to fix it. @yinghdb

yinghdb commented 4 years ago

Maybe you should the input type and the required input type.

TengFeiHan0 commented 4 years ago

I close my issue because the mentioned problem has been fixed. My code adapted to detectron2 will be released in the future. @yinghdb

SkeletonOne commented 4 years ago

@TengFeiHan0 Hi, I am also working on implementing it in detectron2. How is everything going?

TengFeiHan0 commented 4 years ago

@SkeletonOne I have already finished the implementation several days ago, however, given it up due to some certain reasons. So I didn't test my reproduced code.

SkeletonOne commented 4 years ago

@TengFeiHan0 So will it be released in your repository? : )

TengFeiHan0 commented 4 years ago

@SkeletonOne I was planning to release my code，however， I find that centermask performs much better than embedmask。so i stopped to tune my implementation and pay my attention entirely on centermask. It should be mentioned that their source codes includes two parts. One is based on maskrcnn-benchmark,other is finished under detectron2. I guess that you may interested in this work.

SkeletonOne commented 4 years ago

@TengFeiHan0 It sounds fine. I will look into both works. Thanks!

yinghdb / EmbedMask

reformulate embedmask based on detectron2 #9