Training Mask R-CNN using RLE bitmasks error

[alono88]

I am following the Mask R-CNN tutorial and changed the dataset_dict to support segmentation maps in bitmap format using RLE instead of polygons. I confirmed the data is processed properly using detectron2 visualization tool. When trying to train the network, I'm getting an error regarding polygons.

File "/home/ubuntu/detectron2/detectron2/data/detection_utils.py", line 149, in transform_instance_annotations polygons = [np.asarray(p).reshape(-1, 2) for p in annotation["segmentation"]] File "/home/ubuntu/detectron2/detectron2/data/detection_utils.py", line 149, in <listcomp> polygons = [np.asarray(p).reshape(-1, 2) for p in annotation["segmentation"]] ValueError: cannot reshape array of size 1 into shape (2)

It seems there is no support for RLE format during training although the visualization works. Is there any way to train using bitmasks?

[ppwwyyxx]

The model does support training with bitmasks, if instances.gt_masks = BitMasks(..). See the BitMasks class at https://detectron2.readthedocs.io/modules/structures.html#detectron2.structures.BitMasks

But the data loader does not recognize RLE, and therefore does not convert RLE to bitmasks.

[ppwwyyxx]

You can use a different dataloader following https://detectron2.readthedocs.io/tutorials/data_loading.html

[alono88]

Thanks!

[adriaciurana]

I'm with the same problem. To solve it I have edited the following files.

In structures/masks.py BitMasks class add the following method:

@staticmethod
    def from_rle(
        rles
    ) -> "BitMasks":
        """
        Args:
            rles
            height, width (int)
        """
        masks = [mask_utils.decode(rle).astype(np.bool) for rle in rles]
        return BitMasks(torch.stack([torch.from_numpy(x) for x in masks]))

In data/detection_utils.py change:

if len(annos) and "segmentation" in annos[0]:
        polygons = [obj["segmentation"] for obj in annos]
        if mask_format == "polygon":
            masks = PolygonMasks(polygons)
        else:
            assert mask_format == "bitmask", mask_format
            masks = BitMasks.from_polygon_masks(polygons, *image_size)
        target.gt_masks = masks

by:

if len(annos) and "segmentation" in annos[0]:
        polygons = [obj["segmentation"] for obj in annos]
        if mask_format == "polygon":
            masks = PolygonMasks(polygons)
        elif mask_format == "bitmask":
            masks = BitMasks.from_polygon_masks(polygons, *image_size)
        elif mask_format == "rle":
            masks = BitMasks.from_rle(polygons)

        else:
            assert mask_format
        target.gt_masks = masks

In also dataset_mapper.py change:

# USER: Implement additional transformations if you have other types of data
            annos = [
                utils.transform_instance_annotations(
                    obj, transforms, image_shape, keypoint_hflip_indices=self.keypoint_hflip_indices
                )
                for obj in dataset_dict.pop("annotations")
                if obj.get("iscrowd", 0) == 0
            ]

by:

annos = [
                obj
                for obj in dataset_dict.pop("annotations")
                if obj.get("iscrowd", 0) == 0
            ]

In your cfg change cfg.INPUT.MASK_FORMAT = 'rle'

I think I have not had to apply more changes.

[ppwwyyxx]

Now the default dataloader can work with RLE formats inside your dataset. All you need is:

1. use RLE format (documented at https://detectron2.readthedocs.io/tutorials/datasets.html#standard-dataset-dicts) in your dataset
2. set INPUT.MASK_FORMAT='bitmask'.

[pvti]

Hi @ppwwyyxx I'm trying to train a instance segmentaion model with 1 class X using detectron2's tutorial, but with a binary mask dataset. With your instructions, I use RLE format (documented at https://detectron2.readthedocs.io/tutorials/datasets.html#standard-dataset-dicts). In each "annotations", my "segmention" is a dict "represents the per-pixel segmentation mask in COCO’s RLE format. The dict should have keys “size” and “counts”. You can convert a uint8 segmentation mask of 0s and 1s into RLE format by pycocotools.mask.encode(np.asarray(mask, order="F"))." (I verify that bit-mask is successfully converted to RLE format). So in this case, can you explain some of my questions, please:

1. What exactly is the "segmentation"? For example: left: original binary mask of 0s and 1s center: binary mask of the 1st instance right: binary mask of the 2nd instance => "segmentation" = pycocotools.mask.encode(np.asarray(left, order="F")) OR if I'm not wrong, in each instance "segmentation" = pycocotools.mask.encode(np.asarray(center/right , order="F")), right?
2. I trained the model with the boilerplate code in your getting started collab notebook, every thing is ok. But when I try to evaluate the model's performance using AP metric implemented in COCO API as in your example, I got the error: WARNING [03/04 10:32:52 d2.evaluation.coco_evaluation]: json_file was not found in MetaDataCatalog for 'hand10_val'. Trying to convert it to COCO format ... [03/04 10:32:52 d2.data.datasets.coco]: Converting dataset annotations in 'hand10_val' to COCO format ...) [03/04 10:32:52 d2.data.datasets.coco]: Converting dataset dicts into COCO format Traceback (most recent call last): File "/home/n/detectron2/datasets/EgteaGaze+/hand10/test.py", line 132, in <module> evaluator = COCOEvaluator("hand10_val", cfg, False, output_dir="./output/") File "/home/n/detectron2/detectron2/evaluation/coco_evaluation.py", line 71, in __init__ convert_to_coco_json(dataset_name, cache_path) File "/home/n/detectron2/detectron2/data/datasets/coco.py", line 416, in convert_to_coco_json coco_dict = convert_to_coco_dict(dataset_name) File "/home/n/detectron2/detectron2/data/datasets/coco.py", line 334, in convert_to_coco_dict polygons = PolygonMasks([segmentation]) File "/home/n/detectron2/detectron2/structures/masks.py", line 271, in __init__ process_polygons(polygons_per_instance) for polygons_per_instance in polygons File "/home/n/detectron2/detectron2/structures/masks.py", line 271, in <listcomp> process_polygons(polygons_per_instance) for polygons_per_instance in polygons File "/home/n/detectron2/detectron2/structures/masks.py", line 262, in process_polygons "Got '{}' instead.".format(type(polygons_per_instance)) AssertionError: Cannot create polygons: Expect a list of polygons per instance. Got '<class 'dict'>' instead.

   ---

   I guess that it must have to set INPUT.MASK_FORMAT='bitmask' (this is in the case of training config), so in the case of evaluating, what need to be set? I also tried to print("cfg.dict= ", cfg.dict) but only got type(cfg)= <class 'detectron2.config.config.CfgNode'> cfg.__dict__= {'__immutable__': False, '__deprecated_keys__': set(), '__renamed_keys__': {}, '__new_allowed__': False} How can I see all the atribute of the cfg?

3. Can you note all the things relate in case of training with binary mask, is there anything need to be noticed/different with the polygon-training. Many thanks for your great framework and your enthusiastic supports!

[ppwwyyxx]

Evaluation of a generic (not COCO format) dataset with RLE is not yet supported. There is a TODO in https://github.com/facebookresearch/detectron2/blob/5e2a1ecccd228227c5a605c0a98d58e1b2db3640/detectron2/data/datasets/coco.py#L335-L337 (cc @botcs )

[ppwwyyxx]

The above issue about evaluation is hopefully fixed in https://github.com/facebookresearch/detectron2/commit/6901cc7e35ffcef6a4515efe26410a457612ac36

[siyuanfeng-tri]

The above issue about evaluation is hopefully fixed in 6901cc7

I think I need to do a bit more to get this actually working.

1. I had to cast this into int explicitly, otherwise json dump complains about TypeError: Object of type 'uint32' is not JSON serializable https://github.com/facebookresearch/detectron2/blob/master/detectron2/data/datasets/coco.py#L339
2. I had to decode segmentation["counts"] into an ascii string otherwise json dump complains about not being able to serialize bytes. TypeError: Object of type 'bytes' is not JSON serializable

[ppwwyyxx]

There are a few different RLE formats cocoapi recognizes and the 'counts' in coco's original json is actually a list of int. Should figure out how to convert the bytes to that format.

[siyuanfeng-tri]

Thanks for the reply! I am not that familiar with different flavors of coco TBH. The only thing I really cared about here is to preserve the RLE annotation (which comes from pycocotools.mask.encode()) through json conversion and back through detectron2's wrappers, since I have a custom dataset, where everything is an image.

So I looked briefly around https://github.com/facebookresearch/detectron2/blob/master/detectron2/data/datasets/coco.py#L160 and looked up where imgToAnns comes from in https://github.com/cocodataset/cocoapi/blob/master/PythonAPI/pycocotools/coco.py I think my workaround is probably fine for what I care about? As long as json is serializing and deserializing that bytes array the same way I should be fine. I did something like segmentation["counts"] = segmentation["counts"].decode(''ascii") before calling json dump.

[kharyal]

Now the default dataloader can work with RLE formats inside your dataset. All you need is:

1. use RLE format (documented at https://detectron2.readthedocs.io/tutorials/datasets.html#standard-dataset-dicts) in your dataset
2. set INPUT.MASK_FORMAT='bitmask'.

Still not working for me. I am not getting any error but the network is not learning anything. It just runs for 500 and doesn't segment anything. The Epoch error is also dropping to 0.

[Paragjain10]

There are a few different RLE formats cocoapi recognizes and the 'counts' in coco's original json is actually a list of int. Should figure out how to convert the bytes to that format.

Hello @ppwwyyxx,

I am trying to implement CenterMask2. My annotations are in coco rle format, I have set cfg.INPUT.MASK_FORMAT='bitmask' But unfortunately, this error is being raised:

/home/student2/anaconda3/envs/Parag_Centremask2/lib/python3.8/site-packages/detectron2/data/detection_utils.py:414: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at /opt/conda/conda-bld/pytorch_1607370172916/work/torch/csrc/utils/tensor_numpy.cpp:141.) torch.stack([torch.from_numpy(np.ascontiguousarray(x)) for x in masks]) /home/student2/Desktop/Parag_Centremask2/centermask2/centermask/modeling/fcos/fcos_outputs.py:402: UserWarning: This overload of nonzero is deprecated: nonzero() Consider using one of the following signatures instead: nonzero(, bool as_tuple) (Triggered internally at /opt/conda/conda-bld/pytorch_1607370172916/work/torch/csrc/utils/python_arg_parser.cpp:882.) per_candidate_nonzeros = per_candidate_inds.nonzero() Traceback (most recent call last): File "/home/student2/Desktop/Parag_Centremask2/centermask2/train_net.py", line 294, in launch( File "/home/student2/anaconda3/envs/Parag_Centremask2/lib/python3.8/site-packages/detectron2/engine/launch.py", line 62, in launch main_func(args) File "/home/student2/Desktop/Parag_Centremask2/centermask2/train_net.py", line 288, in main return trainer.train() File "/home/student2/Desktop/Parag_Centremask2/centermask2/train_net.py", line 165, in train self.train_loop(self.start_iter, self.max_iter) File "/home/student2/Desktop/Parag_Centremask2/centermask2/train_net.py", line 154, in train_loop self.run_step() File "/home/student2/anaconda3/envs/Parag_Centremask2/lib/python3.8/site-packages/detectron2/engine/defaults.py", line 423, in run_step self._trainer.run_step() File "/home/student2/anaconda3/envs/Parag_Centremask2/lib/python3.8/site-packages/detectron2/engine/train_loop.py", line 228, in run_step loss_dict = self.model(data) File "/home/student2/anaconda3/envs/Parag_Centremask2/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(*input, *kwargs) File "/home/student2/anaconda3/envs/Parag_Centremask2/lib/python3.8/site-packages/detectron2/modeling/metaarch/rcnn.py", line 166, in forward , detector_losses = self.roi_heads(images, features, proposals, gt_instances) File "/home/student2/anaconda3/envs/Parag_Centremask2/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl result = self.forward(input, **kwargs) File "/home/student2/Desktop/Parag_Centremask2/centermask2/centermask/modeling/centermask/center_heads.py", line 401, in forward losses, mask_features, selected_mask, labels, maskiou_targets = self._forward_mask(features, proposals) File "/home/student2/Desktop/Parag_Centremask2/centermask2/centermask/modeling/centermask/center_heads.py", line 476, in _forward_mask loss, selected_mask, labels, maskiou_targets = mask_rcnn_loss(mask_logits, proposals, self.maskiou_on) File "/home/student2/Desktop/Parag_Centremask2/centermask2/centermask/modeling/centermask/mask_head.py", line 80, in mask_rcnn_loss cropped_mask = crop(instances_per_image.gt_masks.polygons, instances_per_image.proposal_boxes.tensor) AttributeError: 'BitMasks' object has no attribute 'polygons'

Process finished with exit code 1

[DavidNguyen95]

w to convert the bytes to that

Did you find the solution? when I try CenterMask.I also got the same error 'BitMasks' object has no attribute 'polygons' like you.

Here is my process: Follow detectron2 document: I convert a bitmask dataset to RLE format using pycocotools.mask.encode(np.asarray(mask, order="F")) . Then I set _cfg.INPUT.MASKFORMAT='bitmask'

`

[sipie800]

cfg.INPUT.MASK_FORMAT = "bitmask" doesn't work. I do use compressed rle in json which other api like torch or mmdetection can read.

[koreandrum97]

cfg.INPUT.MASK_FORMAT = "bitmask" doesn't work. I do use compressed rle in json which other api like torch or mmdetection can read.

Is it working now? I'd like to semantic segmentation task with detectron2. But I wonder which types of annotation(RLE or Polygons) Please reply!! @ppwwyyxx

[leiluoray1]

check the model output here, say if you want to save the first instance segmentation mask, which is pred_masks[0]. you can

mask0 = outputs["instances"].pred_masks[0].cpu().numpy()*255.0
mask0 = mask0.astype(int)
cv2_imshow(mask0)

[hamzagorgulu]

@koreandrum97 have you found a solution? I train the model with rle masks and assuming the task is converted to semantic segmentation, but I only get masks for instance segmentation. In the [docs](https://detectron2.readthedocs.io/en/latest/tutorials/models.html), it says the output dict should contain "sem_seg" key but I dont get the prediction probabilities.