AttributeError: Cannot find field 'gt_masks' in the given Instances! for mask2former with coco-format dataset

[jetsonwork]

Hello, I want to fine tune a swin-based mask2former model, and I register my coco format dataset as below:

from detectron2.data.datasets import register_coco_instances register_coco_instances("my_dataset_train", {}, "/content/drive/MyDrive/ColabNotebooks/data/3D-EM-Platelet/train/3D-EM-platelet-train.json", "/content/drive/MyDrive/ColabNotebooks/data/3D-EM-Platelet/train") register_coco_instances("my_dataset_val", {}, "/content/drive/MyDrive/ColabNotebooks/data/3D-EM-Platelet/val/3D-EM-platelet-val.json", "/content/drive/MyDrive/ColabNotebooks/data/3D-EM-Platelet/val")

train_metadata = MetadataCatalog.get("my_dataset_train") train_dataset_dicts = DatasetCatalog.get("my_dataset_train")

val_metadata = MetadataCatalog.get("my_dataset_val") val_dataset_dicts = DatasetCatalog.get("my_dataset_val")

but when I want to train the model I am facing to the below error. Could you please advise how can I solve the issue?

AttributeError: Cannot find field 'gt_masks' in the given Instances!

[github-actions[bot]]

You've chosen to report an unexpected problem or bug. Unless you already know the root cause of it, please include details about it by filling the issue template. The following information is missing: "Instructions To Reproduce the Issue and Full Logs";

[DimbyTa]

Hello @jetsonwork,

Your issue relates not to how you register your dataset but to the data itself or the model you use. Indeed, after reading the docs: https://detectron2.readthedocs.io/en/latest/_modules/detectron2/structures/instances.html?highlight=AttributeError#, you can see that the AttributeError is raised when: name == "_fields" or name not in self._fields, as stated in the docs of Instances.

If it was related to how you registered your dataset, the error would have stated that it could not find my_dataset_train or my_dataset_val.

If you read the docs on gt_masks, you can see that it is related to Model Input (https://detectron2.readthedocs.io/en/latest/tutorials/models.html?highlight=gt_masks#model-input-format), the docs talk about custom models and what they expect as input and the correct format.

As stated in https://detectron2.readthedocs.io/en/latest/tutorials/models.html?highlight=gt_masks#use-a-model, each model takes as inputs a list[dict] with each dict corresponding to one image. It is also stated that the required keys depend on the model.

That means the mask2former model probably requires the gt_masks field in the inputs (e.g.: [{"image": Tensor, "height": some_value, "width": some_value, ..., "gt_masks": PolygonMasks or BitMasks},{"image": Tensor, "height": some_value, "width": some_value, ..., "gt_masks": PolygonMasks or BitMasks},... ]).

Can you confirm that you do have gt_masks in the inputs? Try: print(train_dataset_dicts[0].keys())

For more information on Instances: https://detectron2.readthedocs.io/en/latest/modules/structures.html#detectron2.structures.Instances

These might help you but I'm not sure: https://colab.research.google.com/github/NielsRogge/Transformers-Tutorials/blob/master/MaskFormer/Fine-tuning/Fine_tuning_MaskFormerForInstanceSegmentation_on_semantic_sidewalk.ipynb

https://detectron2.readthedocs.io/en/latest/tutorials/datasets.html

[github-actions[bot]]

Requested information was not provided in 7 days, so we're closing this issue.

Please open new issue if information becomes available. Otherwise, use github discussions for free-form discussions.