Problems in visualizing grape_detection_syntheticday

[andreaceruti]

On this dataset I can see the bounding boxes with the method viz.visualize_image_and_boxes, but It give errors if you try to run the other methods for showing the masks (visualize_image_and_mask, overlay_segmentation_masks, visualize_overlaid_masks). The problem could be that the coco_info["segmentation"] is a 3D array an not a 2D one.

[amogh7joshi]

Could you post a code snippet showing exactly what you're running? It'll help me to reproduce the issue.

[andreaceruti]

This is the snippet:

!pip install -q agml
import agml
import agml.viz as viz
import numpy as np
from pprint import pprint

loader = agml.data.AgMLDataLoader('grape_detection_syntheticday')
image, coco_info = loader[0]
_ = viz.visualize_image_and_mask(image, coco_info["segmentation"])

And the problem is

ValueError Traceback (most recent call last) in () 1 image, coco_info = loader[0] 2 pprint(cocoinfo) ----> 3 = viz.visualize_image_and_mask(image, coco_info["segmentation"])

/usr/local/lib/python3.7/dist-packages/agml/viz/masks.py in _preprocess_mask(mask) 40 label_indx = 1 41 while True: ---> 42 x_values, yvalues, * = np.where(mask == label_indx) 43 coords = np.stack([x_values, y_values]).T 44 if coords.size == 0:

ValueError: not enough values to unpack (expected at least 2, got 1)

if we see the segmentation part it looks like this:

'segmentation': array([[[576, 415, 599, 415, 599, 499, 576, 499]],> [[324, 404, 348, 404, 348, 479, 324, 479]],> [[259, 418, 283, 418, 283, 505, 259, 505]],> [[451, 386, 492, 386, 492, 497, 451, 497]],> [[405, 363, 447, 363, 447, 514, 405, 514]],> [[196, 420, 239, 420, 239, 578, 196, 578]],> [[143, 423, 185, 423, 185, 553, 143, 553]],> [[0, 459, 36, 459, 36, 613, 0, 613]],> [[0, 459, 46, 459, 46, 603, 0, 603]],> [[373, 393, 396, 393, 396, 476, 373, 476]]], dtype=object)}

[amogh7joshi]

Ah, I see the issue. AgML doesn't technically support instance segmentation datasets yet, so those segmentation dictionaries are simply here as placeholders (for whenever we add support in the future). They're not meant to be used for anything meaningful at the moment.

The visualize_image_with_mask method is meant for the semantic segmentation datasets, which have their segmentation masks in pixel format, not COCO format. That's why the method is failing. Hopefully this answers your question.

[masonearles]

Hi Andrea! Welcome to the AgML developer community. We're excited to have you involved. If you want to open up an issue of "add instance segmentation" and label it as an "enhancement", we can put it on the radar for development. Further, if you want to try to add the feature in a separate branch and make a pull request, that would be fantastic!

On Fri, Apr 15, 2022 at 7:27 AM Amogh Joshi @.***> wrote:

Ah, I see the issue. AgML doesn't technically support instance segmentation datasets yet, so those segmentation dictionaries are simply here as placeholders (for whenever we add support in the future). They're not meant to be used for anything meaningful at the moment.

The visualize_image_with_mask method is meant for the semantic segmentation datasets, which have their segmentation masks in pixel format, not COCO format. That's why the method is failing. Hopefully this answers your question.

— Reply to this email directly, view it on GitHub https://github.com/Project-AgML/AgML/issues/17#issuecomment-1100141785, or unsubscribe https://github.com/notifications/unsubscribe-auth/AG5FGPB7XPADTG46RBKXHELVFF4GBANCNFSM5TO73RTQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>