Determine multi-class region mask organization

[leewujung]

Currently our region mask handles a single class only, but we should generalize it to handle multi-class labels, since it is a common case and we'll likely run into that very soon.

[valentina-s]

Reference for creating 3D Regionmask: https://regionmask.readthedocs.io/en/stable/notebooks/mask_3D.html

[valentina-s]

Some other formats for storing multiclass labels:

• COCO format

[valentina-s]

One open question is if a certain class is missing in an image whether to store a layer for it (i.e. all Nans) in a one-hot format.

[leewujung]

We decided to first implement the 3D ((label, depth, time)) one-hot format first, and then we will deal with the case when some label classes overlap in the 2D ((depth, time)) format.

Related to #92.

[leewujung]

Hey @ctuguinay : The majority of this is addressed in #96, right? If I understand correctly, right now for the scenarios where the multi-class labels do not overlap at any pixels, the conversion between 3d and 2d formats are in place now. For the scenario where the multi-class labels do overlap, it seem the simplest is to only use the 3d format?

And what is the current implementation when one of the classes is missing in a particular dataset, is a "slice" created for that?

[ctuguinay]

@leewujung Yup, the majority of this is addressed in #96. And yes, that problem still exists, so the simplest way is to just use the 3D implementation. I even check that there are no overlapping layers.

When one of the classes is missing in a particular dataset, no slice is created for that. In fact, it takes the classes from the input dataset. This is from the covert 2d to 3d function in https://github.com/OSOceanAcoustics/echoregions/blob/main/echoregions/utils/api.py:

    # Get unique non nan values from the 2d mask
    unique_non_nan = list(np.unique(mask_2d_da.data[~np.isnan(mask_2d_da.data)]))
    if len(unique_non_nan) == 0:
        unique_non_nan = None

    # Create a list of mask objects from one-hot encoding M.data non-nan values
    # and a dictionary to remember said values from one-hot encoded data arrays.
    # If unique_non_nan is None, make mask_dictionary None.
    mask_list = []
    mask_dictionary = {"dims": "label", "data": []}
    if unique_non_nan is not None:
        mask_dictionary = {"dims": "label", "data": []}
        for _, value in enumerate(unique_non_nan):
            # Create new 1d mask
            new_mask_data = xr.where(mask_2d_da == value, 1.0, 0.0)
            # Append data to mask_list and mask_dictionary
            mask_list.append(new_mask_data)
            mask_dictionary_list = mask_dictionary["data"]
            mask_dictionary_list.append(value)
            mask_dictionary["data"] = mask_dictionary_list
        mask_3d_da = xr.concat(mask_list, dim="label")

What I can do is instead of using unique_non_nan to create slices for each class, I can have one of the input arguments be a list or dictionary containing information on which classes (from the unique non nan values) to create slices on.

[leewujung]

Just re-read the above discussion on the case when one of the classes is missing -- it seems that we can stay with the handling right now (i.e. create whatever "slices" of classes that exist in the labels). We can think about this again once we get to more diverse labels and it may become obvious then which way we should go, since this is just an abstract question at the moment. So, I'll close this issue now!