Mapping pixel to label

[ghost]

Hi @mhamilton723, thank you for the great work.

I was wondering if there is any mapping between the image colormap and the label, when trying an image at test time ?

Referring to the colab notebok you published, it seems that linear_pred give the pixelwise prediction but when I checked the shape of model.label_cmap it shows (512, 3) which doesn't seems to match with the number of label that I get from get_class_labels(model.cfg.dataset_name)

I'm not sure to be using the right approach but basically 'm missing the link between the colormaps and the class labels.

Thank you for your reply and again great work.

[mhamilton723]

Hey @skabongo2022, thanks for reaching out.

model.label_cmap maps a class label to a color for pretty plotting. Here's some usage:

https://github.com/mhamilton723/STEGO/blob/9d966c0bd0dcde324a20927b306b632d5d627ee9/src/eval_segmentation.py#L189

To unpack this example:

plot_cluster = (model.label_cmap[
    model.test_cluster_metrics.map_clusters(
        saved_data["cluster_preds"][img_num])]).astype(np.uint8)

saved_data["cluster_preds"][img_num]: cluster predictions of shape [H,W] of (torch.long) encoding K classes model.test_cluster_metrics.map_clusters(...):after running through the examples and adding them to the cluster metrics, this function allows one to map clusters to labels according to the proper hungarian matching. Outputs a [H,W] tensor with K classes (now shuffled to match the ontology) model.label_cmap[...]: the final mapping that takes a class label array [H,W] and maps it to a color [H,W,3]. Sometimes we use pascal voc coloring, but for cityscapes we use their coloring method explicitly. Hope this helps!

[ghost]

Hi @mhamilton723, thank you for the reply.

I will close the issue for now and might re-open it if I still have questions/issues

Thank you for your availability.