mseg-dataset
commented
6 months ago Hi @Aelina29, thanks for your interest in MSeg.
Regarding information about the architecture of the neural network -- we used the HRNetV2-W48 architecture. More information can be found here in the HRNet paper.
K. Sun, Y. Zhao, B. Jiang, T. Cheng, B. Xiao, D. Liu, Y. Mu, X. Wang, W. Liu, and J. Wang, “High-resolution representations for labeling pixels and regions,” arXiv:1904.04514, 2019.
Regarding the grayscale images produced by network inference -- these are predicted semantic segmentation label maps, not confidence maps. Each channel in [0,255] represents a class in the output taxonomy. You can find more information about the ordering of the class labels here:
Load taxonomy classes: https://github.com/mseg-dataset/mseg-api/blob/master/mseg/utils/names_utils.py#L29-L37 Loaded from this TSV file: https://github.com/mseg-dataset/mseg-api/blob/master/mseg/class_remapping_files/MSeg_master.tsv Save grayscale: https://github.com/mseg-dataset/mseg-semantic/blob/master/mseg_semantic/tool/inference_task.py#L394-L398
For example, if you run the MSeg Colab cells, and then execute !ls -lh . in a new cell to list the generated files, you will see that 4 grayscale images are written out, one for each of the 4 demo images:
total 508M
-rw-r--r-- 1 root root 53K Jan 6 16:25 img1_gray.jpg
-rw-r--r-- 1 root root 122K Jun 10 2020 img1.jpg
-rw-r--r-- 1 root root 261K Jan 6 16:25 img1_overlaid_classes.jpg
-rw-r--r-- 1 root root 45K Jan 6 16:26 img2_gray.jpg
-rw-r--r-- 1 root root 316K Jun 10 2020 img2.jpg
-rw-r--r-- 1 root root 192K Jan 6 16:26 img2_overlaid_classes.jpg
-rw-r--r-- 1 root root 61K Jan 6 16:24 img3_gray.jpg
-rw-r--r-- 1 root root 300K Jun 10 2020 img3.jpg
-rw-r--r-- 1 root root 219K Jan 6 16:24 img3_overlaid_classes.jpg
-rw-r--r-- 1 root root 31K Jan 6 16:25 img4_gray.jpg
-rw-r--r-- 1 root root 875K Jun 9 2020 img4.jpg
-rw-r--r-- 1 root root 294K Jan 6 16:25 img4_overlaid_classes.jpg
-rw-r--r-- 1 root root 505M Dec 7 2021 mseg-3m.pth
drwxr-xr-x 8 root root 4.0K Jan 6 16:10 mseg-api
drwxr-xr-x 8 root root 4.0K Jan 6 16:10 mseg-semantic
drwxr-xr-x 1 root root 4.0K Jan 4 14:21 sample_data
Your dataset is amazing. You have done a great job! Where can I find information about the architecture of your neural network? And my main question. Are the gray pictures cards of confidence or the same segmentation, but without labels?