search

CSIPlab / MMSFormer

We propose a novel fusion strategy that can effectively fuse information from different modality combinations. We also propose a new model named Multi-Modal Segmentation TransFormer (MMSFormer) that incorporates the proposed fusion strategy to perform multimodal material and semantic segmentation tasks.
https://csiplab.github.io/MMSFormer/
Apache License 2.0
9 stars 4 forks source link

hi 关于FMB的问题咨询 #5

Closed hhuxuchang closed 7 months ago

hhuxuchang commented 7 months ago

您好，在您的fmb数据集代码中，仅有14类，没有背景类，但plaette有15个？ 在学习您的代码，测试FMB数据集的过程中，计算指标的过程中会持续报错 self.hist += torch.bincount(target[keep] * self.num_classes + pred[keep], minlength=self.num_classes**2).view(self.num_classes, self.num_classes) RuntimeError: shape '[14, 14]' is invalid for input of size 3556

请问这个情况您有遇到过吗？

kaykobad commented 7 months ago

Hi, Thanks for your question. I have not encountered this issue so far. I tried running the code and everything worked fine. I did not consider background and PALETTE does not play any role in performance calculation.

If you look at the dataset (line 71-72), I am assigning unlabeled pixels (255) to background (0). Then I am removing background by subtracting 1 from the ground truth. That will make background class (0) to be set as unlabeled (255). So effectively there are 14 classes.

While calculating the loss, I am ignoring label 255 as defined in the config file. So the number of class and PALETTE should not be an issue.

One possible issue can be the preprocessing of the dataset. Are you using the correct input size of the images as defined in the config file? Can you write down the specific steps to reproduce the issue?

Thanks again.

hhuxuchang commented 7 months ago

非常感谢您的回复

kaykobad commented 7 months ago

Welcome.

LeonSakura commented 1 month ago

您好，我也遇到了这个问题，请问应该如何解决呢？

LeonSakura commented 1 month ago

我解决了这个问题，是windows系统的问题，数据读取的错误，如果在windows系统下运行代码，需要把 fmb.py 中的62-63行的代码换成

thermal = rgb.replace('\\Visible', '\\Infrared')   
lbl_path = rgb.replace('\\Visible', '\\Label')
kaykobad commented 1 month ago

Thank you so much @LeonSakura for pointing this out. I was running in a Linux machine, so did not encounter this issue.