search

CSIPlab / MMSFormer

We propose a novel fusion strategy that can effectively fuse information from different modality combinations. We also propose a new model named Multi-Modal Segmentation TransFormer (MMSFormer) that incorporates the proposed fusion strategy to perform multimodal material and semantic segmentation tasks.
https://csiplab.github.io/MMSFormer/
Apache License 2.0
9 stars 4 forks source link

code #3

Closed Aptaorean closed 9 months ago

Aptaorean commented 10 months ago

Hi there is still the following error File "E:\work\Segment\MMSFormer-main\tools\val_mm.py", line 196, in <module> main(cfg) File "E:\work\Segment\MMSFormer-main\tools\val_mm.py", line 166, in main acc, macc, f1, mf1, ious, miou, _ = evaluate(model, dataloader, device) File "D:\application\anaconda\envs\pytorch13\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context return func(*args, **kwargs) File "E:\work\Segment\MMSFormer-main\tools\val_mm.py", line 83, in evaluate metrics.update(preds.softmax(dim=1), labels) File "E:\work\Segment\MMSFormer-main\semseg\metrics.py", line 15, in update self.hist += torch.bincount(target[keep] * self.num_classes + pred[keep], minlength=self.num_classes**2).view(self.num_classes, self.num_classes) RuntimeError: shape '[14, 14]' is invalid for input of size 3556 May be the following code is not original

def update(self, pred: Tensor, target: Tensor) -> None:
        pred = pred.argmax(dim=1)
        keep = target != self.ignore_label
        self.hist += torch.bincount(target[keep] * self.num_classes + pred[keep], minlength=self.num_classes**2).view(self.num_classes, self.num_classes)
kaykobad commented 9 months ago

Hi, I checked the code and it is running fine. Please add some more details on the issue. When does it happen and how to reproduce it?

Please make sure your configuration file is correct, datasets and model weights are at right directories. Here is the result from my last run on MCubeS dataset.

<All keys matched successfully>
Evaluating...
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████| 51/51 [01:32<00:00,  1.82s/it]
mIoU : 53.11
Results saved in checkpoints/MCubeS/MiT-B4-B1-RGBNAD-MCubeS-53.11.pth