search

CSIPlab / MMSFormer

We propose a novel fusion strategy that can effectively fuse information from different modality combinations. We also propose a new model named Multi-Modal Segmentation TransFormer (MMSFormer) that incorporates the proposed fusion strategy to perform multimodal material and semantic segmentation tasks.
https://csiplab.github.io/MMSFormer/
Apache License 2.0
7 stars 4 forks source link

code #2

Closed Aptaorean closed 8 months ago

Aptaorean commented 8 months ago

Excuse me, I encountered this error when I reproduced your code in order. I wonder if a key configuration file is missing?

  dataset = eval(cfg['DATASET']['NAME'])(cfg['DATASET']['ROOT'], 'val', transform, cfg['DATASET']['MODALS'], case)
   File "<string>", line 1, in <module>
NameError: name 'FMB' is not defined
kaykobad commented 8 months ago

Thank you for pointing this out. I have uploaded the missing files. Please let me know if you need further help reproducing the results.

Aptaorean commented 8 months ago

There is another question here.

File "E:\work\Segment\MMSFormer-main\tools\val_mm.py", line 196, in <module>
    main(cfg)
  File "E:\work\Segment\MMSFormer-main\tools\val_mm.py", line 166, in main
    acc, macc, f1, mf1, ious, miou, _ = evaluate(model, dataloader, device)
  File "D:\application\anaconda\envs\pytorch13\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "E:\work\Segment\MMSFormer-main\tools\val_mm.py", line 83, in evaluate
    metrics.update(preds.softmax(dim=1), labels)
  File "E:\work\Segment\MMSFormer-main\semseg\metrics.py", line 15, in update
    self.hist += torch.bincount(target[keep] * self.num_classes + pred[keep], minlength=self.num_classes**2).view(self.num_classes, self.num_classes)
RuntimeError: shape '[14, 14]' is invalid for input of size 3556