search

CSIPlab / MMSFormer

We propose a novel fusion strategy that can effectively fuse information from different modality combinations. We also propose a new model named Multi-Modal Segmentation TransFormer (MMSFormer) that incorporates the proposed fusion strategy to perform multimodal material and semantic segmentation tasks.
https://csiplab.github.io/MMSFormer/
Apache License 2.0
9 stars 4 forks source link

Issue Encountered When Training FMB Dataset #4

Closed Aptaorean closed 7 months ago

Aptaorean commented 9 months ago

Dear Dr. Kaykobad Reza, I am writing to discuss an issue I encountered while attempting to reproduce some code. Specifically, I have been successful in training the MCubeS dataset, but I have run into errors when attempting to train the FMB dataset. It seems there may be a need to adjust certain parameters when training with other datasets. Could you kindly provide guidance on any parameters or adjustments necessary for training with the FMB dataset or any other datasets besides MCubeS?

File "E:\work\Segment\MMSFormer-main\tools\val_mm.py", line 196, in <module> main(cfg) File "E:\work\Segment\MMSFormer-main\tools\val_mm.py", line 166, in main acc, macc, f1, mf1, ious, miou, _ = evaluate(model, dataloader, device) File "D:\application\anaconda\envs\pytorch13\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context return func(*args, **kwargs) File "E:\work\Segment\MMSFormer-main\tools\val_mm.py", line 83, in evaluate metrics.update(preds.softmax(dim=1), labels) File "E:\work\Segment\MMSFormer-main\semseg\metrics.py", line 15, in update self.hist += torch.bincount(target[keep] * self.num_classes + pred[keep], minlength=self.num_classes**2).view(self.num_classes, self.num_classes) RuntimeError: shape '[14, 14]' is invalid for input of size 3556

kaykobad commented 8 months ago

I could not reproduce the issue. All the parameters are set properly on the configuration files. Configuration file for each dataset is separate and can be found on the config folder. Are you using the right configuration files?