search

billhhh / ShaSpec

The official code repository of ShaSpec model from CVPR 2023 [paper](https://arxiv.org/pdf/2307.14126) "Multi-modal Learning with Missing Modality via Shared-Specific Feature Modelling"
24 stars 3 forks source link

Problems in Eval #2

Closed fybgogogo closed 3 months ago

fybgogogo commented 3 months ago

I think there is an issue with my training results. The output after eval is completely different from the result of tumor segmentation, but I don't know what the problem is with my operation. In outputs, there are basically only some white dots, and the desired results cannot be seen. 1 2

billhhh commented 3 months ago

Hello, can you try the samples from training set, see if it is well trained? Also, may I know if all cases perfom the same in the evaluation set?

Another thing you can try is to train more iterations, see if it helps.

billhhh commented 3 months ago

Oh, I think I know the reason: If you only trained 80000 iters with full modalities and eval with the default eval.sh, as now the model only see full modalities, but in the eval.sh, only --mode=1 is available by default, there is a gap between training and testing. So you either evaluate the model with --mode=0,1,2,3 now or you need to complete the other random drop modality training and evaluate the model again. To evaluate all different combination as in the paper, you should finish the random drop training.

fybgogogo commented 3 months ago

Thank you. I have obtained the segmentation result according to your method.

billhhh commented 3 months ago

cool