Zyun-Y / DconnNet

Codes for CVPR2023 paper "Directional Connectivity-based Segmentation of Medical Images"
131 stars 7 forks source link

multiclass prediction wrong #29

Closed d1488j closed 3 months ago

d1488j commented 4 months ago

Hello,

I got such great help from you last time, could you help me with this issue too? I want to do an image classification with three classes. The ground truth is a 2D png image where the three classes are marked with different gray values (class 1: white, class2: light gray, class3: dark gray). I have no error message, but the problem is that only class 1 (white area) is recognized. So for class 1 DSC is really good, but for class 2 and 3 the classification does not work. DSC: Class1: 0.8673 Class2: 0.001 Class3: 5.4148e-08

Please find attached the GT, the image and the predictions for the 3 classes.

Patient50_class1_pred Patient50_class2_pred Patient50_class3_pred Patient50slice5_image Patient50slice5_mask

I am wondering if the input has the correct format. What kind of input do I need for a multiclass classification? Or what else can be the reason that only class 1 is recognized?

Thank you in advance!

Zyun-Y commented 4 months ago

It seems like an input format error in your case. Please make sure that you checked the following:

  1. set the correct num-class in train.py
  2. call self.multi_class_forward in connect_loss.py
  3. check if your mask/img/feature shape matches the shape wrote in the comment for each step in loss function.
  4. In a typical case of multi-class problem, your mask input will have the shape of (B, H, W) and the value of the mask indicates the class and it will be further converted to one hot mask in the connect_loss.py. You can also check if the conversion works well by output/save the masks.

Also, from the label sample you provided, it seems like your masks are overlapping with each other? E.g., the white region is also a part of your grey region? If this is the case, treating your task as a multi-single-class segmentation (i.e., use the original full mask and also sigmoid instead of softmax as the activation ) might work better. But even if you follow the standard multi-class scheme, your results should be better than what you currently have. Please check the code and your input.

d1488j commented 3 months ago

Thanks, I checked your suggestions. At the end I could solve it, by saving the mask as numpy-array and not as png. It seems like the network can't ectract the classes out of a png for muliclass segmentation.

Zyun-Y commented 3 months ago

The input format could always be an issue when having different package versions. Glad you solved it.