search

VITA-Group / GLNet

[CVPR 2019, Oral] "Collaborative Global-Local Networks for Memory-Efficient Segmentation of Ultra-High Resolution Images" by Wuyang Chen*, Ziyu Jiang*, Zhangyang Wang, Kexin Cui, and Xiaoning Qian
MIT License
347 stars 76 forks source link

Training with Aerial dataset and issue with the metric #21

Closed hmchuong closed 4 years ago

hmchuong commented 4 years ago

Thanks for your appreciate work! I'm trying to run your code on Aerial dataset but I have found some troubles with this kind of foreground-background imbalance data.

  1. Should the number of classes be 1 or 2? If it's 1, I need to change the foreground pixel in the label to 0 and the background pixel to 1, isn't it?
  2. In the metrcics.py, I found that you use the mask to remove the background. However, I realized that if my prediction for the foreground is full of the image, I can easily get the mIoU of 100%. The reason is that there is only one class here and the mask will affect the accuracy of the class.

Thank you and waiting for your response.

chenwydj commented 4 years ago

Hi @hmchuong! Thank you for your interest!

  1. The number of classes should be two. You can keep the foreground as 1 and background as 0.
  2. For the mask, are you talking about this line? The mask here only filters out classes to be ignored, which should not affect your measurement here. If you do have an image with all pixels as foreground, yes you may get 100% IoU, but the overall accuracy will be average over the whole dataset.

Hope that helps!

hmchuong commented 4 years ago

Thanks for your response, I got the mIoU as 0.68 after retraining. However, the prediction mask on the Aerial dataset is not good

Prediction chicago1_mask Groundtruth chicago1

Would you mind sharing your weight so that I can evaluate and compare it with mine?

chenwydj commented 4 years ago

Due to the time being I may not be able to clean the weight files for the Inria dataset.

Is your result from both the global and local branch? (i.e. mode = 3)

hmchuong commented 4 years ago

Due to the time being I may not be able to clean the weight files for the Inria dataset.

Is your result from both the global and local branch? (i.e. mode = 3)

Yes, the mode =3.

chenwydj commented 4 years ago

We used size_g = size_p = 536 in our experiments for Inria. Maybe 508 is small which may cause loss of details on the boundaries of patches.

hmchuong commented 4 years ago

Ok, let's I try this. Thank you

chenwydj commented 4 years ago

Also, you may want to check the intermediate results from the global branch. Though it may not be good, it should predict at least major building areas, leaving only some minor ones.