search

orsic / swiftnet

GNU General Public License v3.0
251 stars 54 forks source link

mIoU about "rn18_single_scale.py". #24

Open dxjundersky opened 4 years ago

dxjundersky commented 4 years ago

Thank you for sharing the train. I tried "python train.py configs/rn18_single_scale.py --store_dir=/path/to/store/experiments" but only get "Best mIoU: 69.45% (epoch 240)". The paper says the mIoU is 75.4% SwiftNetRN-18† val 75.4 39.9 39.3 2048x1024 104.0 52.0 11.8M

So, What do I need to pay attention to when training?

Results: Errors: road IoU accuracy = 62.86 % sidewalk IoU accuracy = 92.66 % building IoU accuracy = 69.72 % wall IoU accuracy = 87.80 % fence IoU accuracy = 37.58 % pole IoU accuracy = 49.45 % traffic light IoU accuracy = 58.32 % traffic sign IoU accuracy = 62.65 % vegetation IoU accuracy = 68.79 % terrain IoU accuracy = 90.06 % sky IoU accuracy = 56.03 % person IoU accuracy = 90.99 % rider IoU accuracy = 77.62 % car IoU accuracy = 60.92 % truck IoU accuracy = 92.69 % bus IoU accuracy = 60.97 % train IoU accuracy = 81.11 % motorcycle IoU accuracy = 68.40 % bicycle IoU accuracy = 50.04 % IoU mean class accuracy -> TP / (TP+FN+FP) = 69.40 % mean class recall -> TP / (TP+FN) = 79.06 % mean class precision -> TP / (TP+FP) = 83.77 % pixel accuracy = 91.32 % Best mIoU: 69.45% (epoch 240)

orsic commented 4 years ago

Hello @dxjundersky, thank you for trying out the training script. The mIoU should be fairly higher. Here are some suggestions which could help us debug this:

  1. What changes did you make to the repository?
  2. Did you make sure that your Cityscapes directory structure is the same as in the README?
  3. Did you download the pre-trained single scale model? Could you evaluate it using the instructions from the README?
orsic commented 4 years ago

One important thing I left out: the label files I use in my training are already remapped. To circumvent this, add the RemapLabels transform to both training and validation transforms.

dxjundersky commented 4 years ago

Oh...... The Road is 62.86 %. But the truck is 92.69 % Obviously, I haven't remapped the label file.

Thank you!

LaCandela commented 4 years ago

@orsic I was also missing a RemapLabels transform to reproduce the mIOU values, thank you very much for pointing it out. It might be meaningful to incorporate it to the main repo...

Thank you, again!