search

alexgkendall / caffe-segnet

Implementation of SegNet: A Deep Convolutional Encoder-Decoder Architecture for Semantic Pixel-Wise Labelling
http://mi.eng.cam.ac.uk/projects/segnet/
Other
1.08k stars 452 forks source link

Train on NYUD and the result is strange #102

Closed sj-li closed 7 years ago

sj-li commented 7 years ago

Hello, I train the Segnet on the NYUD dataset(http://cs.nyu.edu/~silberman/datasets/nyu_depth_v2.html) only using color map.

The loss is under(just for example, I iterate for more time but result is similar)

on NYUD dataset:

I0316 09:21:53.649056 18818 solver.cpp:214] Iteration 120, loss = 2.39523
I0316 09:21:53.649134 18818 solver.cpp:229]     Train net output #0: accuracy = 0.527354
I0316 09:21:53.649150 18818 solver.cpp:229]     Train net output #1: loss = 2.39523 (* 1 = 2.39523 loss)
I0316 09:21:53.649173 18818 solver.cpp:229]     Train net output #2: per_class_accuracy = 0
I0316 09:21:53.649178 18818 solver.cpp:229]     Train net output #3: per_class_accuracy = 1
I0316 09:21:53.649181 18818 solver.cpp:229]     Train net output #4: per_class_accuracy = 0.995979
I0316 09:21:53.649186 18818 solver.cpp:229]     Train net output #5: per_class_accuracy = 0
I0316 09:21:53.649189 18818 solver.cpp:229]     Train net output #6: per_class_accuracy = 0
I0316 09:21:53.649193 18818 solver.cpp:229]     Train net output #7: per_class_accuracy = 0
I0316 09:21:53.649196 18818 solver.cpp:229]     Train net output #8: per_class_accuracy = 1
I0316 09:21:53.649199 18818 solver.cpp:229]     Train net output #9: per_class_accuracy = 1
I0316 09:21:53.649202 18818 solver.cpp:229]     Train net output #10: per_class_accuracy = 1
I0316 09:21:53.649205 18818 solver.cpp:229]     Train net output #11: per_class_accuracy = 0
I0316 09:21:53.649209 18818 solver.cpp:229]     Train net output #12: per_class_accuracy = 0.000391568
I0316 09:21:53.649214 18818 solver.cpp:486] Iteration 120, lr = 0.0005
I0316 09:22:05.837713 18818 solver.cpp:214] Iteration 140, loss = 2.02869
I0316 09:22:05.837749 18818 solver.cpp:229]     Train net output #0: accuracy = 0.558421
I0316 09:22:05.837764 18818 solver.cpp:229]     Train net output #1: loss = 2.02869 (* 1 = 2.02869 loss)
I0316 09:22:05.837769 18818 solver.cpp:229]     Train net output #2: per_class_accuracy = 0
I0316 09:22:05.837774 18818 solver.cpp:229]     Train net output #3: per_class_accuracy = 1
I0316 09:22:05.837779 18818 solver.cpp:229]     Train net output #4: per_class_accuracy = 0.997134
I0316 09:22:05.837783 18818 solver.cpp:229]     Train net output #5: per_class_accuracy = 0
I0316 09:22:05.837788 18818 solver.cpp:229]     Train net output #6: per_class_accuracy = 0
I0316 09:22:05.837792 18818 solver.cpp:229]     Train net output #7: per_class_accuracy = 0
I0316 09:22:05.837796 18818 solver.cpp:229]     Train net output #8: per_class_accuracy = 1
I0316 09:22:05.837800 18818 solver.cpp:229]     Train net output #9: per_class_accuracy = 0
I0316 09:22:05.837805 18818 solver.cpp:229]     Train net output #10: per_class_accuracy = 1
I0316 09:22:05.837808 18818 solver.cpp:229]     Train net output #11: per_class_accuracy = 0
I0316 09:22:05.837812 18818 solver.cpp:229]     Train net output #12: per_class_accuracy = 0.000467126
I0316 09:22:05.837818 18818 solver.cpp:486] Iteration 140, lr = 0.0005
I0316 09:22:52.982698 18818 solver.cpp:214] Iteration 160, loss = 1.01879
I0316 09:22:52.982781 18818 solver.cpp:229]     Train net output #0: accuracy = 0.648835
I0316 09:22:52.982800 18818 solver.cpp:229]     Train net output #1: loss = 1.01879 (* 1 = 1.01879 loss)
I0316 09:22:52.982805 18818 solver.cpp:229]     Train net output #2: per_class_accuracy = 0
I0316 09:22:52.982810 18818 solver.cpp:229]     Train net output #3: per_class_accuracy = 1
I0316 09:22:52.982816 18818 solver.cpp:229]     Train net output #4: per_class_accuracy = 0.997722
I0316 09:22:52.982821 18818 solver.cpp:229]     Train net output #5: per_class_accuracy = 0
I0316 09:22:52.982826 18818 solver.cpp:229]     Train net output #6: per_class_accuracy = 0
I0316 09:22:52.982831 18818 solver.cpp:229]     Train net output #7: per_class_accuracy = 0
I0316 09:22:52.982836 18818 solver.cpp:229]     Train net output #8: per_class_accuracy = 1
I0316 09:22:52.982841 18818 solver.cpp:229]     Train net output #9: per_class_accuracy = 1
I0316 09:22:52.982844 18818 solver.cpp:229]     Train net output #10: per_class_accuracy = 0
I0316 09:22:52.982848 18818 solver.cpp:229]     Train net output #11: per_class_accuracy = 1
I0316 09:22:52.982853 18818 solver.cpp:229]     Train net output #12: per_class_accuracy = 0.000341763
I0316 09:22:52.982859 18818 solver.cpp:486] Iteration 160, lr = 0.0005

on CamVid dataset:

I0316 09:26:25.223773 19297 solver.cpp:229]     Train net output #0: accuracy = 0.0719136
I0316 09:26:25.223793 19297 solver.cpp:229]     Train net output #1: loss = 1.25843 (* 1 = 1.25843 loss)
I0316 09:26:25.223808 19297 solver.cpp:229]     Train net output #2: per_class_accuracy = 0.0829579
I0316 09:26:25.223814 19297 solver.cpp:229]     Train net output #3: per_class_accuracy = 0.0178376
I0316 09:26:25.223819 19297 solver.cpp:229]     Train net output #4: per_class_accuracy = 0.0369708
I0316 09:26:25.223824 19297 solver.cpp:229]     Train net output #5: per_class_accuracy = 0.0863952
I0316 09:26:25.223829 19297 solver.cpp:229]     Train net output #6: per_class_accuracy = 0.0261603
I0316 09:26:25.223834 19297 solver.cpp:229]     Train net output #7: per_class_accuracy = 0.126695
I0316 09:26:25.223840 19297 solver.cpp:229]     Train net output #8: per_class_accuracy = 0.812235
I0316 09:26:25.223846 19297 solver.cpp:229]     Train net output #9: per_class_accuracy = 0.00071048
I0316 09:26:25.223852 19297 solver.cpp:229]     Train net output #10: per_class_accuracy = 0.00387442
I0316 09:26:25.223858 19297 solver.cpp:229]     Train net output #11: per_class_accuracy = 0.0167264
I0316 09:26:25.223865 19297 solver.cpp:229]     Train net output #12: per_class_accuracy = 0
I0316 09:26:25.223884 19297 solver.cpp:486] Iteration 180, lr = 0.001
I0316 09:26:45.770525 19297 solver.cpp:214] Iteration 200, loss = 1.01808
I0316 09:26:45.770599 19297 solver.cpp:229]     Train net output #0: accuracy = 0.053885
I0316 09:26:45.770619 19297 solver.cpp:229]     Train net output #1: loss = 1.01808 (* 1 = 1.01808 loss)
I0316 09:26:45.770623 19297 solver.cpp:229]     Train net output #2: per_class_accuracy = 0.102944
I0316 09:26:45.770628 19297 solver.cpp:229]     Train net output #3: per_class_accuracy = 0.0224036
I0316 09:26:45.770632 19297 solver.cpp:229]     Train net output #4: per_class_accuracy = 0.0235294
I0316 09:26:45.770637 19297 solver.cpp:229]     Train net output #5: per_class_accuracy = 0.0632904
I0316 09:26:45.770642 19297 solver.cpp:229]     Train net output #6: per_class_accuracy = 0.0216417
I0316 09:26:45.770645 19297 solver.cpp:229]     Train net output #7: per_class_accuracy = 0.14127
I0316 09:26:45.770649 19297 solver.cpp:229]     Train net output #8: per_class_accuracy = 0.774158
I0316 09:26:45.770653 19297 solver.cpp:229]     Train net output #9: per_class_accuracy = 1
I0316 09:26:45.770658 19297 solver.cpp:229]     Train net output #10: per_class_accuracy = 0.00418701
I0316 09:26:45.770663 19297 solver.cpp:229]     Train net output #11: per_class_accuracy = 0.0129621
I0316 09:26:45.770668 19297 solver.cpp:229]     Train net output #12: per_class_accuracy = 0.000427168
I0316 09:26:45.770673 19297 solver.cpp:486] Iteration 200, lr = 0.001
I0316 09:27:06.446229 19297 solver.cpp:214] Iteration 220, loss = 1.27126
I0316 09:27:06.446261 19297 solver.cpp:229]     Train net output #0: accuracy = 0.0848843
I0316 09:27:06.446271 19297 solver.cpp:229]     Train net output #1: loss = 1.27126 (* 1 = 1.27126 loss)
I0316 09:27:06.446275 19297 solver.cpp:229]     Train net output #2: per_class_accuracy = 0.172025
I0316 09:27:06.446280 19297 solver.cpp:229]     Train net output #3: per_class_accuracy = 0.0288163
I0316 09:27:06.446283 19297 solver.cpp:229]     Train net output #4: per_class_accuracy = 0.0475551
I0316 09:27:06.446286 19297 solver.cpp:229]     Train net output #5: per_class_accuracy = 0.118193
I0316 09:27:06.446290 19297 solver.cpp:229]     Train net output #6: per_class_accuracy = 0.0368427
I0316 09:27:06.446293 19297 solver.cpp:229]     Train net output #7: per_class_accuracy = 0.106813
I0316 09:27:06.446296 19297 solver.cpp:229]     Train net output #8: per_class_accuracy = 0.664721
I0316 09:27:06.446300 19297 solver.cpp:229]     Train net output #9: per_class_accuracy = 5.05051e-05
I0316 09:27:06.446303 19297 solver.cpp:229]     Train net output #10: per_class_accuracy = 0.00724959
I0316 09:27:06.446307 19297 solver.cpp:229]     Train net output #11: per_class_accuracy = 0.0235964
I0316 09:27:06.446310 19297 solver.cpp:229]     Train net output #12: per_class_accuracy = 0
I0316 09:27:06.446316 19297 solver.cpp:486] Iteration 220, lr = 0.001

Not like on CamVid, on NYUD, the train result is just 0 or 1. And when I test the result, the whole has the same label.

And the only thing I change the train prototxt is dims of unsampling layer to fit 320x240 resolution :

layer {
  name: "upsample5"
  type: "Upsample"
  bottom: "pool5"
  top: "pool5_D"
  bottom: "pool5_mask"
  upsample_param {
    scale: 2
    upsample_w: 20
    upsample_h: 15
  }
}
layer {
  name: "upsample4"
  type: "Upsample"
  bottom: "conv5_1_D"
  top: "pool4_D"
  bottom: "pool4_mask"
  upsample_param {
    scale: 2
    upsample_w: 40
    upsample_h: 30
  }
}

Thanks!

cartosquare commented 7 years ago

I have similar output as you, is this ok?

sandeepvr40 commented 7 years ago

@freedomabcd I am trying to modify segnet_basic to 2 class classification using KITTI data set (http://www.cvlibs.net/datasets/kitti/eval_road.php). I have ended up with the same result as yours (Per class accuracy = 0 or 1 and predicted image has only one class throughout). The only changes I have made so far are:

  1. num_output = 2
  2. weight_by_label_freqs: True ignore_label: 0 # I have classes 0 and 1 in my ground truth class_weighting: 0.3780 class_weighting: 0.7382 If you have solved the issue by now, could you please advice me on how you solved it?