0 Loss and Accuracy while fine-tuning from BVLC CaffeNet

[gavinmh]

Hello,

Thank you for sharing this work. I am attempting to fine-tune a triplet network from BVLC CaffeNet, but the loss and accuracy are rapidly becoming 0, as follows. I decreased the base learning rate to 3e-05. I expect the weights from CaffeNet to be copied for all but the last inner product layer, which I renamed. I doubled the learning rate multiplier for that layer.

I have tried various base learning rates, layer learning rate multipliers, and margins, but the network always reaches 0 accuracy and 0 loss (sadly, it is not a perfectly inaccurate classifier). Any tips for working with your NaiveTripletLoss layer would be appreciated.

I0512 19:44:51.484844 22728 caffe.cpp:219] Starting Optimization
I0512 19:44:51.484886 22728 solver.cpp:280] Solving nlm-pir-triplet
I0512 19:44:51.484891 22728 solver.cpp:281] Learning Rate Policy: fixed
I0512 19:44:51.486361 22728 solver.cpp:338] Iteration 0, Testing net (#0)
I0512 19:44:51.924820 22728 blocking_queue.cpp:50] Data layer prefetch queue empty
I0512 19:44:54.936336 22728 solver.cpp:406]     Test net output #0: accuracy = 0.424
I0512 19:44:54.936384 22728 solver.cpp:406]     Test net output #1: loss = 0.955602 (* 1 = 0.955602 loss)
I0512 19:44:55.095875 22728 solver.cpp:229] Iteration 0, loss = 0.0382693
I0512 19:44:55.095921 22728 solver.cpp:245]     Train net output #0: accuracy = 0.01
I0512 19:44:55.095937 22728 solver.cpp:245]     Train net output #1: loss = 0.0382693 (* 1 = 0.0382693 loss)
I0512 19:44:55.095952 22728 sgd_solver.cpp:106] Iteration 0, lr = 3e-05
I0512 19:45:03.676632 22728 solver.cpp:229] Iteration 20, loss = 0
I0512 19:45:03.676671 22728 solver.cpp:245]     Train net output #0: accuracy = 0
I0512 19:45:03.676684 22728 solver.cpp:245]     Train net output #1: loss = 0 (* 1 = 0 loss)
I0512 19:45:03.676697 22728 sgd_solver.cpp:106] Iteration 20, lr = 3e-05
I0512 19:45:13.121083 22728 solver.cpp:229] Iteration 40, loss = 0
I0512 19:45:13.121331 22728 solver.cpp:245]     Train net output #0: accuracy = 0
I0512 19:45:13.121345 22728 solver.cpp:245]     Train net output #1: loss = 0 (* 1 = 0 loss)
I0512 19:45:13.121352 22728 sgd_solver.cpp:106] Iteration 40, lr = 3e-05
I0512 19:45:22.479270 22728 solver.cpp:229] Iteration 60, loss = 0
I0512 19:45:22.479313 22728 solver.cpp:245]     Train net output #0: accuracy = 0
I0512 19:45:22.479326 22728 solver.cpp:245]     Train net output #1: loss = 0 (* 1 = 0 loss)
I0512 19:45:22.479333 22728 sgd_solver.cpp:106] Iteration 60, lr = 3e-05
I0512 19:45:31.958029 22728 solver.cpp:229] Iteration 80, loss = 0
I0512 19:45:31.958067 22728 solver.cpp:245]     Train net output #0: accuracy = 0
I0512 19:45:31.958083 22728 solver.cpp:245]     Train net output #1: loss = 0 (* 1 = 0 loss)
I0512 19:45:31.958091 22728 sgd_solver.cpp:106] Iteration 80, lr = 3e-05
I0512 19:45:41.431917 22728 solver.cpp:229] Iteration 100, loss = 0
I0512 19:45:41.431968 22728 solver.cpp:245]     Train net output #0: accuracy = 0
I0512 19:45:41.431980 22728 solver.cpp:245]     Train net output #1: loss = 0 (* 1 = 0 loss)
I0512 19:45:41.431988 22728 sgd_solver.cpp:106] Iteration 100, lr = 3e-05
I0512 19:45:50.903672 22728 solver.cpp:229] Iteration 120, loss = 0
I0512 19:45:50.903766 22728 solver.cpp:245]     Train net output #0: accuracy = 0
I0512 19:45:50.903779 22728 solver.cpp:245]     Train net output #1: loss = 0 (* 1 = 0 loss)
I0512 19:45:50.903786 22728 sgd_solver.cpp:106] Iteration 120, lr = 3e-05
I0512 19:46:00.355679 22728 solver.cpp:229] Iteration 140, loss = 0
I0512 19:46:00.355726 22728 solver.cpp:245]     Train net output #0: accuracy = 0
I0512 19:46:00.355744 22728 solver.cpp:245]     Train net output #1: loss = 0 (* 1 = 0 loss)
I0512 19:46:00.355752 22728 sgd_solver.cpp:106] Iteration 140, lr = 3e-05
I0512 19:46:10.074535 22728 solver.cpp:229] Iteration 160, loss = 0
I0512 19:46:10.074635 22728 solver.cpp:245]     Train net output #0: accuracy = 0
I0512 19:46:10.074672 22728 solver.cpp:245]     Train net output #1: loss = 0 (* 1 = 0 loss)
I0512 19:46:10.074702 22728 sgd_solver.cpp:106] Iteration 160, lr = 3e-05
I0512 19:46:19.850801 22728 solver.cpp:229] Iteration 180, loss = 0
I0512 19:46:19.850841 22728 solver.cpp:245]     Train net output #0: accuracy = 0
I0512 19:46:19.850853 22728 solver.cpp:245]     Train net output #1: loss = 0 (* 1 = 0 loss)
I0512 19:46:19.850862 22728 sgd_solver.cpp:106] Iteration 180, lr = 3e-05

The following are my train_val.prototxt and solver.prototxt. train_val.prototxt https://gist.github.com/gavinmh/f302f20963bfbb7e1a78876efa8e08ff solver.prototxt https://gist.github.com/gavinmh/19948e5026a98c68d85737a63505fe10

[gavinmh]

Nevermind, the TripletImageData transform parameters were incorrect.

[WaVEV]

Hello. I have got the same problem What went wrong? Are the current train_val.prototxt and solver.prototxt corrected?