search

facebookarchive / C3D

C3D is a modified version of BVLC caffe to support 3D ConvNets.
Other
1.16k stars 507 forks source link

High Loss? #392

Open guoswang opened 6 years ago

guoswang commented 6 years ago

Hi, When I run the "C3D-v1.1/examples/c3d_ucf101_training/train_simple.sh", I get the high loss like this:

I0525 14:53:43.037530 28821 net.cpp:200] data does not need backward computation.
I0525 14:53:43.037534 28821 net.cpp:242] This network produces output loss
I0525 14:53:43.037562 28821 net.cpp:255] Network initialization done.
I0525 14:53:43.037791 28821 solver.cpp:56] Solver scaffolding done.
I0525 14:53:43.038456 28821 caffe.cpp:248] Starting Optimization
I0525 14:53:43.038466 28821 solver.cpp:273] Solving C3D-UCF101Net
I0525 14:53:43.038470 28821 solver.cpp:274] Learning Rate Policy: step
I0525 14:53:44.757114 28821 solver.cpp:219] Iteration 0 (8.32475e-29 iter/s, 1.71857s/20 iters), loss = 6.07532
I0525 14:53:44.757179 28821 solver.cpp:238]     Train net output #0: loss = 6.07532 (* 1 = 6.07532 loss)
I0525 14:53:44.757207 28821 sgd_solver.cpp:105] Iteration 0, lr = 0.01
I0525 14:53:51.803372 28821 blocking_queue.cpp:49] Waiting for data
I0525 14:54:17.870946 28821 solver.cpp:219] Iteration 20 (0.603976 iter/s, 33.1139s/20 iters), loss = 87.3365
I0525 14:54:17.873360 28821 solver.cpp:238]     Train net output #0: loss = 87.3365 (* 1 = 87.3365 loss)
I0525 14:54:17.873409 28821 sgd_solver.cpp:105] Iteration 20, lr = 0.01
I0525 14:54:50.903730 28821 solver.cpp:219] Iteration 40 (0.605503 iter/s, 33.0304s/20 iters), loss = 87.3365
I0525 14:54:50.906200 28821 solver.cpp:238]     Train net output #0: loss = 87.3365 (* 1 = 87.3365 loss)
I0525 14:54:50.906235 28821 sgd_solver.cpp:105] Iteration 40, lr = 0.01
I0525 14:55:24.096009 28821 solver.cpp:219] Iteration 60 (0.602593 iter/s, 33.1899s/20 iters), loss = 87.3365

I don't know why. Can you help me ?

wuxiubozuishuai commented 6 years ago

Hello, can I be friends with you? I want to seek your help. How did you compile C3D? I often report the following error ：F0913 08:53:10.102206 7468 math_functions.cu:222] Check failed: status == CURAND_STATUS_SUCCESS (201 vs.0）CURAND_STATUS_LAUNCH_FAILURE
but my caffe is ok and I can extracting 3D features but I can't iterate