search

chuckcho / video-caffe

Video-friendly caffe -- comes with the most recent version of Caffe (as of Jan 2019), a video reader, 3D(ND) pooling layer, and an example training script for C3D network and UCF-101 data
Other
175 stars 93 forks source link

Differences between master and refactor branches #75

Closed zhuolinumd closed 7 years ago

zhuolinumd commented 7 years ago

I want to know that what is the main difference for the master and refactor branches? Which one do you suggestion to run?

As you mentioned in the readme file, "One missing feature in the refactor branch (yet) is the python wrapper." Can I run the script in c3d_ucf101/check_data_and_modal.py when I am using the refactor branch?

chuckcho commented 7 years ago

check_data_and_modal.py won't work. It needs some changes in datum definition (https://github.com/chuckcho/video-caffe/blob/master/src/caffe/proto/caffe.proto#L33-L44), memory_data_layer, python codes (_caffe.cpp, etc). Refactor branch is cleaner implementation as "Convolution" / "Pooling" layers are extended for N-d data in general (whereas master branch has separate "NdConvolution" "NdPooling" layers).

zhuolinumd commented 7 years ago

@chuckcho have you fully tested the refactor branch? I have trained the model on the UCF101 dataset using the refactor branch. The clip-level accuracy is not stable. Please see the logs below. Is this because the testing data is different for each evaluation (shuffle: true)?

2896 I1123 02:20:48.354784 77153 solver.cpp:337] Iteration 10000, Testing net (#0) 2897 I1123 02:21:52.356096 77153 blocking_queue.cpp:50] Data layer prefetch queue empty 2898 I1123 02:22:37.424468 77153 solver.cpp:404] Test net output #0: accuracy/top-1 = 0.26 2899 I1123 02:22:37.424598 77153 solver.cpp:404] Test net output #1: accuracy/top-5 = 0.552 2900 I1123 02:22:37.424612 77153 solver.cpp:404] Test net output #2: loss = 3.13017 (* 1 = 3.13017 loss)

4451 I1123 12:52:02.362048 77153 solver.cpp:337] Iteration 20000, Testing net (#0) 4452 I1123 12:53:29.074837 77153 solver.cpp:404] Test net output #0: accuracy/top-1 = 0.392 4453 I1123 12:53:29.075105 77153 solver.cpp:404] Test net output #1: accuracy/top-5 = 0.666 4454 I1123 12:53:29.075119 77153 solver.cpp:404] Test net output #2: loss = 2.79081 (* 1 = 2.79081 loss)

7561 I1124 07:13:30.872079 77153 solver.cpp:337] Iteration 40000, Testing net (#0) 7562 I1124 07:14:58.231197 77153 solver.cpp:404] Test net output #0: accuracy/top-1 = 0.426 7563 I1124 07:14:58.231420 77153 solver.cpp:404] Test net output #1: accuracy/top-5 = 0.672 7564 I1124 07:14:58.231433 77153 solver.cpp:404] Test net output #2: loss = 2.88083 (* 1 = 2.88083 loss)

16579 I1126 13:08:15.460868 77153 solver.cpp:337] Iteration 98000, Testing net (#0) 16580 I1126 13:10:23.628275 77153 solver.cpp:404] Test net output #0: accuracy/top-1 = 0.456 16581 I1126 13:10:23.628458 77153 solver.cpp:404] Test net output #1: accuracy/top-5 = 0.686 16582 I1126 13:10:23.628470 77153 solver.cpp:404] Test net output #2: loss = 2.78102 (* 1 = 2.78102 loss)

22643 I1128 00:10:30.090528 77153 solver.cpp:337] Iteration 137000, Testing net (#0) 22644 I1128 00:12:35.989692 77153 solver.cpp:404] Test net output #0: accuracy/top-1 = 0.446 22645 I1128 00:12:35.989816 77153 solver.cpp:404] Test net output #1: accuracy/top-5 = 0.688 22646 I1128 00:12:35.989828 77153 solver.cpp:404] Test net output #2: loss = 2.80329 (* 1 = 2.80329 loss)

23111 I1128 02:41:27.571787 77153 solver.cpp:337] Iteration 140000, Testing net (#0) 23112 I1128 02:43:29.991505 77153 solver.cpp:404] Test net output #0: accuracy/top-1 = 0.418 23113 I1128 02:43:29.991668 77153 solver.cpp:404] Test net output #1: accuracy/top-5 = 0.69 23114 I1128 02:43:29.991681 77153 solver.cpp:404] Test net output #2: loss = 2.9161 (* 1 = 2.9161 loss)

23421 I1128 04:22:02.990334 77153 solver.cpp:337] Iteration 142000, Testing net (#0) 23422 I1128 04:24:07.995729 77153 solver.cpp:404] Test net output #0: accuracy/top-1 = 0.382 23423 I1128 04:24:07.995863 77153 solver.cpp:404] Test net output #1: accuracy/top-5 = 0.646

My last question is: Is the figure (https://raw.githubusercontent.com/chuckcho/video-caffe/master/examples/c3d_ucf101/c3d_ucf101_train_loss_accuracy.png) generated using refactor or master?

Thanks for your time.

chuckcho commented 7 years ago

Was generated using master branch. Can you plot your accuracy?

On Fri, Dec 9, 2016 at 4:10 PM, jiang2764 notifications@github.com wrote:

@chuckcho https://github.com/chuckcho have you fully tested the refactor branch? I have trained the model on the UCF101 dataset using the refactor branch. The clip-level accuracy is not stable. Please see the logs below. Is this because the testing data is different for each evaluation (shuffle: true)?

2896 I1123 02:20:48.354784 77153 solver.cpp:337] Iteration 10000, Testing net (#0) 2897 I1123 02:21:52.356096 77153 blocking_queue.cpp:50] Data layer prefetch queue empty 2898 I1123 02:22:37.424468 77153 solver.cpp:404] Test net output #0: accuracy/top-1 = 0.26 2899 I1123 02:22:37.424598 77153 solver.cpp:404] Test net output #1 https://github.com/chuckcho/video-caffe/issues/1: accuracy/top-5 = 0.552 2900 I1123 02:22:37.424612 77153 solver.cpp:404] Test net output #2 https://github.com/chuckcho/video-caffe/issues/2: loss = 3.13017 (* 1 = 3.13017 loss)

4451 I1123 12:52:02.362048 77153 solver.cpp:337] Iteration 20000, Testing net (#0) 4452 I1123 12:53:29.074837 77153 solver.cpp:404] Test net output #0: accuracy/top-1 = 0.392 4453 I1123 12:53:29.075105 77153 solver.cpp:404] Test net output #1 https://github.com/chuckcho/video-caffe/issues/1: accuracy/top-5 = 0.666 4454 I1123 12:53:29.075119 77153 solver.cpp:404] Test net output #2 https://github.com/chuckcho/video-caffe/issues/2: loss = 2.79081 (* 1 = 2.79081 loss)

7561 I1124 07:13:30.872079 77153 solver.cpp:337] Iteration 40000, Testing net (#0) 7562 I1124 07:14:58.231197 77153 solver.cpp:404] Test net output #0: accuracy/top-1 = 0.426 7563 I1124 07:14:58.231420 77153 solver.cpp:404] Test net output #1 https://github.com/chuckcho/video-caffe/issues/1: accuracy/top-5 = 0.672 7564 I1124 07:14:58.231433 77153 solver.cpp:404] Test net output #2 https://github.com/chuckcho/video-caffe/issues/2: loss = 2.88083 (* 1 = 2.88083 loss)

16579 I1126 13:08:15.460868 77153 solver.cpp:337] Iteration 98000, Testing net (#0) 16580 I1126 13:10:23.628275 77153 solver.cpp:404] Test net output #0: accuracy/top-1 = 0.456 16581 I1126 13:10:23.628458 77153 solver.cpp:404] Test net output #1 https://github.com/chuckcho/video-caffe/issues/1: accuracy/top-5 = 0.686 16582 I1126 13:10:23.628470 77153 solver.cpp:404] Test net output #2 https://github.com/chuckcho/video-caffe/issues/2: loss = 2.78102 (* 1 = 2.78102 loss)

22643 I1128 00:10:30.090528 77153 solver.cpp:337] Iteration 137000, Testing net (#0) 22644 I1128 00:12:35.989692 77153 solver.cpp:404] Test net output #0: accuracy/top-1 = 0.446 22645 I1128 00:12:35.989816 77153 solver.cpp:404] Test net output #1 https://github.com/chuckcho/video-caffe/issues/1: accuracy/top-5 = 0.688 22646 I1128 00:12:35.989828 77153 solver.cpp:404] Test net output #2 https://github.com/chuckcho/video-caffe/issues/2: loss = 2.80329 (* 1 = 2.80329 loss)

23111 I1128 02:41:27.571787 77153 solver.cpp:337] Iteration 140000, Testing net (#0) 23112 I1128 02:43:29.991505 77153 solver.cpp:404] Test net output #0: accuracy/top-1 = 0.418 23113 I1128 02:43:29.991668 77153 solver.cpp:404] Test net output #1 https://github.com/chuckcho/video-caffe/issues/1: accuracy/top-5 = 0.69 23114 I1128 02:43:29.991681 77153 solver.cpp:404] Test net output #2 https://github.com/chuckcho/video-caffe/issues/2: loss = 2.9161 (* 1 = 2.9161 loss)

23421 I1128 04:22:02.990334 77153 solver.cpp:337] Iteration 142000, Testing net (#0) 23422 I1128 04:24:07.995729 77153 solver.cpp:404] Test net output #0: accuracy/top-1 = 0.382 23423 I1128 04:24:07.995863 77153 solver.cpp:404] Test net output #1 https://github.com/chuckcho/video-caffe/issues/1: accuracy/top-5 = 0.646

My last question is: Is the figure (https://raw.githubusercontent.com/ chuckcho/video-caffe/master/examples/c3ducf101/c3d ucf101_train_loss_accuracy.png) generated using refactor or master?

Thanks for your time.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/chuckcho/video-caffe/issues/75#issuecomment-266124184, or mute the thread https://github.com/notifications/unsubscribe-auth/AMu8TJnR8JH0B7MiuGtX0mL2MJsB_k4Oks5rGcPHgaJpZM4LJRRN .

-- Chuck Cho @ Dextro.co chuck.cho@dextro.co

zhuolinumd commented 7 years ago

how to plot the accuracy? Is the refactor branch include the script?

chuckcho commented 7 years ago

https://github.com/chuckcho/video-caffe/blob/master/tools/extra/plot_training_loss.sh Run plot_training_loss.sh in the directory where log file is located. It's a bit hacky and it may need some effort to make a plot.