pingjun18-li
commented
7 years ago I also encountered the same problem as you, have you solved it?If you have solved, ask me how to do please！
Closed montego7878 closed 7 years ago
pingjun18-li
commented
7 years ago I also encountered the same problem as you, have you solved it?If you have solved, ask me how to do please！
Cysu
commented
7 years ago @montego7878 @pingjunLi Please clone this repo recursively with
git clone --recursive https://github.com/Cysu/dgd_person_reid.gitand then compile the external/caffe.
Also please install the lmdb python package with
pip install lmdbExtracting train set
E1228 15:59:45.808471 20895 extract_features.cpp:54] Using GPU
E1228 15:59:45.808827 20895 extract_features.cpp:60] Using Device_id=1
F1228 15:59:47.935324 20895 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not found: external/exp/snapshots/individually/prid_iter_11000.caffemodel
Check failure stack trace:
@ 0x7fbdf8cd5daa (unknown)
@ 0x7fbdf8cd5ce4 (unknown)
@ 0x7fbdf8cd56e6 (unknown)
@ 0x7fbdf8cd8687 (unknown)
@ 0x7fbdf9073fb3 caffe::ReadProtoFromBinaryFile()
@ 0x7fbdf906e214 caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7fbdf914b9c7 caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7fbdf914ba36 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x4080b8 feature_extraction_pipeline<>()
@ 0x7fbdf8105f45 (unknown)
@ 0x403bce (unknown)
@ (nil) (unknown)
scripts/routines.sh: 行 93: 20895 已放弃 (核心已转储) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 1
Extracting val set
E1228 15:59:50.017843 20935 extract_features.cpp:54] Using GPU
E1228 15:59:50.018328 20935 extract_features.cpp:60] Using Device_id=1
F1228 15:59:52.139801 20935 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not found: external/exp/snapshots/individually/prid_iter_11000.caffemodel
Check failure stack trace:
@ 0x7f28511bbdaa (unknown)
@ 0x7f28511bbce4 (unknown)
@ 0x7f28511bb6e6 (unknown)
@ 0x7f28511be687 (unknown)
@ 0x7f2851559fb3 caffe::ReadProtoFromBinaryFile()
@ 0x7f2851554214 caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7f28516319c7 caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7f2851631a36 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x4080b8 feature_extraction_pipeline<>()
@ 0x7f28505ebf45 (unknown)
@ 0x403bce (unknown)
@ (nil) (unknown)
scripts/routines.sh: 行 93: 20935 已放弃 (核心已转储) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 1
Extracting test_probe set
E1228 15:59:54.131114 20955 extract_features.cpp:54] Using GPU
E1228 15:59:54.131534 20955 extract_features.cpp:60] Using Device_id=1
F1228 15:59:56.317889 20955 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not found: external/exp/snapshots/individually/prid_iter_11000.caffemodel
Check failure stack trace:
@ 0x7f5501ed8daa (unknown)
@ 0x7f5501ed8ce4 (unknown)
@ 0x7f5501ed86e6 (unknown)
@ 0x7f5501edb687 (unknown)
@ 0x7f5502276fb3 caffe::ReadProtoFromBinaryFile()
@ 0x7f5502271214 caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7f550234e9c7 caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7f550234ea36 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x4080b8 feature_extraction_pipeline<>()
@ 0x7f5501308f45 (unknown)
@ 0x403bce (unknown)
@ (nil) (unknown)
scripts/routines.sh: 行 93: 20955 已放弃 (核心已转储) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 1
Extracting test_gallery set
E1228 15:59:58.303810 20973 extract_features.cpp:54] Using GPU
E1228 15:59:58.304404 20973 extract_features.cpp:60] Using Device_id=1
F1228 16:00:00.515791 20973 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not found: external/exp/snapshots/individually/prid_iter_11000.caffemodel
Check failure stack trace:
@ 0x7fa3d0d80daa (unknown)
@ 0x7fa3d0d80ce4 (unknown)
@ 0x7fa3d0d806e6 (unknown)
@ 0x7fa3d0d83687 (unknown)
@ 0x7fa3d111efb3 caffe::ReadProtoFromBinaryFile()
@ 0x7fa3d1119214 caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7fa3d11f69c7 caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7fa3d11f6a36 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x4080b8 feature_extraction_pipeline<>()
@ 0x7fa3d01b0f45 (unknown)
@ 0x403bce (unknown)
@ (nil) (unknown)
scripts/routines.sh: 行 93: 20973 已放弃 (核心已转储) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 1
Traceback (most recent call last):
File "eval/metric_learning.py", line 104, in
Cysu
commented
7 years ago It seems that the error is Check failed: error == cudaSuccess (2 vs. 0) out of memory. May I have your GPU configurations with nvidia-smi?
Cysu
commented
7 years ago It seems that the openmpi has something wrong. Could you please show the results of the following commands?
which mpirun
mpirun --version
ldd external/caffe/build/tools/caffe | grep mpiAlso please include your outputs with proper Markdown syntax. Thank you.
montego7878
commented
7 years ago I change to use one GPU ,but it still have error
I0110 18:01:07.894806 19573 solver.cpp:373] Test net output #0: accuracy = 0.00526316
I0110 18:01:07.894840 19573 solver.cpp:373] Test net output #1: loss = 5.95449 (* 1 = 5.95449 loss)
F0110 18:01:08.005344 19573 syncedmem.cpp:51] Check failed: error == cudaSuccess (2 vs. 0) out of memory
Check failure stack trace:
@ 0x7f1d188d5daa (unknown)
@ 0x7f1d188d5ce4 (unknown)
@ 0x7f1d188d56e6 (unknown)
@ 0x7f1d188d8687 (unknown)
@ 0x7f1d18fa809b caffe::SyncedMemory::mutable_gpu_data()
@ 0x7f1d18fb5352 caffe::Blob<>::mutable_gpu_data()
@ 0x7f1d1901ae40 caffe::CuDNNConvolutionLayer<>::Forward_gpu()
@ 0x7f1d18f950ba caffe::Net<>::ForwardFromTo()
@ 0x7f1d18f95437 caffe::Net<>::ForwardPrefilled()
@ 0x7f1d18ff6ab0 caffe::Solver<>::Step()
@ 0x7f1d18ff7254 caffe::Solver<>::Solve()
@ 0x409fc0 train()
@ 0x408196 main
@ 0x7f1d17be3f45 (unknown)
@ 0x40882f (unknown)
@ (nil) (unknown)
Extracting train set
E0110 18:01:08.168467 19640 extract_features.cpp:54] Using GPU
E0110 18:01:08.168717 19640 extract_features.cpp:60] Using Device_id=0
F0110 18:01:18.770076 19640 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not found: external/exp/snapshots/individually/prid_iter_11000.caffemodel
Check failure stack trace:
@ 0x7f34e718cdaa (unknown)
@ 0x7f34e718cce4 (unknown)
@ 0x7f34e718c6e6 (unknown)
@ 0x7f34e718f687 (unknown)
@ 0x7f34e762ca03 caffe::ReadProtoFromBinaryFile()
@ 0x7f34e76433a4 caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7f34e75f6647 caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7f34e75f66b6 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x4080b8 feature_extraction_pipeline<>()
@ 0x7f34e65bcf45 (unknown)
@ 0x403bce (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 19640 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Extracting val set
E0110 18:01:19.980247 19655 extract_features.cpp:54] Using GPU
E0110 18:01:19.980475 19655 extract_features.cpp:60] Using Device_id=0
F0110 18:01:30.597614 19655 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not found: external/exp/snapshots/individually/prid_iter_11000.caffemodel
Check failure stack trace:
@ 0x7f18657e8daa (unknown)
@ 0x7f18657e8ce4 (unknown)
@ 0x7f18657e86e6 (unknown)
@ 0x7f18657eb687 (unknown)
@ 0x7f1865c88a03 caffe::ReadProtoFromBinaryFile()
@ 0x7f1865c9f3a4 caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7f1865c52647 caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7f1865c526b6 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x4080b8 feature_extraction_pipeline<>()
@ 0x7f1864c18f45 (unknown)
@ 0x403bce (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 19655 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Extracting test_probe set
E0110 18:01:31.850750 19672 extract_features.cpp:54] Using GPU
E0110 18:01:31.850977 19672 extract_features.cpp:60] Using Device_id=0
F0110 18:01:42.372356 19672 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not found: external/exp/snapshots/individually/prid_iter_11000.caffemodel
Check failure stack trace:
@ 0x7f29ac09ddaa (unknown)
@ 0x7f29ac09dce4 (unknown)
@ 0x7f29ac09d6e6 (unknown)
@ 0x7f29ac0a0687 (unknown)
@ 0x7f29ac53da03 caffe::ReadProtoFromBinaryFile()
@ 0x7f29ac5543a4 caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7f29ac507647 caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7f29ac5076b6 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x4080b8 feature_extraction_pipeline<>()
@ 0x7f29ab4cdf45 (unknown)
@ 0x403bce (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 19672 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Extracting test_gallery set
E0110 18:01:43.553402 19710 extract_features.cpp:54] Using GPU
E0110 18:01:43.553637 19710 extract_features.cpp:60] Using Device_id=0
F0110 18:01:54.306596 19710 io.cpp:52] Check failed: fd != -1 (-1 vs. -1) File not found: external/exp/snapshots/individually/prid_iter_11000.caffemodel
Check failure stack trace:
@ 0x7f7e6d9a4daa (unknown)
@ 0x7f7e6d9a4ce4 (unknown)
@ 0x7f7e6d9a46e6 (unknown)
@ 0x7f7e6d9a7687 (unknown)
@ 0x7f7e6de44a03 caffe::ReadProtoFromBinaryFile()
@ 0x7f7e6de5b3a4 caffe::ReadNetParamsFromBinaryFileOrDie()
@ 0x7f7e6de0e647 caffe::Net<>::CopyTrainedLayersFromBinaryProto()
@ 0x7f7e6de0e6b6 caffe::Net<>::CopyTrainedLayersFrom()
@ 0x4080b8 feature_extraction_pipeline<>()
@ 0x7f7e6cdd4f45 (unknown)
@ 0x403bce (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 19710 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Traceback (most recent call last):
File "eval/metric_learning.py", line 104, in
montego7878
commented
7 years ago My nvidia-smi
Cysu
commented
7 years ago It says Check failed: error == cudaSuccess (2 vs. 0) out of memory. You may reduce the batch_size and increase the iter_size in solvers for compensation.
montego7878
commented
7 years ago I reduce batch_size and increase the iter_size in solvers it success Snapshotting to binary proto file external/exp/snapshots/individually/prid_iter_100.caffemodel and Snapshotting solver state to binary proto fileexternal/exp/snapshots/individually/prid_iter_100.solverstate but in the Extract train, val, test probe, and test gallery features is error
Extracting train set
E0111 16:45:52.847748 30130 extract_features.cpp:54] Using GPU
E0111 16:45:52.847967 30130 extract_features.cpp:60] Using Device_id=0
E0111 16:46:03.843616 30130 extract_features.cpp:135] Extacting Features
F0111 16:46:04.056460 30130 syncedmem.cpp:51] Check failed: error == cudaSuccess (2 vs. 0) out of memory
Check failure stack trace:
@ 0x7f2250083daa (unknown)
@ 0x7f2250083ce4 (unknown)
@ 0x7f22500836e6 (unknown)
@ 0x7f2250086687 (unknown)
@ 0x7f22504de25b caffe::SyncedMemory::mutable_gpu_data()
@ 0x7f22504eb512 caffe::Blob<>::mutable_gpu_data()
@ 0x7f225054a32a caffe::BNLayer<>::Forward_gpu()
@ 0x7f22504cacb9 caffe::Net<>::ForwardFromTo()
@ 0x7f22504cb105 caffe::Net<>::ForwardPrefilled()
@ 0x408a63 feature_extraction_pipeline<>()
@ 0x7f224f4b3f45 (unknown)
@ 0x403bce (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 30130 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Extracting val set
E0111 16:46:09.042475 30146 extract_features.cpp:54] Using GPU
E0111 16:46:09.042714 30146 extract_features.cpp:60] Using Device_id=0
E0111 16:46:20.222183 30146 extract_features.cpp:135] Extacting Features
F0111 16:46:20.436499 30146 syncedmem.cpp:51] Check failed: error == cudaSuccess (2 vs. 0) out of memory
Check failure stack trace:
@ 0x7f0bdd2e7daa (unknown)
@ 0x7f0bdd2e7ce4 (unknown)
@ 0x7f0bdd2e76e6 (unknown)
@ 0x7f0bdd2ea687 (unknown)
@ 0x7f0bdd74225b caffe::SyncedMemory::mutable_gpu_data()
@ 0x7f0bdd74f512 caffe::Blob<>::mutable_gpu_data()
@ 0x7f0bdd7ae32a caffe::BNLayer<>::Forward_gpu()
@ 0x7f0bdd72ecb9 caffe::Net<>::ForwardFromTo()
@ 0x7f0bdd72f105 caffe::Net<>::ForwardPrefilled()
@ 0x408a63 feature_extraction_pipeline<>()
@ 0x7f0bdc717f45 (unknown)
@ 0x403bce (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 30146 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Extracting test_probe set
E0111 16:46:21.580566 30161 extract_features.cpp:54] Using GPU
E0111 16:46:21.580788 30161 extract_features.cpp:60] Using Device_id=0
E0111 16:46:32.858496 30161 extract_features.cpp:135] Extacting Features
F0111 16:46:33.071115 30161 syncedmem.cpp:51] Check failed: error == cudaSuccess (2 vs. 0) out of memory
Check failure stack trace:
@ 0x7efc54f17daa (unknown)
@ 0x7efc54f17ce4 (unknown)
@ 0x7efc54f176e6 (unknown)
@ 0x7efc54f1a687 (unknown)
@ 0x7efc5537225b caffe::SyncedMemory::mutable_gpu_data()
@ 0x7efc5537f512 caffe::Blob<>::mutable_gpu_data()
@ 0x7efc553de32a caffe::BNLayer<>::Forward_gpu()
@ 0x7efc5535ecb9 caffe::Net<>::ForwardFromTo()
@ 0x7efc5535f105 caffe::Net<>::ForwardPrefilled()
@ 0x408a63 feature_extraction_pipeline<>()
@ 0x7efc54347f45 (unknown)
@ 0x403bce (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 30161 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Extracting test_gallery set
E0111 16:46:34.218818 30176 extract_features.cpp:54] Using GPU
E0111 16:46:34.219038 30176 extract_features.cpp:60] Using Device_id=0
E0111 16:46:45.814304 30176 extract_features.cpp:135] Extacting Features
F0111 16:46:46.028051 30176 syncedmem.cpp:51] Check failed: error == cudaSuccess (2 vs. 0) out of memory
Check failure stack trace:
@ 0x7faa9a149daa (unknown)
@ 0x7faa9a149ce4 (unknown)
@ 0x7faa9a1496e6 (unknown)
@ 0x7faa9a14c687 (unknown)
@ 0x7faa9a5a425b caffe::SyncedMemory::mutable_gpu_data()
@ 0x7faa9a5b1512 caffe::Blob<>::mutable_gpu_data()
@ 0x7faa9a61032a caffe::BNLayer<>::Forward_gpu()
@ 0x7faa9a590cb9 caffe::Net<>::ForwardFromTo()
@ 0x7faa9a591105 caffe::Net<>::ForwardPrefilled()
@ 0x408a63 feature_extraction_pipeline<>()
@ 0x7faa99579f45 (unknown)
@ 0x403bce (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 30176 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Traceback (most recent call last):
File "eval/metric_learning.py", line 104, in
Cysu
commented
7 years ago montego7878
commented
7 years ago @Cysu Thanks very much.
montego7878@montego7878-All-Series:~/下載/dgd_person_reid/scripts$ ./exp_individually.sh prid
[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.SolverParameter: 16:7: Message type "caffe.SolverParameter" has no field named "min_lr". [libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.SolverParameter: 16:7: Message type "caffe.SolverParameter" has no field named "min_lr". F1208 19:46:38.284919 17113 upgrade_proto.cpp:1095] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse SolverParameter file: models/individually/prid_solver.prototxt F1208 19:46:38.284919 17114 upgrade_proto.cpp:1095] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse SolverParameter file: models/individually/prid_solver.prototxt Check failure stack trace: Check failure stack trace: @ 0x7fbc2053edaa (unknown) @ 0x7fc90f42ddaa (unknown) @ 0x7fbc2053ece4 (unknown) @ 0x7fc90f42dce4 (unknown) @ 0x7fbc2053e6e6 (unknown) @ 0x7fc90f42d6e6 (unknown) @ 0x7fbc20541687 (unknown) @ 0x7fc90f430687 (unknown) @ 0x7fbc20cbcf3e caffe::ReadSolverParamsFromTextFileOrDie() @ 0x407f14 train() @ 0x405b3c main @ 0x7fc90fbabf3e caffe::ReadSolverParamsFromTextFileOrDie() @ 0x407f14 train() @ 0x405b3c main @ 0x7fbc1f54af45 (unknown) @ 0x4063ab (unknown) @ 0x7fc90e439f45 (unknown) @ 0x4063ab (unknown) @ (nil) (unknown) @ (nil) (unknown)
mpirun noticed that process rank 0 with PID 17113 on node montego7878-All-Series exited on signal 6 (Aborted).
2 total processes killed (some possibly by mpirun during cleanup) Extracting train set E1208 19:46:38.383947 17138 extract_features.cpp:52] Using GPU E1208 19:46:38.384215 17138 extract_features.cpp:58] Using Device_id=0 [libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 7:46: Message type "caffe.TransformationParameter" has no field named "crop_height". F1208 19:46:38.587419 17138 upgrade_proto.cpp:88] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /tmp/tmp.S4fBxURPtd Check failure stack trace: @ 0x7fa7e766cdaa (unknown) @ 0x7fa7e766cce4 (unknown) @ 0x7fa7e766c6e6 (unknown) @ 0x7fa7e766f687 (unknown) @ 0x7fa7e7b8698e caffe::ReadNetParamsFromTextFileOrDie() @ 0x7fa7e7b6e61f caffe::Net<>::Net() @ 0x408cf3 feature_extraction_pipeline<>() @ 0x7fa7e6898f45 (unknown) @ 0x4048be (unknown) @ (nil) (unknown) scripts/routines.sh: line 93: 17138 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0 Traceback (most recent call last): File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Extracting val set
E1208 19:46:38.741158 17152 extract_features.cpp:52] Using GPU
E1208 19:46:38.741426 17152 extract_features.cpp:58] Using Device_id=0
[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 7:46: Message type "caffe.TransformationParameter" has no field named "crop_height".
F1208 19:46:38.930703 17152 upgrade_proto.cpp:88] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /tmp/tmp.KOZuJPVKch
Check failure stack trace:
@ 0x7fdb7dc05daa (unknown)
@ 0x7fdb7dc05ce4 (unknown)
@ 0x7fdb7dc056e6 (unknown)
@ 0x7fdb7dc08687 (unknown)
@ 0x7fdb7e11f98e caffe::ReadNetParamsFromTextFileOrDie()
@ 0x7fdb7e10761f caffe::Net<>::Net()
@ 0x408cf3 feature_extraction_pipeline<>()
@ 0x7fdb7ce31f45 (unknown)
@ 0x4048be (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 17152 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Extracting test_probe set
E1208 19:46:39.085443 17166 extract_features.cpp:52] Using GPU
E1208 19:46:39.085688 17166 extract_features.cpp:58] Using Device_id=0
[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 7:46: Message type "caffe.TransformationParameter" has no field named "crop_height".
F1208 19:46:39.277931 17166 upgrade_proto.cpp:88] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /tmp/tmp.MnmAclhImz
Check failure stack trace:
@ 0x7f5029da6daa (unknown)
@ 0x7f5029da6ce4 (unknown)
@ 0x7f5029da66e6 (unknown)
@ 0x7f5029da9687 (unknown)
@ 0x7f502a2c098e caffe::ReadNetParamsFromTextFileOrDie()
@ 0x7f502a2a861f caffe::Net<>::Net()
@ 0x408cf3 feature_extraction_pipeline<>()
@ 0x7f5028fd2f45 (unknown)
@ 0x4048be (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 17166 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Extracting test_gallery set
E1208 19:46:39.435348 17181 extract_features.cpp:52] Using GPU
E1208 19:46:39.435585 17181 extract_features.cpp:58] Using Device_id=0
[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 7:46: Message type "caffe.TransformationParameter" has no field named "crop_height".
F1208 19:46:39.623297 17181 upgrade_proto.cpp:88] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /tmp/tmp.oZi1WOYwfm
Check failure stack trace:
@ 0x7fac5ef6edaa (unknown)
@ 0x7fac5ef6ece4 (unknown)
@ 0x7fac5ef6e6e6 (unknown)
@ 0x7fac5ef71687 (unknown)
@ 0x7fac5f48898e caffe::ReadNetParamsFromTextFileOrDie()
@ 0x7fac5f47061f caffe::Net<>::Net()
@ 0x408cf3 feature_extraction_pipeline<>()
@ 0x7fac5e19af45 (unknown)
@ 0x4048be (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 17181 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Traceback (most recent call last):
File "eval/metric_learning.py", line 104, in
main(args)
File "eval/metric_learning.py", line 73, in main
X, Y = _get_train_data(args.result_dir)
File "eval/metric_learning.py", line 11, in _get_traindata
features = np.r[np.load(osp.join(result_dir, 'train_features.npy')),
File "/home/montego7878/anaconda2/lib/python2.7/site-packages/numpy/lib/npyio.py", line 362, in load
fid = open(file, "rb")
IOError: [Errno 2] No such file or directory: 'external/exp/results/individually/prid_prid_iter_11000_fc7_bn/train_features.npy'
Extracting train set
E1208 19:46:39.992875 17211 extract_features.cpp:52] Using GPU
E1208 19:46:39.993129 17211 extract_features.cpp:58] Using Device_id=0
[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 7:46: Message type "caffe.TransformationParameter" has no field named "crop_height".
F1208 19:46:40.181699 17211 upgrade_proto.cpp:88] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /tmp/tmp.6rvyXNylk5
Check failure stack trace:
@ 0x7f2c5fdb0daa (unknown)
@ 0x7f2c5fdb0ce4 (unknown)
@ 0x7f2c5fdb06e6 (unknown)
@ 0x7f2c5fdb3687 (unknown)
@ 0x7f2c602ca98e caffe::ReadNetParamsFromTextFileOrDie()
@ 0x7f2c602b261f caffe::Net<>::Net()
@ 0x408cf3 feature_extraction_pipeline<>()
@ 0x7f2c5efdcf45 (unknown)
@ 0x4048be (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 17211 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Extracting val set
E1208 19:46:40.338579 17225 extract_features.cpp:52] Using GPU
E1208 19:46:40.338822 17225 extract_features.cpp:58] Using Device_id=0
[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 7:46: Message type "caffe.TransformationParameter" has no field named "crop_height".
F1208 19:46:40.527302 17225 upgrade_proto.cpp:88] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /tmp/tmp.2fKuobB8uG
Check failure stack trace:
@ 0x7f2b7aa3cdaa (unknown)
@ 0x7f2b7aa3cce4 (unknown)
@ 0x7f2b7aa3c6e6 (unknown)
@ 0x7f2b7aa3f687 (unknown)
@ 0x7f2b7af5698e caffe::ReadNetParamsFromTextFileOrDie()
@ 0x7f2b7af3e61f caffe::Net<>::Net()
@ 0x408cf3 feature_extraction_pipeline<>()
@ 0x7f2b79c68f45 (unknown)
@ 0x4048be (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 17225 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Extracting test_probe set
E1208 19:46:40.683516 17239 extract_features.cpp:52] Using GPU
E1208 19:46:40.683779 17239 extract_features.cpp:58] Using Device_id=0
[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 7:46: Message type "caffe.TransformationParameter" has no field named "crop_height".
F1208 19:46:40.879796 17239 upgrade_proto.cpp:88] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /tmp/tmp.7Yctcxn0J1
Check failure stack trace:
@ 0x7f19b94e7daa (unknown)
@ 0x7f19b94e7ce4 (unknown)
@ 0x7f19b94e76e6 (unknown)
@ 0x7f19b94ea687 (unknown)
@ 0x7f19b9a0198e caffe::ReadNetParamsFromTextFileOrDie()
@ 0x7f19b99e961f caffe::Net<>::Net()
@ 0x408cf3 feature_extraction_pipeline<>()
@ 0x7f19b8713f45 (unknown)
@ 0x4048be (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 17239 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Extracting test_gallery set
E1208 19:46:41.036201 17253 extract_features.cpp:52] Using GPU
E1208 19:46:41.036490 17253 extract_features.cpp:58] Using Device_id=0
[libprotobuf ERROR google/protobuf/text_format.cc:245] Error parsing text-format caffe.NetParameter: 7:46: Message type "caffe.TransformationParameter" has no field named "crop_height".
F1208 19:46:41.226066 17253 upgrade_proto.cpp:88] Check failed: ReadProtoFromTextFile(param_file, param) Failed to parse NetParameter file: /tmp/tmp.yAlDCC7flD
Check failure stack trace:
@ 0x7f4ea9623daa (unknown)
@ 0x7f4ea9623ce4 (unknown)
@ 0x7f4ea96236e6 (unknown)
@ 0x7f4ea9626687 (unknown)
@ 0x7f4ea9b3d98e caffe::ReadNetParamsFromTextFileOrDie()
@ 0x7f4ea9b2561f caffe::Net<>::Net()
@ 0x408cf3 feature_extraction_pipeline<>()
@ 0x7f4ea884ff45 (unknown)
@ 0x4048be (unknown)
@ (nil) (unknown)
scripts/routines.sh: line 93: 17253 已經終止 (core dumped) ${CAFFE_DIR}/build/tools/extract_features ${trained_model} ${model} ${blob},label ${result_dir}/${subset}_features_lmdb,${result_dir}/${subset}_labels_lmdb ${num_iters} lmdb GPU 0
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Traceback (most recent call last):
File "tools/convert_lmdb_to_numpy.py", line 2, in
import lmdb
ImportError: No module named lmdb
Traceback (most recent call last):
File "eval/metric_learning.py", line 104, in
main(args)
File "eval/metric_learning.py", line 73, in main
X, Y = _get_train_data(args.result_dir)
File "eval/metric_learning.py", line 11, in _get_traindata
features = np.r[np.load(osp.join(result_dir, 'train_features.npy')),
File "/home/montego7878/anaconda2/lib/python2.7/site-packages/numpy/lib/npyio.py", line 362, in load
fid = open(file, "rb")
IOError: [Errno 2] No such file or directory: 'external/exp/results/individually/prid_prid_iter_11000_fc7_bn/train_features.npy'