search

msracver / Deep-Feature-Flow

Deep Feature Flow for Video Recognition
MIT License
1.3k stars 297 forks source link

Training Deep-Feature-Flow with pretraining models #15

Closed TerryLovesLife closed 7 years ago

TerryLovesLife commented 7 years ago

Hi Orpine,

Thank you for solving the previous issues. When I train deep feature flow with r-fcn on imagenet dataset, I encounter this problem. could you please give a favor? waiting online.......

---------------------------------- error log below -----------------------------------------------------

ImageNetVID_DET_train_30classes gt roidb loaded from ./data/cache/ImageNetVID_DET_train_30classes_gt_roidb.pkl append flipped images to roidb num_images 57834 ImageNetVID_VID_train_15frames gt roidb loaded from ./data/cache/ImageNetVID_VID_train_15frames_gt_roidb.pkl append flipped images to roidb filtered 3316 roidb entries: 222946 -> 219630 providing maximum shape [('data', (1, 3, 600, 1000)), ('data_ref', (1, 3, 600, 1000)), ('eq_flag', (1,)), ('gt_boxes', (1, 100, 5))] [('label', (1, 21546)), ('bbox_target', (1, 36, 38, 63)), ('bbox_weight', (1, 36, 38, 63))] {'bbox_target': (1L, 36L, 35L, 63L), 'bbox_weight': (1L, 36L, 35L, 63L), 'data': (1L, 3L, 556L, 1000L), 'data_ref': (1L, 3L, 556L, 1000L), 'eq_flag': (1L,), 'gt_boxes': (1L, 1L, 5L), 'im_info': (1L, 3L), 'label': (1L, 19845L)} ('lr', 0.00025, 'lr_epoch_diff', [1.333], 'lr_iters', [73191]) [14:46:31] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied. [14:46:31] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied. [14:46:31] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied. [14:46:31] src/operator/convolution.cu:87: This convolution is not supported by cudnn, MXNET convolution is applied. [14:46:31] /home/terry/mxnet/dmlc-core/include/dmlc/./logging.h:304: [14:46:31] /home/terry/mxnet/mshadow/mshadow/./tensor_gpu-inl.h:35: Check failed: e == cudaSuccess CUDA: invalid device ordinal

Stack trace returned 5 entries: [bt] (0) /usr/local/lib/python2.7/dist-packages/mxnet-0.10.1-py2.7.egg/mxnet/libmxnet.so(_ZN4dmlc15LogMessageFatalD1Ev+0x3c) [0x7f065d6c6d7c] [bt] (1) /usr/local/lib/python2.7/dist-packages/mxnet-0.10.1-py2.7.egg/mxnet/libmxnet.so(_ZNSt17_Function_handlerIFvvEZZN5mxnet6engine23ThreadedEnginePerDevice13PushToExecuteEPNS2_8OprBlockEbENKUlvE1_clEvEUlvE_E9_M_invokeERKSt9_Any_data+0x160) [0x7f065e48a650] [bt] (2) /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb8c80) [0x7f0679c9fc80] [bt] (3) /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba) [0x7f067c8826ba] [bt] (4) /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d) [0x7f067c5b83dd]

terminate called after throwing an instance of 'dmlc::Error' what(): [14:46:31] /home/terry/mxnet/mshadow/mshadow/./tensor_gpu-inl.h:35: Check failed: e == cudaSuccess CUDA: invalid device ordinal

----------------------------------------error log above------------------------------------------------

Thanks!

YuwenXiong commented 7 years ago

Make sure your GPU ID in config is correct, do you have enough GPU?