alfonsolink / tf_rfcn

experimental tensorflow implementation of R-FCN
41 stars 15 forks source link

exceptions.TypeError: 'numpy.float64' object cannot be interpreted as an index #6

Open zhangleiedu opened 7 years ago

zhangleiedu commented 7 years ago

(tensorflow) zlei@l02482:~/tensorflow/tf_rfcn-master/tf_rfcn_dynamic$ python resnet_rfcn_v2.py /home/zlei/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/gradients_impl.py:93: UserWarning: Converting sparse IndexedSlices to a dense Tensor of unknown shape. This may consume a large amount of memory. "Converting sparse IndexedSlices to a dense Tensor of unknown shape. " WARNING:tensorflow:From resnet_rfcn_v2.py:296: all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02. Instructions for updating: Please use tf.global_variables instead. WARNING:tensorflow:From resnet_rfcn_v2.py:297: all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02. Instructions for updating: Please use tf.global_variables instead. WARNING:tensorflow:From resnet_rfcn_v2.py:298: all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02. Instructions for updating: Please use tf.global_variables instead. WARNING:tensorflow:From /home/zlei/anaconda2/lib/python2.7/site-packages/tensorflow/python/util/tf_should_use.py:170: initialize_all_variables (from tensorflow.python.ops.variables) is deprecated and will be removed after 2017-03-02. Instructions for updating: Use tf.global_variables_initializer instead. 2017-07-05 10:00:58.656417: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.1 instructions, but these are available on your machine and could speed up CPU computations. 2017-07-05 10:00:58.656439: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use SSE4.2 instructions, but these are available on your machine and could speed up CPU computations. 2017-07-05 10:00:58.656447: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX instructions, but these are available on your machine and could speed up CPU computations. 2017-07-05 10:00:58.656454: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use AVX2 instructions, but these are available on your machine and could speed up CPU computations. 2017-07-05 10:00:58.656460: W tensorflow/core/platform/cpu_feature_guard.cc:45] The TensorFlow library wasn't compiled to use FMA instructions, but these are available on your machine and could speed up CPU computations. 2017-07-05 10:00:59.437122: I tensorflow/core/common_runtime/gpu/gpu_device.cc:938] Found device 0 with properties: name: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate (GHz) 1.7335 pciBusID 0000:02:00.0 Total memory: 7.92GiB Free memory: 2.76GiB 2017-07-05 10:00:59.708112: W tensorflow/stream_executor/cuda/cuda_driver.cc:523] A non-primary context 0xa9895b0 exists before initializing the StreamExecutor. We haven't verified StreamExecutor works with that. 2017-07-05 10:00:59.708794: I tensorflow/core/common_runtime/gpu/gpu_device.cc:938] Found device 1 with properties: name: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate (GHz) 1.7335 pciBusID 0000:03:00.0 Total memory: 7.92GiB Free memory: 6.31GiB 2017-07-05 10:00:59.959091: W tensorflow/stream_executor/cuda/cuda_driver.cc:523] A non-primary context 0xa98cf30 exists before initializing the StreamExecutor. We haven't verified StreamExecutor works with that. 2017-07-05 10:00:59.959800: I tensorflow/core/common_runtime/gpu/gpu_device.cc:938] Found device 2 with properties: name: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate (GHz) 1.7335 pciBusID 0000:05:00.0 Total memory: 7.92GiB Free memory: 5.97GiB 2017-07-05 10:01:00.096731: W tensorflow/stream_executor/cuda/cuda_driver.cc:523] A non-primary context 0xa9908b0 exists before initializing the StreamExecutor. We haven't verified StreamExecutor works with that. 2017-07-05 10:01:00.097414: I tensorflow/core/common_runtime/gpu/gpu_device.cc:938] Found device 3 with properties: name: GeForce GTX 1080 major: 6 minor: 1 memoryClockRate (GHz) 1.7335 pciBusID 0000:06:00.0 Total memory: 7.92GiB Free memory: 5.78GiB 2017-07-05 10:01:00.103593: I tensorflow/core/common_runtime/gpu/gpu_device.cc:959] DMA: 0 1 2 3 2017-07-05 10:01:00.103613: I tensorflow/core/common_runtime/gpu/gpu_device.cc:969] 0: Y Y Y Y 2017-07-05 10:01:00.103620: I tensorflow/core/common_runtime/gpu/gpu_device.cc:969] 1: Y Y Y Y 2017-07-05 10:01:00.103626: I tensorflow/core/common_runtime/gpu/gpu_device.cc:969] 2: Y Y Y Y 2017-07-05 10:01:00.103632: I tensorflow/core/common_runtime/gpu/gpu_device.cc:969] 3: Y Y Y Y 2017-07-05 10:01:00.103643: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1028] Creating TensorFlow device (/gpu:0) -> (device: 0, name: GeForce GTX 1080, pci bus id: 0000:02:00.0) 2017-07-05 10:01:00.103652: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1028] Creating TensorFlow device (/gpu:1) -> (device: 1, name: GeForce GTX 1080, pci bus id: 0000:03:00.0) 2017-07-05 10:01:00.103663: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1028] Creating TensorFlow device (/gpu:2) -> (device: 2, name: GeForce GTX 1080, pci bus id: 0000:05:00.0) 2017-07-05 10:01:00.103669: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1028] Creating TensorFlow device (/gpu:3) -> (device: 3, name: GeForce GTX 1080, pci bus id: 0000:06:00.0) epoch: 0 iter: 0 2017-07-05 10:04:36.911663: W tensorflow/core/common_runtime/bfc_allocator.cc:217] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.05GiB. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory is available. 2017-07-05 10:04:36.944670: W tensorflow/core/common_runtime/bfc_allocator.cc:217] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.05GiB. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory is available. 2017-07-05 10:04:36.971146: W tensorflow/core/common_runtime/bfc_allocator.cc:217] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.07GiB. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory is available. 2017-07-05 10:04:38.531113: W tensorflow/core/common_runtime/bfc_allocator.cc:217] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.07GiB. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory is available. 2017-07-05 10:04:38.570674: W tensorflow/core/common_runtime/bfc_allocator.cc:217] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.06GiB. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory is available. 2017-07-05 10:04:38.645047: W tensorflow/core/common_runtime/bfc_allocator.cc:217] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.05GiB. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory is available. 2017-07-05 10:04:38.680857: W tensorflow/core/common_runtime/bfc_allocator.cc:217] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.03GiB. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory is available. 2017-07-05 10:04:38.761680: W tensorflow/core/common_runtime/bfc_allocator.cc:217] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.05GiB. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory is available.

model saved

2017-07-05 10:04:44.078761: W tensorflow/core/common_runtime/bfc_allocator.cc:217] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.05GiB. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory is available. 2017-07-05 10:04:44.142480: W tensorflow/core/common_runtime/bfc_allocator.cc:217] Allocator (GPU_0_bfc) ran out of memory trying to allocate 2.05GiB. The caller indicates that this is not a failure, but may mean that there could be performance gains if more memory is available. 2017-07-05 10:04:46.467171: I tensorflow/core/common_runtime/gpu/pool_allocator.cc:247] PoolAllocator: After 2001 get requests, put_count=1800 evicted_count=1000 eviction_rate=0.555556 and unsatisfied allocation rate=0.650175 2017-07-05 10:04:46.467208: I tensorflow/core/common_runtime/gpu/pool_allocator.cc:259] Raising pool_sizelimit from 100 to 110 2017-07-05 10:04:53.937866: W tensorflow/core/framework/op_kernel.cc:1165] Invalid argument: exceptions.TypeError: 'numpy.float64' object cannot be interpreted as an index 2017-07-05 10:04:53.938044: W tensorflow/core/framework/op_kernel.cc:1165] Invalid argument: exceptions.TypeError: 'numpy.float64' object cannot be interpreted as an index [[Node: PyFunc_5 = PyFunc[Tin=[DT_FLOAT, DT_INT64], Tout=[DT_INT32, DT_INT64, DT_FLOAT, DT_FLOAT, DT_FLOAT], token="pyfunc_5", _device="/job:localhost/replica:0/task:0/cpu:0"](Reshape_12/_747, Reshape_1/_699)]] Traceback (most recent call last): File "resnet_rfcn_v2.py", line 344, in feed_dict = {lr : learn_rate, gate : [0.0], prep_img : img_train, gt_box : gt_train, im_info : im_info_x, im_height : im_height_x, im_width : im_width_x}) File "/home/zlei/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 894, in run run_metadata_ptr) File "/home/zlei/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1106, in _run feed_dict_tensor, options, run_metadata) File "/home/zlei/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1259, in _do_run options, run_metadata) File "/home/zlei/anaconda2/lib/python2.7/site-packages/tensorflow/python/client/session.py", line 1278, in _do_call raise type(e)(node_def, op, message) tensorflow.python.framework.errors_impl.InvalidArgumentError: exceptions.TypeError: 'numpy.float64' object cannot be interpreted as an index [[Node: PyFunc_5 = PyFunc[Tin=[DT_FLOAT, DT_INT64], Tout=[DT_INT32, DT_INT64, DT_FLOAT, DT_FLOAT, DT_FLOAT], token="pyfunc_5", _device="/job:localhost/replica:0/task:0/cpu:0"](Reshape_12/_747, Reshape_1/_699)]]

Caused by op u'PyFunc_5', defined at: File "resnet_rfcn_v2.py", line 216, in tf.py_func(proposal_target, [rpn_rois, gt_boxbatch], [tf.int32, tf.int64, tf.float32, tf.float32, tf.float32]) File "/home/zlei/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/script_ops.py", line 203, in py_func input=inp, token=token, Tout=Tout, name=name) File "/home/zlei/anaconda2/lib/python2.7/site-packages/tensorflow/python/ops/gen_script_ops.py", line 38, in _py_func name=name) File "/home/zlei/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py", line 767, in apply_op op_def=op_def) File "/home/zlei/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 2528, in create_op original_op=self._default_original_op, op_def=op_def) File "/home/zlei/anaconda2/lib/python2.7/site-packages/tensorflow/python/framework/ops.py", line 1203, in init self._traceback = self._graph._extract_stack() # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): exceptions.TypeError: 'numpy.float64' object cannot be interpreted as an index [[Node: PyFunc_5 = PyFunc[Tin=[DT_FLOAT, DT_INT64], Tout=[DT_INT32, DT_INT64, DT_FLOAT, DT_FLOAT, DT_FLOAT], token="pyfunc_5", _device="/job:localhost/replica:0/task:0/cpu:0"](Reshape_12/_747, Reshape_1/_699)]]

I encounter above error, when run the code [python resnet_rfcn_v2.py], can anyone solve for me? thanks. bwt, when i use roidb_maker-master/roidb_maker_pascal_xml/pre_proc.py code generate roidb.pkl, the roidb.pkl is very small, which only about 10M, is it normal? thanks