austinegri
commented
6 years ago I believe I am having an error like this as well.. I am running multi images on GPU and get a "slice index out of bounds" error
Open janismdhanbad opened 6 years ago
austinegri
commented
6 years ago I believe I am having an error like this as well.. I am running multi images on GPU and get a "slice index out of bounds" error
RyanLBWoods
commented
5 years ago I had the same "slice index out of bounds" error. When I change the IMAGE_PER_GPU to 8, the error was gone. I was trying to set the IMAGE_PER_GPU to 16. I have two machines, one is with two 12G TITAN X, the other is with four 12G GTX1080. On the TITAN machine, 16 images per GPU works fine, however, it has to be 8 on the GTX1080 machine. According to the numbers, I'm guessing there might be a limit of batch size, or a bug occurred if the batch size is bigger than 32. I didn't test more and I don't know why but I hope this may temporally help you solve the issue. Hope experts can solve it in a decent way.
jimmy15923
commented
5 years ago Same problem! Can't inference when batch size is over 32!
codexponent
commented
5 years ago Change config.IMAGES_PER_GPU = 1 config.GPU_COUNT = 1 config.BATCH_SIZE = 1 while testing! :)
codexponent
commented
5 years ago @jimmy15923 , Please follow the above instructions.
recallrui
commented
5 years ago Edit ./mrcnn/model.py (About line 2524):
detections, _, _, mrcnn_mask, _, _, _, *feature_maps =\
self.keras_model.predict([molded_images, image_metas, anchors], verbose=0)
results = []to:
self.keras_model.predict([molded_images, image_metas, anchors], batch_size=len(images), verbose=0)
I trained my model successfully and now I am trying to predict on images. I am specifying "IMAGES_PER_GPU" in InferenceConfig class. Till 32 "IMAGES_PER_GPU" there is no error but after that, it's giving me the error below. I checked some issues on tensorflow repository and I think it might be related to the input that is being passed. Also, the below error is giving an error in line 828 of utils.py
inputs_slice = [x[i] for x in inputs]
InvalidArgumentError Traceback (most recent call last) ~/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/tensorflow/python/client/session.py in _do_call(self, fn, args) 1321 try: -> 1322 return fn(args) 1323 except errors.OpError as e:
~/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/tensorflow/python/client/session.py in _run_fn(feed_dict, fetch_list, target_list, options, run_metadata) 1306 return self._call_tf_sessionrun( -> 1307 options, feed_dict, fetch_list, target_list, run_metadata) 1308
~/anaconda3/envs/tensorflow_p36/lib/python3.6/site-packages/tensorflow/python/client/session.py in _call_tf_sessionrun(self, options, feed_dict, fetch_list, target_list, run_metadata) 1408 self._session, options, feed_dict, fetch_list, target_list, -> 1409 run_metadata) 1410 else:
InvalidArgumentError: slice index 33 of dimension 0 out of bounds. [[Node: ROI_1/strided_slice_228 = StridedSlice[Index=DT_INT32, T=DT_FLOAT, begin_mask=0, ellipsis_mask=0, end_mask=0, new_axis_mask=0, shrink_axis_mask=1, _device="/job:localhost/replica:0/task:0/device:GPU:0"](_arg_input_anchors_1_0_0/_8681, mrcnn_detection_1/strided_slice_744/stack_1, mrcnn_detection_1/strided_slice_767/stack_1, mrcnn_mask_deconv_1/strided_slice/stack_1)]] [[Node: mrcnn_detection_1/map_24/while/Identity/_10586 = _Recv client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_19325_mrcnn_detection_1/map_24/while/Identity", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]]
During handling of the above exception, another exception occurred:
InvalidArgumentError Traceback (most recent call last)