gavincmartin
commented
5 years ago The huge speed disparity between mask and box architectures displayed here appears similar to that seen in #6254
Closed gavincmartin closed 4 years ago
gavincmartin
commented
5 years ago The huge speed disparity between mask and box architectures displayed here appears similar to that seen in #6254
jeansitr
commented
5 years ago The problem here is that you reload your tensors for each image when you really only need to load them once. Plus, you resize the mask tensor every time wich take a lot of processing power on my end at least. And since your images are resized to always be the same size, you can do this instead wich will defitely reduce the time it takes to run the algorithm:
def run_inference_for_single_image(image, session, tensor_dict, image_tensor):
# Run inference
output_dict = sess.run(
tensor_dict, feed_dict={image_tensor: np.expand_dims(image, 0)})
# all outputs are float32 numpy arrays, so convert types as appropriate
output_dict['num_detections'] = int(output_dict['num_detections'][0])
output_dict['detection_classes'] = output_dict['detection_classes'][
0].astype(np.uint8)
output_dict['detection_boxes'] = output_dict['detection_boxes'][0]
output_dict['detection_scores'] = output_dict['detection_scores'][0]
if 'detection_masks' in output_dict:
output_dict['detection_masks'] = output_dict['detection_masks'][0]
return output_dict
MODEL_NAMES = [
'ssd_inception_v2_coco_2018_01_28',
'mask_rcnn_inception_v2_coco_2018_01_28'
]
for MODEL_NAME in MODEL_NAMES:
print("Model Name: {}".format(MODEL_NAME))
MODEL_FILE = MODEL_NAME + '.tar.gz'
DOWNLOAD_BASE = 'http://download.tensorflow.org/models/object_detection/'
# Path to frozen detection graph. This is the actual model that is used for the object detection.
PATH_TO_FROZEN_GRAPH = MODEL_NAME + '/frozen_inference_graph.pb'
# List of the strings that is used to add correct label for each box.
PATH_TO_LABELS = os.path.join('data', 'mscoco_label_map.pbtxt')
if not os.path.exists(PATH_TO_FROZEN_GRAPH):
opener = urllib.request.URLopener()
opener.retrieve(DOWNLOAD_BASE + MODEL_FILE, MODEL_FILE)
tar_file = tarfile.open(MODEL_FILE)
for file in tar_file.getmembers():
file_name = os.path.basename(file.name)
if 'frozen_inference_graph.pb' in file_name:
tar_file.extract(file, os.getcwd())
detection_graph = tf.Graph()
with detection_graph.as_default():
od_graph_def = tf.GraphDef()
with tf.gfile.GFile(PATH_TO_FROZEN_GRAPH, 'rb') as fid:
serialized_graph = fid.read()
od_graph_def.ParseFromString(serialized_graph)
tf.import_graph_def(od_graph_def, name='')
category_index = label_map_util.create_category_index_from_labelmap(
PATH_TO_LABELS, use_display_name=True)
# For the sake of simplicity we will use only 2 images:
# If you want to test the code with your images, just add path to the images to the TEST_IMAGE_PATHS.
PATH_TO_TEST_IMAGES_DIR = 'test_images'
TEST_IMAGE_PATHS = []
for file in os.scandir(PATH_TO_TEST_IMAGES_DIR):
if file.name.endswith(".jpg"):
TEST_IMAGE_PATHS.append(
os.path.join(PATH_TO_TEST_IMAGES_DIR, file.name))
with detection_graph.as_default():
with tf.Session() as sess:
tensor_dict = {}
image_tensor = ''
for i, image_path in enumerate(TEST_IMAGE_PATHS):
image = Image.open(image_path)
image = image.resize((640, 480))
# the array based representation of the image will be used later in order to prepare the
# result image with boxes and labels on it.
image_np = load_image_into_numpy_array(image)
#this will only load the tensors and resize the masks once wich saves a lot of processing power
if i == 0:
# Get handles to input and output tensors
ops = tf.get_default_graph().get_operations()
all_tensor_names = {output.name for op in ops for output in op.outputs}
for key in [
'num_detections', 'detection_boxes', 'detection_scores',
'detection_classes', 'detection_masks'
]:
tensor_name = key + ':0'
if tensor_name in all_tensor_names:
tensor_dict[key] = tf.get_default_graph().get_tensor_by_name(
tensor_name)
if 'detection_masks' in tensor_dict:
# The following processing is only for single image
detection_boxes = tf.squeeze(tensor_dict['detection_boxes'], [0])
detection_masks = tf.squeeze(tensor_dict['detection_masks'], [0])
# Reframe is required to translate mask from box coordinates to image coordinates and fit the image size.
real_num_detection = tf.cast(tensor_dict['num_detections'][0],
tf.int32)
detection_boxes = tf.slice(detection_boxes, [0, 0],
[real_num_detection, -1])
detection_masks = tf.slice(detection_masks, [0, 0, 0],
[real_num_detection, -1, -1])
detection_masks_reframed = utils_ops.reframe_box_masks_to_image_masks(
detection_masks, detection_boxes, image_np.shape[0], image_np.shape[1])
detection_masks_reframed = tf.cast(
tf.greater(detection_masks_reframed, 0.5), tf.uint8)
# Follow the convention by adding back the batch dimension
tensor_dict['detection_masks'] = tf.expand_dims(
detection_masks_reframed, 0)
image_tensor = tf.get_default_graph().get_tensor_by_name('image_tensor:0')
# Expand dimensions since the model expects images to have shape: [1, None, None, 3]
image_np_expanded = np.expand_dims(image_np, axis=0)
# Actual detection.
start = time.time()
output_dict = run_inference_for_single_image(image_np, sess, tensor_dict, image_tensor)
end = time.time()
print("Inference time {}: {:.4f}".format(i, end - start))
# Visualization of the results of a detection.
vis_util.visualize_boxes_and_labels_on_image_array(
image_np,
output_dict['detection_boxes'],
output_dict['detection_classes'],
output_dict['detection_scores'],
category_index,
instance_masks=output_dict.get('detection_masks'),
use_normalized_coordinates=True,
line_thickness=8)Since I've done it this way I haven't seen the error "axis attribute is out of bounds: 0".
tensorflowbutler
commented
4 years ago Hi There, We are checking to see if you still need help on this, as this seems to be an old issue. Please update this issue with the latest information, code snippet to reproduce your issue and error you are seeing. If we don't hear from you in the next 7 days, this issue will be closed automatically. If you don't need help on this issue any more, please consider closing this.
System information
Describe the problem
I am encountering two odd (and possibly related) behaviors when attempting to run the "mask_rcnn_inception_v2_coco_2018_01_28" model.
Source code / logs
I have modified the object_detection_tutorial.ipynb code slightly to run multiple models sequentially for comparison in script form. I also added a few additional images to the test_images directory within object_detection. This script compares the performance of SSD_Inception_V2 vs Mask_RCNN_Inception_V2 and produces the error described above.
This produces the following console output:
I am running the script from within a venv that has the following dependencies installed:
Interestingly enough, if I downgrade to TensorFlow 1.12 and run this script again, the speeds are still problematic, but the ArithmeticOptimizer error goes away: