Batch inference is not working with exported model

[what2help]

Hi All,

Need help on how to inference images in batches. I get this error when I try with current exported model

ValueError: Cannot feed value of shape (1, 26, 218, 1) for Tensor 'Placeholder:0', which has shape '(?, ?, 1)'

Tried changing preprocess_image_for_prediction() in src/data_handler.py but getting one or the other error. Current code is

def serving_input_fn():
        # define placeholder for input image
        image = tf.placeholder(dtype=tf.float32, shape=[None, None, None, 1])
        print("image shape ", tf.shape(image))
        shape = tf.shape(image)
        print("Shape variable ", shape)
        # Assert shape is h x w x c with c = 1
        ratio = tf.divide(shape[2], shape[1])
        print("ratio ", ratio)
        increment = CONST.DIMENSION_REDUCTION_W_POOLING
        new_width = tf.cast(tf.round((ratio * fixed_height) / increment) * increment, tf.int32)
        print("new_width ", new_width)
        resized_image = tf.cond(new_width < tf.constant(min_width, dtype=tf.int32),
                                true_fn=lambda: tf.image.resize_images(image, size=(fixed_height, min_width)),
                                false_fn=lambda: tf.image.resize_images(image, size=(fixed_height, new_width))
                                )
        print("resized image shape ", tf.shape(resized_image))
        print("new_width[None] ", new_width[None])
        print("type of resized_input ", type(resized_image[None]))
        # Features to serve
        features = {'images': resized_image,  # cast to 1 x h x w x c
                    'images_widths': new_width[None]  # cast to tensor
                    }
        # Inputs received
        receiver_inputs = {'images': image}
        return tf.estimator.export.ServingInputReceiver(features, receiver_inputs)

and I get bellow error with this code.

Traceback (most recent call last): File "/home/ubuntu/.local/lib/python3.5/site-packages/tensorflow/python/framework/tensor_util.py", line 460, in make_tensor_proto str_values = [compat.as_bytes(x) for x in proto_values] File "/home/ubuntu/.local/lib/python3.5/site-packages/tensorflow/python/framework/tensor_util.py", line 460, in str_values = [compat.as_bytes(x) for x in proto_values] File "/home/ubuntu/.local/lib/python3.5/site-packages/tensorflow/python/util/compat.py", line 65, in as_bytes (bytes_or_text,)) TypeError: Expected binary or unicode string, got None During handling of the above exception, another exception occurred: Traceback (most recent call last): File "train.py", line 114, in preprocess_image_for_prediction(min_width=10)) File "/home/ubuntu/.local/lib/python3.5/site-packages/tensorflow/python/estimator/estimator.py", line 436, in export_savedmodel mode=model_fn_lib.ModeKeys.PREDICT) File "/home/ubuntu/.local/lib/python3.5/site-packages/tensorflow/python/estimator/estimator.py", line 615, in _call_model_fn model_fn_results = self._model_fn(features=features, **kwargs) File "/home/ubuntu/ctnn/tf-crnn/src/model.py", line 236, in crnn_fn conv = deep_cnn(features['images'], (mode == tf.estimator.ModeKeys.TRAIN), summaries=False) File "/home/ubuntu/ctnn/tf-crnn/src/model.py", line 160, in deep_cnn name='reshaped') # [batch, width, height x features] File "/home/ubuntu/.local/lib/python3.5/site-packages/tensorflow/python/ops/gen_array_ops.py", line 2619, in reshape name=name) File "/home/ubuntu/.local/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 493, in apply_op raise err File "/home/ubuntu/.local/lib/python3.5/site-packages/tensorflow/python/framework/op_def_library.py", line 490, in apply_op preferred_dtype=default_dtype) File "/home/ubuntu/.local/lib/python3.5/site-packages/tensorflow/python/framework/ops.py", line 676, in internal_convert_to_tensor ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref) File "/home/ubuntu/.local/lib/python3.5/site-packages/tensorflow/python/framework/constant_op.py", line 121, in _constant_tensor_conversion_function return constant(v, dtype=dtype, name=name) File "/home/ubuntu/.local/lib/python3.5/site-packages/tensorflow/python/framework/constant_op.py", line 102, in constant tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape)) File "/home/ubuntu/.local/lib/python3.5/site-packages/tensorflow/python/framework/tensor_util.py", line 464, in make_tensor_proto "supported type." % (type(values), values)) TypeError: Failed to convert object of type <class 'list'> to Tensor. Contents: [None, -1, 512]. Consider casting elements to a supported type.

Let me know if anyone has solved this.

[solivr]

Hi, Batch inference is not supported yet with exported model. It's on my todo list, once I get back to this code. ;)

[vikram-ai]

Hi solivr, Are you planning to implement batch mode?. Currently, PredictionModelBatch is not working with filenames too.