Error while tf.image.crop_and_resize

[SumithRZT]

I am trying to crop and resize an image with a list of co-ordinates using tf.image.crop_and_resize() but am getting the following error:

TypeError: Expected binary or unicode string, got 960,

Below is the code which I am using

ori_image = Image.open('/home/sumith/imagepyramids/1.jpg')
img_data = np.asarray(ori_image)
with tf.Session() as sess:
  sess.run(init)
  im_string = sess.run(tf.image.encode_jpeg(img_data, format="rgb"))
  img_data_tensor = [im_string, img_data.shape[0], img_data.shape[1], 3]
  cropped_list = sess.run(tf.image.crop_and_resize(image=img_data_tensor, boxes=extracted_data,
                                                   crop_size=[40, 36], box_ind=[20]))

and I am getting the above said error. The complete stack trace is given below.

Traceback (most recent call last):
  File "/home/sumith/PycharmProjects/fddb/tryouts_3.py", line 87, in <module>
    crop_size=[40, 36], box_ind=[20]))
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/ops/gen_image_ops.py", line 151, in crop_and_resize
    name=name)
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/op_def_library.py", line 493, in apply_op
    raise err
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/op_def_library.py", line 490, in apply_op
    preferred_dtype=default_dtype)
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/ops.py", line 669, in convert_to_tensor
    ret = conversion_func(value, dtype=dtype, name=name, as_ref=as_ref)
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/constant_op.py", line 176, in _constant_tensor_conversion_function
    return constant(v, dtype=dtype, name=name)
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/constant_op.py", line 165, in constant
    tensor_util.make_tensor_proto(value, dtype=dtype, shape=shape, verify_shape=verify_shape))
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/tensor_util.py", line 441, in make_tensor_proto
    tensor_proto.string_val.extend([compat.as_bytes(x) for x in proto_values])
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/tensor_util.py", line 441, in <listcomp>
    tensor_proto.string_val.extend([compat.as_bytes(x) for x in proto_values])
  File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/util/compat.py", line 65, in as_bytes
    (bytes_or_text,))
TypeError: Expected binary or unicode string, got 960

I am passing a string returned by tf.image.encode_jpeg still it is not working. Can I get any help ?

[SumithRZT]

I tried passing the UnicodeString ori_image.tobytes().decode('utf-8') in place of im_string, Still getting the same error

[MicaelCarvalho]

EDIT : removed bad solution, I misinterpreted the problem.

[SumithRZT]

I am getting error

ValueError: could not convert string to float: b'\xff\xd8\xff\xe0\x00\x10JFIF\x00\x01\x01\x01\x01,\x01,\x00\x00\xff\xdb\x00C\x00\x02\x01\x01\x01\x01\x01\x02\x01\x01\x01\x02

since im_string is datatype of str and numpy do not allow non numerical data types.

[MicaelCarvalho]

I didn't pay much attention to your construction of the image parameter, it seems you misinterpreted the docs: the image parameter is of shape [batch, image_height, image_width, depth] — that doesn't mean it should have the values of image_height, image_width and depth inside of it, the doc is only explaining its shape — meaning: it has the shape of an image, with the first index referring to the batch position (4D instead of 3D). Since you seem to be processing a single image, we can simply add a dimension to the beginning of the tensor. Here is a working example:

ori_image = Image.open('/home/sumith/imagepyramids/1.jpg')
img_data = np.expand_dims(np.asarray(ori_image).astype(np.float32), axis=0)
sess.run([tf.global_variables_initializer(), tf.local_variables_initializer()])
cropped_list = sess.run(tf.image.crop_and_resize(image=img_data, boxes=[[.1,.1,.6,.6]], crop_size=[40, 36], box_ind=[0]))

[SumithRZT]

the code snippet you gave works like charm!! . But is giving error when more than one box is passed in the boxes list cropped_list = sess.run( tf.image.crop_and_resize(image=img_data, boxes=[[1, 1, 6, 6],[1, 1, 6, 6]], crop_size=[40, 36], box_ind=[0])) Error is ValueError: Dimensions must be equal, but are 2 and 1 for 'CropAndResize' (op: 'CropAndResize') with input shapes: [1,480,640,3], [2,4], [1], [2].

Can you suggest where the error is, is it because of the batch size is 1 in img_data array ? Thanks for your help.

[MicaelCarvalho]

You didn't update the parameter box_ind:

box_ind: A Tensor of type int32. A 1-D tensor of shape [num_boxes] with int32 values in [0, batch). The value of box_ind[i] specifies the image that the i-th box refers to.

In your case, you have 2 boxes, so it should be of size 2 as well. For a single image, this will be a vector of zeroes with the same size as dimension 1 of boxes (i.e. [0,0]).

[prb12]

Note for future reference: This question is better asked on StackOverflow since it is not a bug or feature request. There is also a larger community that reads questions there. Thanks!

[prb12]

Closing since there doesn't seem to be a bug here.

[SumithRZT]

Thanks a lot for your help !!