Script not working with shape polygon

[PhanDuc]

Hi rky,

I tried your script with shape polygon but it didn't work.

Is there any way to make it works with that shape?

[rky0930]

Hi, @PhanDuc The script is for rectangle annotation. In via, rect use [x, y, width, height] and polygon use [all_points_x, all_points_y]. I suggest you change the script just the part related above differences. If you have a further question or issue, feel free to contact with me. Thanks!

[PhanDuc]

Hi @rky0930 , I made some changes to the script "via_to_tfrecord.py":

for attribute  in data["regions"]:
    #print(attribute)
  #for region_id, attribute in data['regions']:
    try:

      shape = attribute['shape_attributes']
      x = shape["all_points_x"]
      y = shape["all_points_y"]
      region_x = x[np.argmin(x)]
      region_y = y[np.argmin(x)]

      shape_width = abs(x[np.argmin(y)] - region_x)
      shape_height = abs(y[np.argmax(y)] - region_y)
      region_xmax = region_x + shape_width  
      region_ymax = region_y + shape_height 

After generated tfrecords, I used "mask_rcnn_inception_v2" to training, but I got this error:

InvalidArgumentError (see above for traceback): assertion failed: [] [Condition x == y did not hold element-wise:] [x (Loss/BoxClassifierLoss/assert_equal/x:0) = ] [0] [y (Loss/BoxClassifierLoss/assert_equal/y:0) = ] [35] [[node Loss/BoxClassifierLoss/assert_equal/Assert/Assert (defined at /var/www/food-ir/fruits_veges/models/research/object_detection/utils/shape_utils.py:315) = Assert[T=[DT_STRING, DT_STRING, DT_STRING, DT_INT32, DT_STRING, DT_INT32], summarize=3, _device="/job:localhost/replica:0/task:0/device:CPU:0"](Loss/BoxClassifierLoss/assert_equal/All/_177, Loss/RPNLoss/assert_equal/Assert/Assert/data_0, Loss/RPNLoss/assert_equal/Assert/Assert/data_1, Loss/BoxClassifierLoss/assert_equal/Assert/Assert/data_2, Loss/BoxClassifierLoss/assert_equal/x/_179, Loss/BoxClassifierLoss/assert_equal/Assert/Assert/data_4, Loss/RPNLoss/assert_equal/y/_181)]]

Did I generated wrong tfrecord files ?

[rky0930]

@PhanDuc You have to check what is the mask_rcnn_inception_v2 code's data format. I'm not sure VIA polygon data format is the same with mask rcnn segmentation dataset format.
if is the same, you have to sync the key between tfrecord and program the bold text is key value when tensorflow read tfrecord, ''''''''''''' example = tf.train.Example(features=tf.train.Features(feature={ 'image/height': dataset_util.int64_feature(image_height), 'image/width': dataset_util.int64_feature(image_width), 'image/filename': dataset_util.bytes_feature(filename), 'image/source_id': dataset_util.bytes_feature(filename), 'image/key/sha256': dataset_util.bytes_feature(key.encode('utf8')), 'image/encoded': dataset_util.bytes_feature(encoded_jpg), 'image/format': dataset_util.bytes_feature('jpeg'.encode('utf8')), 'image/object/bbox/xmin': dataset_util.float_list_feature(xmin), 'image/object/bbox/xmax': dataset_util.float_list_feature(xmax), 'image/object/bbox/ymin': dataset_util.float_list_feature(ymin), 'image/object/bbox/ymax': dataset_util.float_list_feature(ymax),

'image/object/class/text': dataset_util.bytes_list_feature(classes_text),

  'image/object/class/label': dataset_util.int64_list_feature(classes),

'''''''''''''''' If you want to check your tfrecord is made correctly, You can make tfrecord_to_via script and check the dataset using VIA again. This is why i made tfrecord_to_via script !

I hope this is helpful answer to you !

[PhanDuc]

Hi @rky0930 , Thank you for comments, I generated tfrecord files but still got the error above, I think the mask_rcnn_inception_v2 code's data format is the problem. I'm still trying to figure it out.

[PhanDuc]

@rky0930 , Could you please review the code above where I tried to convert from polygon to rect. Did I do it in the right way?

[rky0930]

Sorry for late reply. You may already solve this problem. I don't know what polygon are you using.. Is that you are plan to do that ? Check the comment below.

import numpy as np
shape = {"all_points_x":[189,159,268,296,289,189],"all_points_y":[127,166,229,167,123,127]}
x = shape["all_points_x"]
y = shape["all_points_y"]
region_x = x[np.argmin(x)] # most left point  
region_y = y[np.argmin(x)] # most left point  

shape_width = abs(x[np.argmin(y)] - region_x)  # distance between most top point and most left point
shape_height = abs(y[np.argmax(y)] - region_y) # distance between most top point and most left point
region_xmax = region_x + shape_width   # max point  
region_ymax = region_y + shape_height  # max point 

print(shape_width, shape_height)
print(region_xmax, region_ymax)

I think if you want to make a rect which cover all polygon, you should find min and max point and combine the points together. Let me know what is your plan. Let's think together !

[PhanDuc]

Hi rky, You're right! I was finding the upper left point then calculated the distance. Tried to form them in the shape of the rectangle. I still got an error after using the tfrecord from that way. train.py error: InvalidArgumentError (see above for traceback): assertion failed: [Groundtruth boxes and labels have incompatible shapes!] [Condition x == y did not hold element-wise:] [x (Loss/BoxClassifierLoss/strided_slice_1:0) = ] · Issue #2737 · tensorflow/models What do you think ?