[FEATURE REQUEST] convert tf.py_func to a TensorFlow graph

[hcnhatnam]

In the anchor_target_layer of model_train.py, using tf.py_func operator which enables you to run arbitrary Python code in the middle of a TensorFlow graph. However, tf.py_func is not supported in TensorFlow Serving that to deploy tensorflow models to production?

I have been looking for a solution to this problem and solution almost convert tf.py_func to a TensorFlow graph. Can someone please help me with this?

[eragonruan]

@hcnhatnam hi, anchor_target_layer is only used during training. For inference, no tf.py_func is used, so you don't have to handle tf.py_func for serving.

[hcnhatnam]

@eragonruan I know but in serving of tensorflow, anchor_target_layer is considered as Ops(detail in https://github.com/tensorflow/serving/issues/54). Therefore, I met error: E tensorflow_serving/core/aspired_versions_manager.cc:358] Servable {name: simplesave version: 1} cannot be loaded: Not found: Op type not registered 'PyFunc' in binary running on 040951f81f58. Make sure the Op and Kernel are registered in the binary running in this process. Note that if you are loading a saved graph which used ops from tf.contrib, accessing (e.g.) tf.contrib.resampler should be done before importing the graph, as contrib ops are lazily registered when the module is first accessed.

• simplesave folder contains model which was created as follows(in main/train.py): tf.saved_model.simple_save(sess, "simplesave/", inputs={"input_image": input_image,"input_im_info":input_im_info}, outputs={"bbox_pred": bbox_pred,"cls_pred":cls_pred})

[wqt2019]

hi,the output of the model is cls_prob and box_pred, this are not the final results of the text detection, the postprocess is needed . if you want get the final detection boxes in TensorFlow Serving , you need to implement the proposal_layer(demo_pb.py) function by tf-api .

[hcnhatnam]

hi,the output of the model is cls_prob and box_pred, this are not the final results of the text detection, the postprocess is needed . if you want get the final detection boxes in TensorFlow Serving , you need to implement the proposal_layer(demo_pb.py) function by tf-api .

thank you, but this problem is TF Serving (which written in C++) which doesn't support PyFunc yet (which written in Python) and author used to tf.py_func(anchor_target_layer function in model_train) in the train process. The problem has been mentioned https://github.com/eragonruan/text-detection-ctpn/issues/134 but there is almost no solution.

[wqt2019]

＠hcnhatnam, i have successed deploy the CTPN in TensorFlow Serving, but i cann't share the all code ,sorry. my solution is implement the proposal_layer and textdetector.detect (demo_pb.py) function by tf-api (not python-numpy) . the implementation of original code is the numpy which cann't merge to the TensorFlow Serving model . the outputs of the CTPN model are output_tensor_cls_prob,output_tensor_box_pred. when i get these two temp-results, i conbine the all 16pixes for final detection boxes(postprocess function) . the proposal_layer_tf_api function is the implementation by tf-api which reference to the proposal_layer and textdetector.detect function.

def postprocess(cls_prob_tf, box_pred_tf , im_info_tf): boxes=proposal_layer_tf_api(cls_prob_tf, box_pred_tf, im_info_tf) return boxes

with tf.Session() as sess: with tf.gfile.GFile(args.model, "rb") as f: restored_graph_def = tf.GraphDef() restored_graph_def.ParseFromString(f.read()) tf.import_graph_def( restored_graph_def, input_map=None, return_elements=None, name="" )

#打印节点信息
tensor_name_list = [tensor.name for tensor in tf.get_default_graph().as_graph_def().node]
for tensor_name in tensor_name_list:
    print(tensor_name, '\n')

export_path_base = args.export_model_dir
export_path = os.path.join(tf.compat.as_bytes(export_path_base),
    tf.compat.as_bytes(str(args.model_version)))
print('Exporting trained model to', export_path)
builder = tf.saved_model.builder.SavedModelBuilder(export_path)

raw_image =  tf.placeholder(tf.float32, shape=[None, None, None, 3])  #输入原始图像
jpeg,im_info = preprocess_image(raw_image)  #预处理,缩放

output_tensor_cls_prob,output_tensor_box_pred = tf.import_graph_def\
                            (tf.get_default_graph().as_graph_def(),
                           input_map={'Placeholder:0': jpeg},
                           return_elements=['Reshape_2:0','rpn_bbox_pred/Reshape_1:0'])

tensor_info_input = tf.saved_model.utils.build_tensor_info(raw_image)
tensor_info_output_cls_prob = tf.saved_model.utils.build_tensor_info(output_tensor_cls_prob)
tensor_info_output_box_pred = tf.saved_model.utils.build_tensor_info(output_tensor_box_pred)

prediction_signature = (
    tf.saved_model.signature_def_utils.build_signature_def(
        inputs={'images': tensor_info_input},
        outputs={'cls_prob': tensor_info_output_cls_prob,
                 'box_pred': tensor_info_output_box_pred,
                 },
        method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME
    ))

im_info_output = tf.saved_model.utils.build_tensor_info(im_info)

print('\n')
print('im_info_input tensor shape', im_info.shape)

# ctpn后处理,合并宽度为16的boxes,得到最终的文本框
result_boxes=postprocess(output_tensor_cls_prob,output_tensor_box_pred,im_info)

#-------根据检测的文本框裁剪图像，等比例缩放到高32，padding到同一大小----------
crop_resize_img,crop_resize_im_info = crop_resize_image(jpeg, result_boxes)
output_crop_resize_img = tf.saved_model.utils.build_tensor_info(crop_resize_img)
output_crop_resize_img_info = tf.saved_model.utils.build_tensor_info(crop_resize_im_info)
#----------

tensor_info_output_boxes = tf.saved_model.utils.build_tensor_info(result_boxes)

prediction_post_signature = (
    tf.saved_model.signature_def_utils.build_signature_def(
        inputs={'images': tensor_info_input},
        outputs={'detection_boxes': tensor_info_output_boxes,
                 'resize_im_info':im_info_output,
                 'crop_resize_img': output_crop_resize_img,
                 'crop_resize_im_info': output_crop_resize_img_info,},
        method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME
    ))

builder.add_meta_graph_and_variables(
    sess, [tf.saved_model.tag_constants.SERVING],
    signature_def_map={
        'predict_images':prediction_signature,
        'predict_images_post': prediction_post_signature,
    })

builder.save(as_text=False)

print('Done exporting!')

[hcnhatnam]

@evanfly Can you realse proposal_layer_tf_api?

[qiaoning]

@evanfly beg a contact method，email，phone，WeChat，blog，whatever，just can contact with you. I have some ctpn tensorflow serving problems consult with you. Thanks a lot.

[LuWei6896]

@evanfly can you send some tips for implement the proposal_lay&&detect_lay by tf_api？my wechat:IT666666TI, ths a lot!