能使用tfserving部署吗？

[sveits6]

如题

[yyccR]

可以的，train.py文件模型导出改成pb格式就行

[whalefa1I]

https://github.com/yyccR/yolov5_in_tf2_keras/blob/3e6645cbf94d2a1e11c33663e80113daa4590321/train.py#L72-L74 我后面加了个yolov5.save(log_dir + '/yolov5-tf', save_format='tf')，想保存成savedmodel部署到tf seving上，但是报错哎

  File "H:/face_landmark/bd-face-detector/face_details_detector/yolov5_in_tf2_keras-master/train.py", line 107, in main
    yolov5.save(log_dir + '/yolov5-tf', save_format='tf')
  File "D:\ProgramData\Anaconda3\lib\site-packages\keras\utils\traceback_utils.py", line 67, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "D:\ProgramData\Anaconda3\lib\json\encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "D:\ProgramData\Anaconda3\lib\json\encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
TypeError: Unable to serialize True to JSON. Unrecognized type <class 'tensorflow.python.framework.ops.EagerTensor'>.

Process finished with exit code 1

[whalefa1I]

在这里保存是不是batch size就被固定住了

[yyccR]

@whalefa1I 你用的tf是哪个版本？

[whalefa1I]

2.7.0,换成2.3.0好像好一点

[yyccR]

已经添加了相关pb模型保存代码，测试ok，可以拉一下再试试 @whalefa1I

[yyccR]

顺带修复了layers.py里面保存pb时的一些小问题.

[whalefa1I]

好的我试一下~！

[whalefa1I]

https://github.com/yyccR/yolov5_in_tf2_keras/blob/269385d53dcffc416064852462327db026207678/train.py#L100-L111 能在这个位置中间保存一下，然后后面做inference吗

            '''
            生成saved model
            '''
            inputs = tf.keras.Input(shape=(640, 640, 3))
            yolo_preds = yolov5(inputs, training=True)
            yolo_head_output = yolo.yolo_head(yolo_preds, is_training=False)
            nms_output = yolo.nms(yolo_head_output, iou_thres=0.3)
            model = tf.keras.Model(inputs, nms_output )
            model.save(r'./whole_model_20220309',  save_format='tf')

我保存下来的savedmodel在inference的时候效果和tensorboard里面看到的不一样，效果比较差

[whalefa1I]

嗯我知道，如果这么inference的话必须传batch_size张图片，所以想换个形式保存看看 报错如下，我的batch_size为16，172800 * 16 = 2764800 Traceback (most recent call last): File "H:/face_landmark/bd-face-detector/face_details_detector/yolov5_in_tf2_keras-master/inference.py", line 188, in <module> yolo_preds = yolo.yolov5.predict(input_data) File "D:\ProgramData\Anaconda3\lib\site-packages\keras\utils\traceback_utils.py", line 67, in error_handler raise e.with_traceback(filtered_tb) from None File "D:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\eager\execute.py", line 59, in quick_execute inputs, attrs, num_outputs) tensorflow.python.framework.errors_impl.InvalidArgumentError: Input to reshape is a tensor with 172800 values, but the requested shape has 2764800 [[node model_1/model/tf.reshape/Reshape (defined at D:\ProgramData\Anaconda3\lib\site-packages\keras\layers\core\tf_op_layer.py:261) ]] [Op:__inference_predict_function_9035]

[yyccR]

你可以试试把batch-size设置为1，如果你多张图片就写个循环？

[whalefa1I]

哈哈哈哈好的，input_data = np.tile(np.expand_dims(input_data, axis=0), (16, 1, 1, 1))，我先这么弄看看，还是有点报错，辛苦辛苦

[whalefa1I]

yolo = Yolo(
        image_shape=image_shape,
        batch_size=batch_size,
        num_class=num_class,
        is_training=False,
        anchors=anchors,
        anchor_masks=anchor_masks,
        strides=[8, 16, 32],
        net_type='5s',
        model_path=r'C:\Users\Administrator\Desktop\yolov5-20220309-5s-tf-20.h5'
    )
# image shape(batch, h, w, 3)
yolo.yolov5.summary()

我summary了一下为啥出来的结果都堆在一起了,类似这样

 tf.reshape_3 (TFOpLambda)      (16, 19200, 9)       0           ['concatenate[0][0]']            

 tf.reshape_4 (TFOpLambda)      (16, 4800, 9)        0           ['concatenate_1[0][0]']          

 tf.reshape_5 (TFOpLambda)      (16, 1200, 9)        0           ['concatenate_2[0][0]']          

 tf.concat (TFOpLambda)         (16, 25200, 9)       0           ['tf.reshape_3[0][0]',           
                                                                  'tf.reshape_4[0][0]',           
                                                                  'tf.reshape_5[0][0]']           

==================================================================================================
Total params: 7,132,369
Trainable params: 7,112,849
Non-trainable params: 19,520
__________________________________________________________________________________________________

Process finished with exit code 0

[yyccR]

是的，推理模式的时候，在模型里面已经把3层预测tensor的每一层都规整到图片的大小尺度下，所以只需要做进一步的nms就行。

[whalefa1I]

哦哦哦，那就是刚才inference的时候yolo_head_output = yolo.yolo_head(yolo_preds, is_training=False)这个不需要，只需要

yolo = Yolo(
        image_shape=image_shape,
        batch_size=batch_size,
        num_class=num_class,
        is_training=False,
        anchors=anchors,
        anchor_masks=anchor_masks,
        strides=[8, 16, 32],
        net_type='5s',
        model_path=r'C:\Users\Administrator\Desktop\yolov5-20220309-5s-tf-20.h5'
    )
# image shape(batch, h, w, 3)
yolo.yolov5.summary()
yolo_preds = yolo.yolov5.predict(input_data)
nms_output = yolo.nms(yolo_preds, iou_thres=0.3)
nms_output = nms_output[0]
for box_obj_cls in nms_output:
    if box_obj_cls[4] > 0.5:
        label = int(box_obj_cls[5]) - 1
        if label < 5:
            class_name = label_list[label]
            xmin, ymin, xmax, ymax = box_obj_cls[:4]
            image = draw_bounding_box(image, class_name, box_obj_cls[4], int(xmin), int(ymin),
                                         int(xmax), int(ymax))

cv2.imshow('test', image)
cv2.waitKey(0)

[yyccR]

事实上，yolo.predict 就行，已经封装好了一个predict的方法.

[whalefa1I]

对的，我也很奇怪，明明里面封装好了，为啥我调不到

[yyccR]

yolo = Yolo(
        image_shape=image_shape,
        batch_size=batch_size,
        num_class=num_class,
        is_training=False,
        anchors=anchors,
        anchor_masks=anchor_masks,
        strides=[8, 16, 32],
        net_type='5s',
        model_path=r'C:\Users\Administrator\Desktop\yolov5-20220309-5s-tf-20.h5'
    )
# images shape(batch, h, w, 3)
yolo.predict(images)

[whalefa1I]

https://github.com/yyccR/yolov5_in_tf2_keras/blob/269385d53dcffc416064852462327db026207678/yolo.py#L91 如果把这个-1变成写出来，，前面的batch_size变成-1， 是不是前面的batch_size就不用固定了，那部署起来模型调用的时候就可以动态调用了

[yyccR]

可以尝试下