yyccR / yolov5_in_tf2_keras

YOLOv5 in TF2 > TFLite > ONNX > TensorRT
368 stars 85 forks source link

能使用tfserving部署吗? #1

Open sveits6 opened 2 years ago

sveits6 commented 2 years ago

如题

yyccR commented 2 years ago

可以的,train.py文件模型导出改成pb格式就行

whalefa1I commented 2 years ago

https://github.com/yyccR/yolov5_in_tf2_keras/blob/3e6645cbf94d2a1e11c33663e80113daa4590321/train.py#L72-L74 我后面加了个yolov5.save(log_dir + '/yolov5-tf', save_format='tf'),想保存成savedmodel部署到tf seving上,但是报错哎

  File "H:/face_landmark/bd-face-detector/face_details_detector/yolov5_in_tf2_keras-master/train.py", line 107, in main
    yolov5.save(log_dir + '/yolov5-tf', save_format='tf')
  File "D:\ProgramData\Anaconda3\lib\site-packages\keras\utils\traceback_utils.py", line 67, in error_handler
    raise e.with_traceback(filtered_tb) from None
  File "D:\ProgramData\Anaconda3\lib\json\encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "D:\ProgramData\Anaconda3\lib\json\encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
TypeError: Unable to serialize True to JSON. Unrecognized type <class 'tensorflow.python.framework.ops.EagerTensor'>.

Process finished with exit code 1
whalefa1I commented 2 years ago

在这里保存是不是batch size就被固定住了

yyccR commented 2 years ago

@whalefa1I 你用的tf是哪个版本?

whalefa1I commented 2 years ago

2.7.0,换成2.3.0好像好一点

yyccR commented 2 years ago

已经添加了相关pb模型保存代码,测试ok,可以拉一下再试试 @whalefa1I

yyccR commented 2 years ago

顺带修复了layers.py里面保存pb时的一些小问题.

whalefa1I commented 2 years ago

好的我试一下~!

whalefa1I commented 2 years ago

https://github.com/yyccR/yolov5_in_tf2_keras/blob/269385d53dcffc416064852462327db026207678/train.py#L100-L111 能在这个位置中间保存一下,然后后面做inference吗

            '''
            生成saved model
            '''
            inputs = tf.keras.Input(shape=(640, 640, 3))
            yolo_preds = yolov5(inputs, training=True)
            yolo_head_output = yolo.yolo_head(yolo_preds, is_training=False)
            nms_output = yolo.nms(yolo_head_output, iou_thres=0.3)
            model = tf.keras.Model(inputs, nms_output )
            model.save(r'./whole_model_20220309',  save_format='tf')

我保存下来的savedmodel在inference的时候效果和tensorboard里面看到的不一样,效果比较差

whalefa1I commented 2 years ago

嗯我知道,如果这么inference的话必须传batch_size张图片,所以想换个形式保存看看 报错如下,我的batch_size为16,172800 * 16 = 2764800 Traceback (most recent call last): File "H:/face_landmark/bd-face-detector/face_details_detector/yolov5_in_tf2_keras-master/inference.py", line 188, in <module> yolo_preds = yolo.yolov5.predict(input_data) File "D:\ProgramData\Anaconda3\lib\site-packages\keras\utils\traceback_utils.py", line 67, in error_handler raise e.with_traceback(filtered_tb) from None File "D:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\eager\execute.py", line 59, in quick_execute inputs, attrs, num_outputs) tensorflow.python.framework.errors_impl.InvalidArgumentError: Input to reshape is a tensor with 172800 values, but the requested shape has 2764800 [[node model_1/model/tf.reshape/Reshape (defined at D:\ProgramData\Anaconda3\lib\site-packages\keras\layers\core\tf_op_layer.py:261) ]] [Op:__inference_predict_function_9035]

yyccR commented 2 years ago

你可以试试把batch-size设置为1,如果你多张图片就写个循环?

whalefa1I commented 2 years ago

哈哈哈哈好的,input_data = np.tile(np.expand_dims(input_data, axis=0), (16, 1, 1, 1)),我先这么弄看看,还是有点报错,辛苦辛苦

whalefa1I commented 2 years ago
yolo = Yolo(
        image_shape=image_shape,
        batch_size=batch_size,
        num_class=num_class,
        is_training=False,
        anchors=anchors,
        anchor_masks=anchor_masks,
        strides=[8, 16, 32],
        net_type='5s',
        model_path=r'C:\Users\Administrator\Desktop\yolov5-20220309-5s-tf-20.h5'
    )
# image shape(batch, h, w, 3)
yolo.yolov5.summary()

我summary了一下为啥出来的结果都堆在一起了,类似这样

 tf.reshape_3 (TFOpLambda)      (16, 19200, 9)       0           ['concatenate[0][0]']            

 tf.reshape_4 (TFOpLambda)      (16, 4800, 9)        0           ['concatenate_1[0][0]']          

 tf.reshape_5 (TFOpLambda)      (16, 1200, 9)        0           ['concatenate_2[0][0]']          

 tf.concat (TFOpLambda)         (16, 25200, 9)       0           ['tf.reshape_3[0][0]',           
                                                                  'tf.reshape_4[0][0]',           
                                                                  'tf.reshape_5[0][0]']           

==================================================================================================
Total params: 7,132,369
Trainable params: 7,112,849
Non-trainable params: 19,520
__________________________________________________________________________________________________

Process finished with exit code 0
yyccR commented 2 years ago

是的,推理模式的时候,在模型里面已经把3层预测tensor的每一层都规整到图片的大小尺度下,所以只需要做进一步的nms就行。

whalefa1I commented 2 years ago

哦哦哦,那就是刚才inference的时候yolo_head_output = yolo.yolo_head(yolo_preds, is_training=False)这个不需要,只需要

yolo = Yolo(
        image_shape=image_shape,
        batch_size=batch_size,
        num_class=num_class,
        is_training=False,
        anchors=anchors,
        anchor_masks=anchor_masks,
        strides=[8, 16, 32],
        net_type='5s',
        model_path=r'C:\Users\Administrator\Desktop\yolov5-20220309-5s-tf-20.h5'
    )
# image shape(batch, h, w, 3)
yolo.yolov5.summary()
yolo_preds = yolo.yolov5.predict(input_data)
nms_output = yolo.nms(yolo_preds, iou_thres=0.3)
nms_output = nms_output[0]
for box_obj_cls in nms_output:
    if box_obj_cls[4] > 0.5:
        label = int(box_obj_cls[5]) - 1
        if label < 5:
            class_name = label_list[label]
            xmin, ymin, xmax, ymax = box_obj_cls[:4]
            image = draw_bounding_box(image, class_name, box_obj_cls[4], int(xmin), int(ymin),
                                         int(xmax), int(ymax))

cv2.imshow('test', image)
cv2.waitKey(0)
yyccR commented 2 years ago

事实上,yolo.predict 就行,已经封装好了一个predict的方法.

whalefa1I commented 2 years ago

对的,我也很奇怪,明明里面封装好了,为啥我调不到

yyccR commented 2 years ago
yolo = Yolo(
        image_shape=image_shape,
        batch_size=batch_size,
        num_class=num_class,
        is_training=False,
        anchors=anchors,
        anchor_masks=anchor_masks,
        strides=[8, 16, 32],
        net_type='5s',
        model_path=r'C:\Users\Administrator\Desktop\yolov5-20220309-5s-tf-20.h5'
    )
# images shape(batch, h, w, 3)
yolo.predict(images)
whalefa1I commented 2 years ago

https://github.com/yyccR/yolov5_in_tf2_keras/blob/269385d53dcffc416064852462327db026207678/yolo.py#L91 如果把这个-1变成写出来,,前面的batch_size变成-1, 是不是前面的batch_size就不用固定了,那部署起来模型调用的时候就可以动态调用了

yyccR commented 2 years ago

可以尝试下