search

PaddlePaddle / PaddleOCR

Awesome multilingual OCR toolkits based on PaddlePaddle (practical ultra lightweight OCR system, support 80+ languages recognition, provide data annotation and synthesis tools, support training and deployment among server, mobile, embedded and IoT devices)
https://paddlepaddle.github.io/PaddleOCR/
Apache License 2.0
44.47k stars 7.84k forks source link

使用paddle.jit.to_static 转换 ppstructure模型失败？未能正确转换 #13855

Closed wangzy0327 closed 2 months ago

wangzy0327 commented 2 months ago

🔎 Search before asking

🐛 Bug (问题描述)

参照文档执行ppstruture https://github.com/PaddlePaddle/PaddleOCR/blob/main/ppstructure/docs/inference.md#11-%E7%89%88%E9%9D%A2%E5%88%86%E6%9E%90%E8%A1%A8%E6%A0%BC%E8%AF%86%E5%88%AB 通过添加paddle.jit.to_static 代码，准备利用CINN后端进行进一步优化和性能提升。参照 predict_system.py文件进行修改执行，通过添加 to_cinn_net 函数来完成 cinn后端转换。具体修改如下：

def to_cinn_net(net, **kwargs):
    build_strategy = paddle.static.BuildStrategy()
    build_strategy.build_cinn_pass = True
    return paddle.jit.to_static(
        net,
        build_strategy=build_strategy,
        full_graph=True,
        **kwargs
    )

class StructureSystem(object):    
    def __init__(self, args):
        self.mode = args.mode
        self.recovery = args.recovery
        self.use_cinn = True
        logger.info("Init Structure System {} cinn".format(self.use_cinn))

        self.image_orientation_predictor = None
        if args.image_orientation:
            import paddleclas

            self.image_orientation_predictor = paddleclas.PaddleClas(
                model_name="text_image_orientation"
            )

        if self.mode == "structure":
            if not args.show_log:
                logger.setLevel(logging.INFO)
            if args.layout == False and args.ocr == True:
                args.ocr = False
                logger.warning(
                    "When args.layout is false, args.ocr is automatically set to false"
                )
            # init model
            self.layout_predictor = None
            self.text_system = None
            self.table_system = None
            if args.layout:
                logger.info("Init Structure layout ........")
                self.layout_predictor = LayoutPredictor(args)
                if self.use_cinn :
                    self.layout_predictor = to_cinn_net(self.layout_predictor)
                if args.ocr:
                    self.text_system = TextSystem(args)
                    if self.use_cinn :
                        self.text_system = to_cinn_net(self.text_system)
            if args.table:
                if self.text_system is not None:
                    logger.info("Init Structure table System using Text System ........")
                    self.table_system = TableSystem(
                        args,
                        self.text_system.text_detector,
                        self.text_system.text_recognizer,
                    )
                    if self.use_cinn :
                        self.table_system = to_cinn_net(self.table_system)
                else:
                    self.table_system = TableSystem(args)
                    if self.use_cinn :
                    self.table_system = to_cinn_net(self.table_system)

执行命令

~/build-python3-10-14/bin/python3 predict_system.py --layout_model_dir=inference/picodet_lcnet_x1_0_layout_infer \
>                           --image_dir=./docs/table/1.png \
>                           --output=../output \
>                           --table=false \
>                           --ocr=false --vis_font_path=../doc/fonts/simfang.ttf

执行时出现的log如下：

I0912 02:37:30.027549 1315167 init.cc:234] ENV [CUSTOM_DEVICE_ROOT]=/home/wzy/build-python3-10-14/lib/python3.10/site-packages/paddle_custom_device
I0912 02:37:30.027586 1315167 init.cc:143] Try loading custom device libs from: [/home/wzy/build-python3-10-14/lib/python3.10/site-packages/paddle_custom_device]
I0912 02:37:30.046777 1315167 custom_device.cc:1099] Succeed in loading custom runtime in lib: /home/wzy/build-python3-10-14/lib/python3.10/site-packages/paddle_custom_device/libpaddle-custom-mlu.so
I0912 02:37:30.049186 1315167 custom_kernel.cc:63] Succeed in loading 262 custom kernel(s) from loaded lib(s), will be used like native ones.
I0912 02:37:30.049300 1315167 init.cc:155] Finished in LoadCustomDevice with libs_path: [/home/wzy/build-python3-10-14/lib/python3.10/site-packages/paddle_custom_device]
I0912 02:37:30.049327 1315167 init.cc:240] CustomDevice: mlu, visible devices count: 1
[2024/09/12 02:37:30] ppocr INFO: Init Structure System True cinn
[2024/09/12 02:37:30] ppocr INFO: Init Structure layout ........
E0912 02:37:30.943815 1315167 analysis_config.cc:127] Please use PaddlePaddle with GPU version.
Traceback (most recent call last):
  File "/home/wzy/paddle_tests/models/PaddleOCR/ppstructure/predict_system.py", line 419, in <module>
    main(args)
  File "/home/wzy/paddle_tests/models/PaddleOCR/ppstructure/predict_system.py", line 319, in main
    structure_sys = StructureSystem(args)
  File "/home/wzy/paddle_tests/models/PaddleOCR/ppstructure/predict_system.py", line 85, in __init__
    self.layout_predictor = to_cinn_net(self.layout_predictor)
  File "/home/wzy/paddle_tests/models/PaddleOCR/ppstructure/predict_system.py", line 47, in to_cinn_net
    return paddle.jit.to_static(
  File "/home/wzy/build-python3-10-14/lib/python3.10/site-packages/paddle/jit/api.py", line 250, in to_static
    return decorated(function)
  File "/home/wzy/build-python3-10-14/lib/python3.10/site-packages/paddle/jit/api.py", line 219, in decorated
    static_layer = copy_decorator_attrs(
  File "/home/wzy/build-python3-10-14/lib/python3.10/site-packages/paddle/jit/api.py", line 90, in copy_decorator_attrs
    decorated_obj.__name__ = original_func.__name__
AttributeError: 'LayoutPredictor' object has no attribute '__name__'. Did you mean: '__ne__'?

如何解决上述问题，完成 ppstructure 到CINN后端 的转换？

🏃‍♂️ Environment (运行环境)

OS：Ubuntu20.04 Python 3.10 PaddleOCR 2.8.0 paddlepaddle 2.6

🌰 Minimal Reproducible Example (最小可复现问题的Demo)

predict_system.py 参照上面的predict_system.py 增加 to_cinn_net 函数

jingsongliujing commented 2 months ago

to_cinn_net 函数期望接收一个函数作为输入，但是在 StructureSystem 类的初始化过程中，你将 self.layout_predictor 作为一个类实例传递给了 to_cinn_net，可以尝试

def to_cinn_net(net, **kwargs):
    build_strategy = paddle.static.BuildStrategy()
    build_strategy.build_cinn_pass = True
    return paddle.jit.to_static(
        net.__call__,
        build_strategy=build_strategy,
        full_graph=True,
        **kwargs
    )
wangzy0327 commented 2 months ago

to_cinn_net 函数期望接收一个函数作为输入，但是在 StructureSystem 类的初始化过程中，你将 self.layout_predictor 作为一个类实例传递给了 to_cinn_net，可以尝试

def to_cinn_net(net, **kwargs):
    build_strategy = paddle.static.BuildStrategy()
    build_strategy.build_cinn_pass = True
    return paddle.jit.to_static(
        net.__call__,
        build_strategy=build_strategy,
        full_graph=True,
        **kwargs
    )

按您说的修改后，修改之后执行打印log如下：

I0912 03:31:32.091351 1318574 init.cc:234] ENV [CUSTOM_DEVICE_ROOT]=/home/wzy/build-python3-10-14/lib/python3.10/site-packages/paddle_custom_device
I0912 03:31:32.091378 1318574 init.cc:143] Try loading custom device libs from: [/home/wzy/build-python3-10-14/lib/python3.10/site-packages/paddle_custom_device]
I0912 03:31:32.103998 1318574 custom_device.cc:1099] Succeed in loading custom runtime in lib: /home/wzy/build-python3-10-14/lib/python3.10/site-packages/paddle_custom_device/libpaddle-custom-mlu.so
I0912 03:31:32.105597 1318574 custom_kernel.cc:63] Succeed in loading 262 custom kernel(s) from loaded lib(s), will be used like native ones.
I0912 03:31:32.105671 1318574 init.cc:155] Finished in LoadCustomDevice with libs_path: [/home/wzy/build-python3-10-14/lib/python3.10/site-packages/paddle_custom_device]
I0912 03:31:32.105688 1318574 init.cc:240] CustomDevice: mlu, visible devices count: 1
[2024/09/12 03:31:32] ppocr INFO: Init Structure System True cinn
[2024/09/12 03:31:32] ppocr INFO: Init Structure layout ........
E0912 03:31:32.809263 1318574 analysis_config.cc:127] Please use PaddlePaddle with GPU version.
Traceback (most recent call last):
  File "/home/wzy/paddle_tests/models/PaddleOCR/ppstructure/predict_system.py", line 419, in <module>
    main(args)
  File "/home/wzy/paddle_tests/models/PaddleOCR/ppstructure/predict_system.py", line 319, in main
    structure_sys = StructureSystem(args)
  File "/home/wzy/paddle_tests/models/PaddleOCR/ppstructure/predict_system.py", line 85, in __init__
    self.layout_predictor = to_cinn_net(self.layout_predictor)
  File "/home/wzy/paddle_tests/models/PaddleOCR/ppstructure/predict_system.py", line 47, in to_cinn_net
    return paddle.jit.to_static(
  File "/home/wzy/build-python3-10-14/lib/python3.10/site-packages/paddle/jit/api.py", line 250, in to_static
    return decorated(function)
  File "/home/wzy/build-python3-10-14/lib/python3.10/site-packages/paddle/jit/api.py", line 221, in decorated
    decorated_obj=StaticClass(
  File "/home/wzy/build-python3-10-14/lib/python3.10/site-packages/paddle/jit/dy2static/program_translator.py", line 774, in __init__
    super().__init__(function, input_spec, **kwargs)
  File "/home/wzy/build-python3-10-14/lib/python3.10/site-packages/paddle/jit/dy2static/program_translator.py", line 337, in __init__
    raise TypeError(
TypeError: When using 'to_static' to convert method of a class, please ensure the class inherits from nn.Layer
GreatV commented 2 months ago

这里的predictor是对paddle inference的封装，而且类继承是的python的object，不是paddle.nn.Layer，所以不能用to_static。要对paddle模型用to_static

wangzy0327 commented 2 months ago

这里的predictor是对paddle inference的封装，而且类继承是的python的object，不是paddle.nn.Layer，所以不能用to_static。要对paddle模型用to_static

如何对Paddle模型用to_static呢？

GreatV commented 2 months ago

https://github.com/PaddlePaddle/PaddleOCR/blob/1c8233d5bcaa1b545e92fab6d937ee3a5ee0501e/tools/export_model.py