PaddlePaddle / PaddleOCR

Awesome multilingual OCR toolkits based on PaddlePaddle (practical ultra lightweight OCR system, support 80+ languages recognition, provide data annotation and synthesis tools, support training and deployment among server, mobile, embedded and IoT devices)
https://paddlepaddle.github.io/PaddleOCR/
Apache License 2.0
44.25k stars 7.82k forks source link

Bugs when trying to export `det_r50_dcn_fce_ctw` model #7027

Closed tann9949 closed 2 years ago

tann9949 commented 2 years ago

:mage: 请提供下述完整信息以便快速定位问题/Please provide the following information to quickly locate the problem

Hi, I was trying to export my trained det_r50_dcn_fce_ctw model and faced the issue when trying to export the model.

I use the code modified from tools/export_model.py.

:computer: Here's my code: The code bugged on the last line `export_single_model` ```python from ppocr.modeling.architectures import build_model from ppocr.postprocess import build_post_process from ppocr.utils.save_load import load_model from ppocr.utils.logging import get_logger from tools.program import load_config, merge_config, ArgsParser from tools.export_model import export_single_model class FLAGS: config = "configs/rec/rec_svtrnet-wandb.yml" profiler_options = None opt = {} config = load_config(FLAGS.config) config = merge_config(config, FLAGS.opt) logger = get_logger() post_process_class = build_post_process(config["PostProcess"], config["Global"]) # build character assert hasattr(post_process_class, "character") char_num = len(getattr(post_process_class, "character")) config["Architecture"]["Head"]["out_channels"] = char_num model = build_model(config["Architecture"]) load_model(config, model, model_type="rec") model.eval() save_path = config["Global"]["save_inference_dir"] arch_config = config["Architecture"] save_path = os.path.join(save_path, "inference") export_single_model(model, arch_config, save_path, logger) ```
:wrench: Here's my config file: ```yaml Global: use_gpu: true epoch_num: 1500 log_smooth_window: 20 print_batch_step: 20 save_model_dir: ./output/det_r50_dcn_fce_ctw/ save_epoch_step: 100 # evaluation is run every 835 iterations eval_batch_step: [0, 835] cal_metric_during_train: False pretrained_model: ./pretrain_models/ResNet50_vd_ssld_pretrained checkpoints: ./output/det_r50_dcn_fce_ctw/latest save_inference_dir: output/det_r50_dcn_fce_ctw/inference use_visualdl: False infer_img: doc/imgs_en/img_10.jpg save_res_path: ./output/det_fce/prediacts_fce.txt use_wandb: True wandb: project: det_r50_dcn_fce_ctw name: trial_run entity: my-entity Architecture: model_type: det algorithm: FCE Transform: Backbone: name: ResNet layers: 50 dcn_stage: [False, True, True, True] out_indices: [1,2,3] Neck: name: FCEFPN out_channels: 256 has_extra_convs: False extra_stage: 0 Head: name: FCEHead fourier_degree: 5 Loss: name: FCELoss fourier_degree: 5 num_sample: 50 Optimizer: name: Adam beta1: 0.9 beta2: 0.999 lr: learning_rate: 0.0001 regularizer: name: 'L2' factor: 0 PostProcess: name: FCEPostProcess scales: [8, 16, 32] alpha: 1.0 beta: 1.0 fourier_degree: 5 box_type: 'poly' Metric: name: DetFCEMetric main_indicator: hmean Train: dataset: name: SimpleDataSet data_dir: ./train_data/ctw1500/imgs/ label_file_list: - ./train_data/ctw1500/imgs/training.txt transforms: - DecodeImage: # load image img_mode: BGR channel_first: False ignore_orientation: True - DetLabelEncode: # Class handling label - ColorJitter: brightness: 0.142 saturation: 0.5 contrast: 0.5 - RandomScaling: - RandomCropFlip: crop_ratio: 0.5 - RandomCropPolyInstances: crop_ratio: 0.8 min_side_ratio: 0.3 - RandomRotatePolyInstances: rotate_ratio: 0.5 max_angle: 30 pad_with_fixed_color: False - SquareResizePad: target_size: 800 pad_ratio: 0.6 - IaaAugment: augmenter_args: - { 'type': Fliplr, 'args': { 'p': 0.5 } } - FCENetTargets: fourier_degree: 5 - NormalizeImage: scale: 1./255. mean: [0.485, 0.456, 0.406] std: [0.229, 0.224, 0.225] order: 'hwc' - ToCHWImage: - KeepKeys: keep_keys: ['image', 'p3_maps', 'p4_maps', 'p5_maps'] # dataloader will return list in this order loader: shuffle: True drop_last: False batch_size_per_card: 6 num_workers: 8 Eval: dataset: name: SimpleDataSet data_dir: ./train_data/ctw1500/imgs/ label_file_list: - ./train_data/ctw1500/imgs/test.txt transforms: - DecodeImage: # load image img_mode: BGR channel_first: False ignore_orientation: True - DetLabelEncode: # Class handling label - DetResizeForTest: limit_type: 'min' limit_side_len: 736 - NormalizeImage: scale: 1./255. mean: [0.485, 0.456, 0.406] std: [0.229, 0.224, 0.225] order: 'hwc' - Pad: - ToCHWImage: - KeepKeys: keep_keys: ['image', 'shape', 'polys', 'ignore_tags'] loader: shuffle: False drop_last: False batch_size_per_card: 1 # must be 1 num_workers: 2 ```
:bug: The code raised the following error: ``` --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) /tmp/ipykernel_252/2829029886.py in ----> 1 export_single_model(model, arch_config, save_path, logger) /paddle/PaddleOCR/tools/export_model.py in export_single_model(model, arch_config, save_path, logger, quanter) 97 98 if quanter is None: ---> 99 paddle.jit.save(model, save_path) 100 else: 101 quanter.save_quantized_model(model, save_path) /usr/local/lib/python3.7/dist-packages/decorator.py in fun(*args, **kw) 230 if not kwsyntax: 231 args, kw = fix(args, kw, sig) --> 232 return caller(func, *(extras + args), **kw) 233 fun.__name__ = func.__name__ 234 fun.__doc__ = func.__doc__ /usr/local/lib/python3.7/dist-packages/paddle/fluid/wrapped_decorator.py in __impl__(func, *args, **kwargs) 23 def __impl__(func, *args, **kwargs): 24 wrapped_func = decorator_func(func) ---> 25 return wrapped_func(*args, **kwargs) 26 27 return __impl__ /usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/base.py in __impl__(*args, **kwargs) 38 def __impl__(*args, **kwargs): 39 with framework._dygraph_guard(None): ---> 40 return func(*args, **kwargs) 41 42 return __impl__ /usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/jit.py in save(layer, path, input_spec, **configs) 724 if isinstance(static_func, StaticFunction): 725 concrete_program = static_func.concrete_program_specify_input_spec( --> 726 inner_input_spec) 727 elif 'forward' == attr_func: 728 # transform in jit.save, if input_spec is incomplete, declarative will throw error /usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py in concrete_program_specify_input_spec(self, input_spec) 489 if has_input_spec: 490 concrete_program, _ = self.get_concrete_program( --> 491 *desired_input_spec) 492 return concrete_program 493 else: /usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py in get_concrete_program(self, *args, **kwargs) 399 400 # 3. check whether hit the cache or build a new program for the input arguments --> 401 concrete_program, partial_program_layer = self._program_cache[cache_key] 402 return concrete_program, partial_program_layer 403 /usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py in __getitem__(self, item) 712 713 if item not in self._caches: --> 714 self._caches[item] = self._build_once(item) 715 # Note: raise warnings if number of traced program is more than `max_tracing_count` 716 current_tracing_count = len(self._caches) /usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py in _build_once(self, cache_key) 703 input_spec=cache_key.input_args_with_spec, 704 input_kwargs_spec=cache_key.input_kwargs_with_spec, --> 705 class_instance=cache_key.class_instance) 706 return concrete_program, partial_program_from(concrete_program) 707 /usr/local/lib/python3.7/dist-packages/decorator.py in fun(*args, **kw) 230 if not kwsyntax: 231 args, kw = fix(args, kw, sig) --> 232 return caller(func, *(extras + args), **kw) 233 fun.__name__ = func.__name__ 234 fun.__doc__ = func.__doc__ /usr/local/lib/python3.7/dist-packages/paddle/fluid/wrapped_decorator.py in __impl__(func, *args, **kwargs) 23 def __impl__(func, *args, **kwargs): 24 wrapped_func = decorator_func(func) ---> 25 return wrapped_func(*args, **kwargs) 26 27 return __impl__ /usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/base.py in __impl__(*args, **kwargs) 38 def __impl__(*args, **kwargs): 39 with framework._dygraph_guard(None): ---> 40 return func(*args, **kwargs) 41 42 return __impl__ /usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/program_translator.py in from_func_spec(func_spec, input_spec, input_kwargs_spec, class_instance) 659 error_data = getattr(e, error.ERROR_DATA, None) 660 if error_data: --> 661 error_data.raise_new_exception() 662 raise 663 /usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/error.py in raise_new_exception(self) 190 # 2. Use exec to bypass syntax error checking in Python 2. 191 --> 192 six.exec_("raise new_exception from None") 193 else: 194 raise new_exception /usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/error.py in AttributeError: In transformed code: File "/tmp/tmpuk648npq.py", line 29, in forward false_fn_1, (x,), (x,), (x,)) File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/convert_operators.py", line 210, in convert_ifelse return _run_py_ifelse(pred, true_fn, false_fn, true_args, false_args) File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/convert_operators.py", line 235, in _run_py_ifelse return true_fn(*true_args) if pred else false_fn(*false_args) File "/paddle/PaddleOCR/ppocr/modeling/architectures/base_model.py", line 82, in forward (* user code *) x = self.neck(x) File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 902, in __call__ outputs = self.forward(*inputs, **kwargs) File "/tmp/tmpw35ww39x.py", line 36, in forward self, x), (__return_value_4, self, x), (__return_value_4, x)) File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/convert_operators.py", line 210, in convert_ifelse return _run_py_ifelse(pred, true_fn, false_fn, true_args, false_args) File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/dygraph_to_static/convert_operators.py", line 235, in _run_py_ifelse return true_fn(*true_args) if pred else false_fn(*false_args) File "/paddle/PaddleOCR/ppocr/modeling/necks/rnn.py", line 184, in forward (* user code *) x = self.encoder_reshape(x) File "/usr/local/lib/python3.7/dist-packages/paddle/fluid/dygraph/layers.py", line 902, in __call__ outputs = self.forward(*inputs, **kwargs) File "/paddle/PaddleOCR/ppocr/modeling/necks/rnn.py", line 32, in forward (* user code *) B, C, H, W = x.shape AttributeError: 'tuple' object has no attribute 'shape' ```

:palm_tree: Environments

tann9949 commented 2 years ago

I've dig on this bug a little more, it seems that the model failed to forward after the function to_static was called. I assumed that somehow to_statc function breaks the forward pipeline?

tann9949 commented 2 years ago

This looks like #4658

tann9949 commented 2 years ago

From what I have debugged, it seems that the to_static function causing the input that parsed on ImSeq class (at PaddleOCR/ppocr/modeling/neck/rnn.py to be tuple instead of paddle's Tensor.

I got this fixed by modifying ImSeq class as follows:

class Im2Seq(nn.Layer):
    def __init__(self, in_channels, **kwargs):
        super().__init__()
        self.out_channels = in_channels

    def forward(self, x):
        if isinstance(x, tuple):
            x = x[0]
        B, C, H, W = x.shape
        assert H == 1
        x = x.squeeze(axis=2)
        x = x.transpose([0, 2, 1])  # (NTC)(batch, width, channels)
        return x

and now things work just fine. I personally think this is not a good solution, but i'll let the Paddle team decide on this issue. Closing.