下载的Chinese and English ultra-lightweight PP-OCRv2 model（11.6M）Detection model 里的训练模型，拿icdar2015的数据评估模型,hmean等指标一直为0

[mikiihuang]

请提供下述完整信息以便快速定位问题/Please provide the following information to quickly locate the problem

• 系统环境/System Environment：bclinux 7.6

• 版本号/Version：Paddle：release-2.5版本 PaddleOCR： 问题相关组件/Related components：

• 运行指令/Command Code： python3 tools/eval.py -c configs/det/ch_PP-OCRv2/ch_PP-OCRv2_det_distill.yml -o Global.checkpoints="pretrain_models/ch_PP-OCRv2_det_distill_train/best_accuracy"

• 完整报错/Complete Error Message： [2023/02/02 08:10:16] ppocr INFO: resume from pretrain_models/ch_PP-OCRv2_det_distill_train/best_accuracy eval model:: 100%|█████████████████████████████████████████████████████████████████████████████████████████████| 500/500 [00:35<00:00, 14.09it/s] [2023/02/02 08:10:52] ppocr INFO: metric eval *** [2023/02/02 08:10:52] ppocr INFO: precision:0 [2023/02/02 08:10:52] ppocr INFO: recall:0.0 [2023/02/02 08:10:52] ppocr INFO: hmean:0 [2023/02/02 08:10:52] ppocr INFO: fps:15.99793361651514

yml配置文件没有做任何改动，如下：

Global:
  use_gpu: true
  epoch_num: 1200
  log_smooth_window: 20
  print_batch_step: 2
  save_model_dir: ./output/ch_db_mv3/
  save_epoch_step: 1200
  # evaluation is run every 5000 iterations after the 4000th iteration
  eval_batch_step: [3000, 2000]
  cal_metric_during_train: False
  pretrained_model: ./pretrain_models/MobileNetV3_large_x0_5_pretrained
  checkpoints:
  save_inference_dir:
  use_visualdl: False
  infer_img: doc/imgs_en/img_10.jpg
  save_res_path: ./output/det_db/predicts_db.txt

Architecture:
  name: DistillationModel
  algorithm: Distillation
  model_type: det
  Models:
    Student:
      pretrained: ./pretrain_models/MobileNetV3_large_x0_5_pretrained
      freeze_params: false
      return_all_feats: false
      model_type: det
      algorithm: DB
      Backbone:
        name: MobileNetV3
        scale: 0.5
        model_name: large
        disable_se: True
      Neck:
        name: DBFPN
        out_channels: 96
      Head:
        name: DBHead
        k: 50
    Teacher:
      pretrained: ./pretrain_models/ch_ppocr_server_v2.0_det_train/best_accuracy
      freeze_params: true
      return_all_feats: false
      model_type: det
      algorithm: DB
      Transform:
      Backbone:
        name: ResNet
        layers: 18
      Neck:
        name: DBFPN
        out_channels: 256
      Head:
        name: DBHead
        k: 50

Loss:
  name: CombinedLoss
  loss_config_list:
  - DistillationDilaDBLoss:
      weight: 1.0
      model_name_pairs:
      - ["Student", "Teacher"]
      key: maps
      balance_loss: true
      main_loss_type: DiceLoss
      alpha: 5
      beta: 10
      ohem_ratio: 3
  - DistillationDBLoss:
      weight: 1.0
      model_name_list: ["Student"]
      name: DBLoss
      balance_loss: true
      main_loss_type: DiceLoss
      alpha: 5
      beta: 10
      ohem_ratio: 3

Optimizer:
  name: Adam
  beta1: 0.9
  beta2: 0.999
  lr:
    name: Cosine
    learning_rate: 0.001
    warmup_epoch: 2
  regularizer:
    name: 'L2'
    factor: 0

PostProcess:
  name: DistillationDBPostProcess
  model_name: ["Student"]
  key: head_out
  thresh: 0.3
  box_thresh: 0.6
  max_candidates: 1000
  unclip_ratio: 1.5

Metric:
  name: DistillationMetric
  base_metric_name: DetMetric
  main_indicator: hmean
  key: "Student"

Train:
  dataset:
    name: SimpleDataSet
    data_dir: ./train_data/icdar2015/text_localization/
    label_file_list:
      - ./train_data/icdar2015/text_localization/train_icdar2015_label.txt
    ratio_list: [1.0]
    transforms:
      - DecodeImage: # load image
          img_mode: BGR
          channel_first: False
      - DetLabelEncode: # Class handling label
      - CopyPaste:
      - IaaAugment:
          augmenter_args:
            - { 'type': Fliplr, 'args': { 'p': 0.5 } }
            - { 'type': Affine, 'args': { 'rotate': [-10, 10] } }
            - { 'type': Resize, 'args': { 'size': [0.5, 3] } }
      - EastRandomCropData:
          size: [960, 960]
          max_tries: 50
          keep_ratio: true
      - MakeBorderMap:
          shrink_ratio: 0.4
          thresh_min: 0.3
          thresh_max: 0.7
      - MakeShrinkMap:
          shrink_ratio: 0.4
          min_text_size: 8
      - NormalizeImage:
          scale: 1./255.
          mean: [0.485, 0.456, 0.406]
          std: [0.229, 0.224, 0.225]
          order: 'hwc'
      - ToCHWImage:
      - KeepKeys:
          keep_keys: ['image', 'threshold_map', 'threshold_mask', 'shrink_map', 'shrink_mask'] # the order of the dataloader list
  loader:
    shuffle: True
    drop_last: False
    batch_size_per_card: 8
    num_workers: 4

Eval:
  dataset:
    name: SimpleDataSet
    data_dir: ./train_data/icdar2015/text_localization/
    label_file_list:
      - ./train_data/icdar2015/text_localization/test_icdar2015_label.txt
    transforms:
      - DecodeImage: # load image
          img_mode: BGR
          channel_first: False
      - DetLabelEncode: # Class handling label
      - DetResizeForTest:
#           image_shape: [736, 1280]
      - NormalizeImage:
          scale: 1./255.
          mean: [0.485, 0.456, 0.406]
          std: [0.229, 0.224, 0.225]
          order: 'hwc'
      - ToCHWImage:
      - KeepKeys:
          keep_keys: ['image', 'shape', 'polys', 'ignore_tags']
  loader:
    shuffle: False
    drop_last: False
    batch_size_per_card: 1 # must be 1
    num_workers: 2

在icdar2015数据集上做的评估.help!

[LDOUBLEV]

标注格式不同 PaddleOCR发布的PP-OCR系列模型的检测识别是以『行』为单位的，icdar的是『word』为单位检测识别的

[LDOUBLEV]

如果需要在开源数据集上的模型，可以去翻翻文档，比如DRRG的 https://github.com/PaddlePaddle/PaddleOCR/blob/release%2F2.6/doc/doc_ch/algorithm_det_drrg.md

[mikiihuang]

如果需要在开源数据集上的模型，可以去翻翻文档，比如DRRG的 https://github.com/PaddlePaddle/PaddleOCR/blob/release%2F2.6/doc/doc_ch/algorithm_det_drrg.md

请问大佬有没有什么公开的数据集是和PP-OCR系列模型的检测识别使用的一样的标注格式呀？

[LDOUBLEV]

请问大佬有没有什么公开的数据集是和PP-OCR系列模型的检测识别使用的一样的标注格式呀？

LSVT CTW MTWI 这些都是，句子级别标注的都可以