训练时的相关问题

PhilCuriosity commented 3 years ago

目前的训练文档很多链接是空的，后续会补齐吗？

WenmuZhou commented 3 years ago

麻烦指出一下是哪些链接，我们好补

PhilCuriosity commented 3 years ago

第一个红框中的链接是空的，模型可以从第二个链接中找到。我现在使用PPOCRLabel标注好数据以后，对数据划分为了train.txt和test.txt中，图片放在同一个文件夹中启动训练，但没有结果配置文件写的是SimpleDataSet，我想问的是需要对图片进行移动，将其变为icdar的那种训练图片和测试图片分开的形式吗？

PhilCuriosity commented 3 years ago

像这种，没有报错，没继续训练，不知道原因是什么。

PhilCuriosity commented 3 years ago

全部打印：

--》 python tools/train.py -c configs/det/det_mv3_db.yml D:\SoftwareWork\Anaconda3\lib\site-packages\numpy_distributor_init.py:30: UserWarning: loaded more than 1 DLL from .libs: D:\SoftwareWork\Anaconda3\lib\site-packages\numpy.libs\libopenblas.PYQHXLVVQ7VESDPUVUADXEVJOBGHJPAY.gfortran-win_amd64.dll D:\SoftwareWork\Anaconda3\lib\site-packages\numpy.libs\libopenblas.QVLO2T66WEPI7JZ63PS3HMOHFEY472BC.gfortran-win_amd64.dll warnings.warn("loaded more than 1 DLL from .libs:" D:\SoftwareWork\Anaconda3\lib\site-packages\urllib3\util\selectors.py:14: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collecti ons.abc' is deprecated since Python 3.3, and in 3.9 it will stop working from collections import namedtuple, Mapping D:\SoftwareWork\Anaconda3\lib\site-packages\urllib3_collections.py:2: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections .abc' is deprecated since Python 3.3, and in 3.9 it will stop working from collections import Mapping, MutableMapping [2021/07/14 10:22:58] root INFO: Architecture : [2021/07/14 10:22:58] root INFO: Backbone : [2021/07/14 10:22:58] root INFO: model_name : large [2021/07/14 10:22:58] root INFO: name : MobileNetV3 [2021/07/14 10:22:58] root INFO: scale : 1.0 [2021/07/14 10:22:58] root INFO: Head : [2021/07/14 10:22:58] root INFO: k : 50 [2021/07/14 10:22:58] root INFO: name : DBHead [2021/07/14 10:22:58] root INFO: Neck : [2021/07/14 10:22:58] root INFO: name : DBFPN [2021/07/14 10:22:58] root INFO: out_channels : 256 [2021/07/14 10:22:58] root INFO: Transform : None [2021/07/14 10:22:58] root INFO: algorithm : DB [2021/07/14 10:22:58] root INFO: model_type : det [2021/07/14 10:22:58] root INFO: Eval : [2021/07/14 10:22:58] root INFO: dataset : [2021/07/14 10:22:58] root INFO: data_dir : D:\Data**\Print\testSavingImages\20210709\Train\print [2021/07/14 10:22:58] root INFO: label_file_list : ['D:\Data\**\Print\testSavingImages\20210709\Train\print\test.txt'] [2021/07/14 10:22:58] root INFO: name : SimpleDataSet [2021/07/14 10:22:58] root INFO: transforms : [2021/07/14 10:22:58] root INFO: DecodeImage : [2021/07/14 10:22:58] root INFO: channel_first : False [2021/07/14 10:22:58] root INFO: img_mode : BGR [2021/07/14 10:22:58] root INFO: DetLabelEncode : None [2021/07/14 10:22:58] root INFO: DetResizeForTest : [2021/07/14 10:22:58] root INFO: image_shape : [736, 1280] [2021/07/14 10:22:58] root INFO: NormalizeImage : [2021/07/14 10:22:58] root INFO: mean : [0.485, 0.456, 0.406] [2021/07/14 10:22:58] root INFO: order : hwc [2021/07/14 10:22:58] root INFO: scale : 1./255. [2021/07/14 10:22:58] root INFO: std : [0.229, 0.224, 0.225] [2021/07/14 10:22:58] root INFO: ToCHWImage : None [2021/07/14 10:22:58] root INFO: KeepKeys : [2021/07/14 10:22:58] root INFO: keep_keys : ['image', 'shape', 'polys', 'ignore_tags'] [2021/07/14 10:22:58] root INFO: loader : [2021/07/14 10:22:58] root INFO: batch_size_per_card : 1 [2021/07/14 10:22:58] root INFO: drop_last : False [2021/07/14 10:22:58] root INFO: num_workers : 8 [2021/07/14 10:22:58] root INFO: shuffle : False [2021/07/14 10:22:58] root INFO: use_shared_memory : False [2021/07/14 10:22:58] root INFO: Global : [2021/07/14 10:22:58] root INFO: cal_metric_during_train : False [2021/07/14 10:22:58] root INFO: checkpoints : None [2021/07/14 10:22:58] root INFO: debug : False [2021/07/14 10:22:58] root INFO: distributed : False [2021/07/14 10:22:58] root INFO: epoch_num : 1200 [2021/07/14 10:22:58] root INFO: eval_batch_step : [0, 10] [2021/07/14 10:22:58] root INFO: infer_img : None [2021/07/14 10:22:58] root INFO: log_smooth_window : 20 [2021/07/14 10:22:58] root INFO: pretrained_model : ./pretrain_models/MobileNetV3_large_x1_0_ssld_pretrained [2021/07/14 10:22:58] root INFO: print_batch_step : 10 [2021/07/14 10:22:58] root INFO: save_epoch_step : 5 [2021/07/14 10:22:58] root INFO: save_inference_dir : None [2021/07/14 10:22:58] root INFO: save_model_dir : ./output/db_mv3/ [2021/07/14 10:22:58] root INFO: save_res_path : None [2021/07/14 10:22:58] root INFO: use_gpu : True [2021/07/14 10:22:58] root INFO: use_visualdl : False [2021/07/14 10:22:58] root INFO: Loss : [2021/07/14 10:22:58] root INFO: alpha : 5 [2021/07/14 10:22:58] root INFO: balance_loss : True [2021/07/14 10:22:58] root INFO: beta : 10 [2021/07/14 10:22:58] root INFO: main_loss_type : DiceLoss [2021/07/14 10:22:58] root INFO: name : DBLoss [2021/07/14 10:22:58] root INFO: ohem_ratio : 3 [2021/07/14 10:22:58] root INFO: Metric : [2021/07/14 10:22:58] root INFO: main_indicator : hmean [2021/07/14 10:22:58] root INFO: name : DetMetric [2021/07/14 10:22:58] root INFO: Optimizer : [2021/07/14 10:22:58] root INFO: beta1 : 0.9 [2021/07/14 10:22:58] root INFO: beta2 : 0.999 [2021/07/14 10:22:58] root INFO: lr : [2021/07/14 10:22:58] root INFO: learning_rate : 0.001 [2021/07/14 10:22:58] root INFO: name : Adam [2021/07/14 10:22:58] root INFO: regularizer : [2021/07/14 10:22:58] root INFO: factor : 0 [2021/07/14 10:22:58] root INFO: name : L2 [2021/07/14 10:22:58] root INFO: PostProcess : [2021/07/14 10:22:58] root INFO: box_thresh : 0.6 [2021/07/14 10:22:58] root INFO: max_candidates : 1000 [2021/07/14 10:22:58] root INFO: name : DBPostProcess [2021/07/14 10:22:58] root INFO: thresh : 0.3 [2021/07/14 10:22:58] root INFO: unclip_ratio : 1.5 [2021/07/14 10:22:58] root INFO: Train : [2021/07/14 10:22:58] root INFO: dataset : [2021/07/14 10:22:58] root INFO: data_dir : D:\Data**\Print\testSavingImages\20210709\Train\print [2021/07/14 10:22:58] root INFO: label_file_list : ['D:\Data\**\Print\testSavingImages\20210709\Train\print\train.txt'] [2021/07/14 10:22:58] root INFO: name : SimpleDataSet [2021/07/14 10:22:58] root INFO: ratio_list : [1.0] [2021/07/14 10:22:58] root INFO: transforms : [2021/07/14 10:22:58] root INFO: DecodeImage : [2021/07/14 10:22:58] root INFO: channel_first : False [2021/07/14 10:22:58] root INFO: img_mode : BGR [2021/07/14 10:22:58] root INFO: DetLabelEncode : None [2021/07/14 10:22:58] root INFO: IaaAugment : [2021/07/14 10:22:58] root INFO: augmenter_args : [2021/07/14 10:22:58] root INFO: args : [2021/07/14 10:22:58] root INFO: p : 0.5 [2021/07/14 10:22:58] root INFO: type : Fliplr [2021/07/14 10:22:58] root INFO: args : [2021/07/14 10:22:58] root INFO: rotate : [-15, 15] [2021/07/14 10:22:58] root INFO: type : Affine [2021/07/14 10:22:58] root INFO: args : [2021/07/14 10:22:58] root INFO: size : [0.5, 3] [2021/07/14 10:22:58] root INFO: type : Resize [2021/07/14 10:22:58] root INFO: EastRandomCropData : [2021/07/14 10:22:58] root INFO: keep_ratio : True [2021/07/14 10:22:58] root INFO: max_tries : 50 [2021/07/14 10:22:58] root INFO: size : [488, 288] [2021/07/14 10:22:58] root INFO: MakeBorderMap : [2021/07/14 10:22:58] root INFO: shrink_ratio : 0.4 [2021/07/14 10:22:58] root INFO: thresh_max : 0.7 [2021/07/14 10:22:58] root INFO: thresh_min : 0.3 [2021/07/14 10:22:58] root INFO: MakeShrinkMap : [2021/07/14 10:22:58] root INFO: min_text_size : 8 [2021/07/14 10:22:58] root INFO: shrink_ratio : 0.4 [2021/07/14 10:22:58] root INFO: NormalizeImage : [2021/07/14 10:22:58] root INFO: mean : [0.485, 0.456, 0.406] [2021/07/14 10:22:58] root INFO: order : hwc [2021/07/14 10:22:58] root INFO: scale : 1./255. [2021/07/14 10:22:58] root INFO: std : [0.229, 0.224, 0.225] [2021/07/14 10:22:58] root INFO: ToCHWImage : None [2021/07/14 10:22:58] root INFO: KeepKeys : [2021/07/14 10:22:58] root INFO: keep_keys : ['image', 'threshold_map', 'threshold_mask', 'shrink_map', 'shrink_mask'] [2021/07/14 10:22:58] root INFO: loader : [2021/07/14 10:22:58] root INFO: batch_size_per_card : 16 [2021/07/14 10:22:58] root INFO: drop_last : False [2021/07/14 10:22:58] root INFO: num_workers : 8 [2021/07/14 10:22:58] root INFO: shuffle : True [2021/07/14 10:22:58] root INFO: use_shared_memory : False [2021/07/14 10:22:58] root INFO: train with paddle 2.1.0 and device CUDAPlace(0) [2021/07/14 10:22:58] root INFO: Initialize indexs of datasets:['D:\Data\\Print\testSavingImages\20210709\Train\print\train.txt'] [2021/07/14 10:22:58] root INFO: Initialize indexs of datasets:['D:\Data\\Print\testSavingImages\20210709\Train\print\test.txt'] W0714 10:22:58.576777 9768 device_context.cc:404] Please NOTE: device: 0, GPU Compute Capability: 7.5, Driver API Version: 11.2, Runtime API Version: 10.2 W0714 10:22:58.586751 9768 device_context.cc:422] device: 0, cuDNN Version: 7.6. [2021/07/14 10:23:02] root INFO: load pretrained model from ['./pretrain_models/MobileNetV3_large_x1_0_ssld_pretrained'] [2021/07/14 10:23:02] root INFO: train dataloader has 11 iters [2021/07/14 10:23:02] root INFO: valid dataloader has 18 iters [2021/07/14 10:23:02] root INFO: During the training process, after the 0th iteration, an evaluation is run every 10 iterations [2021/07/14 10:23:02] root INFO: Initialize indexs of datasets:['D:\Data\****\Print\testSavingImages\20210709\Train\print\train.txt']

PhilCuriosity commented 3 years ago

数据目录结构：

启动训练命令： python tools/train.py -c configs/det/ch_ppocr_v2.0/ch_det_mv3_db_v2.0.yml

ch_det_mv3_db_v2.0.yml相关配置 Global: use_gpu: true epoch_num: 1200 log_smooth_window: 20 print_batch_step: 2 save_model_dir: ./output/ch_db_mv3/ save_epoch_step: 1200 eval_batch_step: 5
cal_metric_during_train: False pretrained_model: ./pretrain_models/ch_ppocr_mobile_v2.0_det_train/best_accuracy
checkpoints: save_inference_dir: use_visualdl: False infer_img:
save_res_path:

Architecture: model_type: det algorithm: DB Transform: Backbone: name: MobileNetV3 scale: 0.5 model_name: large disable_se: True Neck: name: DBFPN out_channels: 96 Head: name: DBHead k: 50

Loss: name: DBLoss balance_loss: true main_loss_type: DiceLoss alpha: 5 beta: 10 ohem_ratio: 3

Optimizer: name: Adam beta1: 0.9 beta2: 0.999 lr: name: Cosine learning_rate: 0.001 warmup_epoch: 2 regularizer: name: 'L2' factor: 0

PostProcess: name: DBPostProcess thresh: 0.3 box_thresh: 0.6 max_candidates: 1000 unclip_ratio: 1.5

Metric: name: DetMetric main_indicator: hmean

Train: dataset: name: SimpleDataSet data_dir: D:\Data\￥\Print\testSavingImages\20210709\Train\print
label_file_list:

D:\Data\￥\Print\testSavingImages\20210709\Train\print\train.txt
ratio_list: [1.0] transforms:
DecodeImage: # load image img_mode: BGR channel_first: False
DetLabelEncode: # Class handling label
IaaAugment: augmenter_args:
- { 'type': Fliplr, 'args': { 'p': 0.5 } }
- { 'type': Affine, 'args': { 'rotate': [-10, 10] } }
- { 'type': Resize, 'args': { 'size': [0.5, 3] } }
EastRandomCropData: size: [488, 288] max_tries: 50 keep_ratio: true
MakeBorderMap: shrink_ratio: 0.4 thresh_min: 0.3 thresh_max: 0.7
MakeShrinkMap: shrink_ratio: 0.4 min_text_size: 8
NormalizeImage: scale: 1./255. mean: [0.485, 0.456, 0.406] std: [0.229, 0.224, 0.225] order: 'hwc'
ToCHWImage:
KeepKeys: keep_keys: ['image', 'threshold_map', 'threshold_mask', 'shrink_map', 'shrink_mask'] # the order of the dataloader list loader: shuffle: True drop_last: False batch_size_per_card: 8 num_workers: 4

Eval: dataset: name: SimpleDataSet data_dir: D:\Data\￥\Print\testSavingImages\20210709\Train\print label_file_list:

D:\Data\￥\Print\testSavingImages\20210709\Train\print\test.txt
transforms:
DecodeImage: # load image img_mode: BGR channel_first: False
DetLabelEncode: # Class handling label
DetResizeForTest: image_shape: [488, 288]
NormalizeImage: scale: 1./255. mean: [0.485, 0.456, 0.406] std: [0.229, 0.224, 0.225] order: 'hwc'
ToCHWImage:
KeepKeys: keep_keys: ['image', 'shape', 'polys', 'ignore_tags'] loader: shuffle: False drop_last: False batch_size_per_card: 1 # must be 1 num_workers: 2

PaddlePaddle / PaddleOCR

训练时的相关问题 #3320