PaddlePaddle / Paddle

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice (『飞桨』核心框架,深度学习&机器学习高性能单机、分布式训练和跨平台部署)
http://www.paddlepaddle.org/
Apache License 2.0
22.08k stars 5.55k forks source link

使用paddle2.0先后使用两个模型进行预测,提示[XXXX]have be been used. In dygraph mode,该怎么解决 #30856

Closed cgq0816 closed 3 years ago

cgq0816 commented 3 years ago

使用paddle2.0时分别训练两个模型,依次使用两个模型进行预测,其中一个模型预测成功后,再使用另外一个模型在进行预测的时候提示[XXXX]have be been used. In dygraph mode, the name of parameter can't be same.Please check the parameter attr value passed to self.create_parameter or constructor of dygraph Layers 请问一下这个问题怎么解决?

paddle-bot-old[bot] commented 3 years ago

您好,我们已经收到了您的问题,会安排技术人员尽快解答您的问题,请耐心等待。请您再次检查是否提供了清晰的问题描述、复现代码、环境&版本、报错信息等。同时,您也可以通过查看官网API文档常见问题历史IssueAI社区来寻求解答。祝您生活愉快~

Hi! We've received your issue and please be patient to get responded. We will arrange technicians to answer your questions as soon as possible. Please make sure that you have posted enough message to demo your request. You may also check out the APIFAQGithub Issue and AI community to get the answer.Have a nice day!

chenwhql commented 3 years ago

麻烦贴一下demo代码,和完整的报错信息

cgq0816 commented 3 years ago

麻烦贴一下demo代码,和完整的报错信息 环境: paddleocr 2.0.2 paddlepaddle-gpu 2.0.0rc1.post101

用的是paddleocr做的训练任务,分别训练检测模型和识别模型,然后分别使用两个模型对数据进行预测

调用的train.py和eval.py https://github.com/PaddlePaddle/PaddleOCR/tree/dygraph/tools

用训练好的模型对数据进行预测 DET模型 image

REC模型 image

训练之后保存的模型都是best_accuracy的: best_accuracy.pdopt best_accuracy.pdparams best_accuracy.states

报错信息如下 INFO:root:Architecture : INFO:root: Backbone : INFO:root: disable_se : True INFO:root: layers : 18 INFO:root: name : ResNet INFO:root: Head : INFO:root: k : 50 INFO:root: name : DBHead INFO:root: Neck : INFO:root: name : DBFPN INFO:root: out_channels : 256 INFO:root: Transform : None INFO:root: algorithm : DB INFO:root: model_type : det INFO:root:Eval : INFO:root: dataset : INFO:root: data_dir : image/370/OCR_DET/predict/ INFO:root: label_file_list : ['image/370/OCR_DET/predict/eval_label.txt'] INFO:root: name : SimpleDataSet INFO:root: transforms : INFO:root: DecodeImage : INFO:root: channel_first : False INFO:root: img_mode : BGR INFO:root: DetLabelEncode : None INFO:root: DetResizeForTest : None INFO:root: NormalizeImage : INFO:root: mean : [0.485, 0.456, 0.406] INFO:root: order : hwc INFO:root: scale : 1./255. INFO:root: std : [0.229, 0.224, 0.225] INFO:root: ToCHWImage : None INFO:root: KeepKeys : INFO:root: keep_keys : ['image', 'shape', 'polys', 'ignore_tags'] INFO:root: loader : INFO:root: batch_size_per_card : 1 INFO:root: drop_last : False INFO:root: num_workers : 1 INFO:root: shuffle : False INFO:root:Global : INFO:root: cal_metric_during_train : False INFO:root: checkpoints : models/1194/370/OCR_DET/OCR/best_accuracy INFO:root: debug : False INFO:root: distributed : False INFO:root: epoch_num : 1200 INFO:root: eval_batch_step : [3000, 2000] INFO:root: infer_img : doc/imgs_en/img_10.jpg INFO:root: load_static_weights : True INFO:root: log_smooth_window : 20 INFO:root: pretrained_model : ./pretrain_models/ResNet18_vd_pretrained INFO:root: print_batch_step : 2 INFO:root: save_epoch_step : 1200 INFO:root: save_inference_dir : None INFO:root: save_model_dir : ./output/ch_db_res18/ INFO:root: save_res_path : ./output/det_db/predicts_db.txt INFO:root: use_gpu : False INFO:root: use_visualdl : False INFO:root:Loss : INFO:root: alpha : 5 INFO:root: balance_loss : True INFO:root: beta : 10 INFO:root: main_loss_type : DiceLoss INFO:root: name : DBLoss INFO:root: ohem_ratio : 3 INFO:root:Metric : INFO:root: main_indicator : hmean INFO:root: name : DetMetric INFO:root:Optimizer : INFO:root: beta1 : 0.9 INFO:root: beta2 : 0.999 INFO:root: lr : INFO:root: learning_rate : 0.001 INFO:root: name : Cosine INFO:root: warmup_epoch : 2 INFO:root: name : Adam INFO:root: regularizer : INFO:root: factor : 0 INFO:root: name : L2 INFO:root:PostProcess : INFO:root: box_thresh : 0.5 INFO:root: max_candidates : 1000 INFO:root: name : DBPostProcess INFO:root: thresh : 0.3 INFO:root: unclip_ratio : 1.5 INFO:root:Train : INFO:root: dataset : INFO:root: data_dir : ./train_data/icdar2015/text_localization/ INFO:root: label_file_list : ['./train_data/icdar2015/text_localization/train_icdar2015_label.txt'] INFO:root: name : SimpleDataSet INFO:root: ratio_list : [1.0] INFO:root: transforms : INFO:root: DecodeImage : INFO:root: channel_first : False INFO:root: img_mode : BGR INFO:root: DetLabelEncode : None INFO:root: IaaAugment : INFO:root: augmenter_args : INFO:root: args : INFO:root: p : 0.5 INFO:root: type : Fliplr INFO:root: args : INFO:root: rotate : [-10, 10] INFO:root: type : Affine INFO:root: args : INFO:root: size : [0.5, 3] INFO:root: type : Resize INFO:root: EastRandomCropData : INFO:root: keep_ratio : True INFO:root: max_tries : 50 INFO:root: size : [960, 960] INFO:root: MakeBorderMap : INFO:root: shrink_ratio : 0.4 INFO:root: thresh_max : 0.7 INFO:root: thresh_min : 0.3 INFO:root: MakeShrinkMap : INFO:root: min_text_size : 8 INFO:root: shrink_ratio : 0.4 INFO:root: NormalizeImage : INFO:root: mean : [0.485, 0.456, 0.406] INFO:root: order : hwc INFO:root: scale : 1./255. INFO:root: std : [0.229, 0.224, 0.225] INFO:root: ToCHWImage : None INFO:root: KeepKeys : INFO:root: keep_keys : ['image', 'threshold_map', 'threshold_mask', 'shrink_map', 'shrink_mask'] INFO:root: loader : INFO:root: batch_size_per_card : 1 INFO:root: drop_last : False INFO:root: num_workers : 1 INFO:root: shuffle : True INFO:root:train with paddle 2.0.0-rc1 and device CPUPlace INFO:root:Initialize indexs of datasets:['image/370/OCR_DET/predict/eval_label.txt'] INFO:root:resume from models/1194/370/OCR_DET/OCR/best_accuracy INFO:root:metric in ckpt *** INFO:root:hmean:0 INFO:root:precision:0 INFO:root:recall:0.0 INFO:root:fps:1.7757967219999933 INFO:root:best_epoch:10 INFO:root:start_epoch:11

eval model:: 0%| | 0/5 [00:00<?, ?it/s] eval model:: 20%|██ | 1/5 [00:05<00:23, 5.99s/it] eval model:: 40%|████ | 2/5 [00:11<00:17, 5.75s/it] eval model:: 60%|██████ | 3/5 [00:16<00:11, 5.57s/it] eval model:: 80%|████████ | 4/5 [00:21<00:05, 5.48s/it] eval model:: 100%|██████████| 5/5 [00:26<00:00, 5.40s/it] eval model:: 100%|██████████| 5/5 [00:26<00:00, 5.36s/it] INFO:root:metric eval *** INFO:root:precision:0.0 INFO:root:recall:0.0 INFO:root:hmean:0 INFO:root:fps:0.18841917037334524

INFO:root:Architecture : INFO:root: Backbone : INFO:root: layers : 34 INFO:root: name : ResNet INFO:root: Head : INFO:root: fc_decay : 4e-05 INFO:root: name : CTCHead INFO:root: Neck : INFO:root: encoder_type : rnn INFO:root: hidden_size : 256 INFO:root: name : SequenceEncoder INFO:root: Transform : None INFO:root: algorithm : CRNN INFO:root: model_type : rec INFO:root:Eval : INFO:root: dataset : INFO:root: data_dir : image/370/OCR_REC/predict/ INFO:root: label_file_list : ['image/370/OCR_REC/predict/eval_label.txt'] INFO:root: name : SimpleDataSet INFO:root: transforms : INFO:root: DecodeImage : INFO:root: channel_first : False INFO:root: img_mode : BGR INFO:root: CTCLabelEncode : None INFO:root: RecResizeImg : INFO:root: image_shape : [3, 32, 320] INFO:root: KeepKeys : INFO:root: keep_keys : ['image', 'label', 'length'] INFO:root: loader : INFO:root: batch_size_per_card : 256 INFO:root: drop_last : False INFO:root: num_workers : 8 INFO:root: shuffle : False INFO:root:Global : INFO:root: cal_metric_during_train : True INFO:root: character_dict_path : ./PaddleOCR/ppocr/utils/ppocr_keys_v1.txt INFO:root: character_type : ch INFO:root: checkpoints : models/1195/370/OCR_REC/OCR/best_accuracy INFO:root: debug : False INFO:root: distributed : False INFO:root: epoch_num : 500 INFO:root: eval_batch_step : [0, 2000] INFO:root: infer_img : doc/imgs_words/ch/word_1.jpg INFO:root: infer_mode : False INFO:root: load_static_weights : True INFO:root: log_smooth_window : 20 INFO:root: max_text_length : 25 INFO:root: pretrained_model : None INFO:root: print_batch_step : 10 INFO:root: save_epoch_step : 3 INFO:root: save_inference_dir : None INFO:root: save_model_dir : ./output/rec_chinese_common_v2.0 INFO:root: save_res_path : ./output/det_db/predicts_db.txt INFO:root: use_gpu : False INFO:root: use_space_char : True INFO:root: use_visualdl : False INFO:root:Loss : INFO:root: alpha : 5 INFO:root: balance_loss : True INFO:root: beta : 10 INFO:root: main_loss_type : DiceLoss INFO:root: name : CTCLoss INFO:root: ohem_ratio : 3 INFO:root:Metric : INFO:root: main_indicator : acc INFO:root: name : RecMetric INFO:root:Optimizer : INFO:root: beta1 : 0.9 INFO:root: beta2 : 0.999 INFO:root: lr : INFO:root: learning_rate : 0.001 INFO:root: name : Cosine INFO:root: name : Adam INFO:root: regularizer : INFO:root: factor : 4e-05 INFO:root: name : L2 INFO:root:PostProcess : INFO:root: box_thresh : 0.5 INFO:root: max_candidates : 1000 INFO:root: name : CTCLabelDecode INFO:root: thresh : 0.3 INFO:root: unclip_ratio : 1.5 INFO:root:Train : INFO:root: dataset : INFO:root: data_dir : ./train_data/ INFO:root: label_file_list : ['./train_data/train_list.txt'] INFO:root: name : SimpleDataSet INFO:root: transforms : INFO:root: DecodeImage : INFO:root: channel_first : False INFO:root: img_mode : BGR INFO:root: RecAug : None INFO:root: CTCLabelEncode : None INFO:root: RecResizeImg : INFO:root: image_shape : [3, 32, 320] INFO:root: KeepKeys : INFO:root: keep_keys : ['image', 'label', 'length'] INFO:root: loader : INFO:root: batch_size_per_card : 1 INFO:root: drop_last : True INFO:root: num_workers : 1 INFO:root: shuffle : True INFO:root:train with paddle 2.0.0-rc1 and device CPUPlace INFO:root:Initialize indexs of datasets:['image/370/OCR_REC/predict/eval_label.txt'] ERROR:tornado.application:Uncaught exception POST /test (::1) HTTPServerRequest(protocol='http', host='localhost:31001', method='POST', uri='/test', version='HTTP/1.1', remote_ip='::1') Traceback (most recent call last): File "D:\Users\cgq\Anaconda3\Lib\site-packages\tornado\web.py", line 1543, in _execute result = yield result File "D:\Users\cgq\Anaconda3\Lib\site-packages\tornado\gen.py", line 1099, in run value = future.result() File "D:\Users\cgq\Anaconda3\Lib\site-packages\tornado\gen.py", line 296, in wrapper result = func(*args, kwargs) File "d:\Users\cgq\Anaconda3\lib\types.py", line 248, in wrapped coro = func(*args, *kwargs) File "d:\Demonstration\semantic-delegate-service\rxhui\service\service_generate.py", line 107, in post self.executor(body_data) File "d:\Demonstration\semantic-delegate-service\rxhui\service\service_generate.py", line 118, in executor test(str(param),self.url,self.MONGO_URI,self.MONGO_DB) File "d:\Demonstration\semantic-delegate-service\rxhui\learn\test.py", line 140, in test test_executor(param, db, url) File "d:\Demonstration\semantic-delegate-service\rxhui\learn\test.py", line 123, in test_executor test_result_analysis=model.test(df,labelInfo) File "d:\Demonstration\semantic-delegate-service\rxhui\base\model_base.py", line 1890, in test pred_result=self.predict(df,args) File "d:\Demonstration\semantic-delegate-service\rxhui\impl\ocr\ocr_rec.py", line 187, in predict metirc=ocr_test(args,self.best_model) File "d:\Demonstration\semantic-delegate-service\rxhui\impl\ocr\ocr_test.py", line 74, in ocr_test return main(config, device, logger, vdl_writer,model) File "d:\Demonstration\semantic-delegate-service\rxhui\impl\ocr\ocr_test.py", line 52, in main model = build_model(config['Architecture']) File "d:\Demonstration\semantic-delegate-service\PaddleOCR\ppocr\modeling\architectures__init__.py", line 23, in build_model module_class = BaseModel(config) File "d:\Demonstration\semantic-delegate-service\PaddleOCR\ppocr\modeling\architectures\base_model.py", line 52, in init self.backbone = build_backbone(config["Backbone"], model_type) File "d:\Demonstration\semantic-delegate-service\PaddleOCR\ppocr\modeling\backbones__init__.py", line 35, in build_backbone module_class = eval(module_name)(config) File "d:\Demonstration\semantic-delegate-service\PaddleOCR\ppocr\modeling\backbones\rec_resnet_vd.py", line 206, in init name="conv1_1") File "d:\Demonstration\semantic-delegate-service\PaddleOCR\ppocr\modeling\backbones\rec_resnet_vd.py", line 51, in init bias_attr=False) File "D:\Users\cgq\Anaconda3\Lib\site-packages\paddle\nn\layer\conv.py", line 623, in init data_format=data_format) File "D:\Users\cgq\Anaconda3\Lib\site-packages\paddle\nn\layer\conv.py", line 136, in init default_initializer=_get_default_param_initializer()) File "D:\Users\cgq\Anaconda3\Lib\site-packages\paddle\fluid\dygraph\layers.py", line 390, in create_parameter default_initializer) File "D:\Users\cgq\Anaconda3\Lib\site-packages\paddle\fluid\layer_helper_base.py", line 366, in create_parameter "constructor of dygraph Layers".format(attr.name)) ValueError: parameter name [conv1_1_weights] have be been used. In dygraph mode, the name of parameter can't be same.Please check the parameter attr value passed to self.create_parameter or constructor of dygraph Layers

littletomatodonkey commented 3 years ago

你好,检测和识别需要分别训练哈,因为其中的mobilenet骨干网络会有重名的问题

cgq0816 commented 3 years ago

你好,检测和识别需要分别训练哈,因为其中的mobilenet骨干网络会有重名的问题

您好,我想问一下我怎么删除其中一个模型,在使用另外一个模型进行预测或者评估呢,让这个两个不能相互干扰

littletomatodonkey commented 3 years ago

可以给模型定义添加一个prefix name的参数,让两个模型的名称不同

paddle-bot-old[bot] commented 3 years ago

Are you satisfied with the resolution of your issue?

YES No

cgq0816 commented 3 years ago

使用该方法可以解决该问题 (重名的问题)
fluid.dygraph.disable_dygraph() fluid.dygraph.enable_dygraph()

KamToHung commented 2 years ago

你好,请问下这个参数在哪配置

cgq0816 commented 2 years ago

你好,请问下这个参数在哪配置 如果用到多个相同模型(ocr)的时候,需要依次测试每个模型,那么在加载模型之前进行配置

darkzoee commented 4 months ago

在paddle2.5版本后,fluid被废弃,该如何实现同样功能呢 from paddle.fluid import fluid fluid.dygraph.disable_dygraph() fluid.dygraph.enable_dygraph()

你好,请问下这个参数在哪配置 如果用到多个相同模型(ocr)的时候,需要依次测试每个模型,那么在加载模型之前进行配置