Open wangpuxiong opened 2 years ago
ppyoloe/ppyoloe_plus_crn_x_80e_coco.yml 确定可以完整训完1个epoch吗?环境是几卡显存多大? 可以尝试改小bs或删去这里的几个大尺度,https://github.com/PaddlePaddle/PaddleDetection/blob/develop/configs/visdrone/ppyoloe_crn_l_alpha_largesize_80e_visdrone.yml#L31
ppyoloe/ppyoloe_plus_crn_x_80e_coco.yml 已经完成60个epoch的训练了,环境1卡显存32G
i have the same problem
问题确认 Search before asking
Bug组件 Bug Component
Training
Bug描述 Describe the Bug
### 大佬们帮忙看看,采用visdrone/ppyoloe_crn_l_alpha_largesize_80e_visdrone.yml模型报错,但是采用ppyoloe/ppyoloe_plus_crn_x_80e_coco.yml 可以正常 运行。我希望检测小目标因此还是希望把visdrone/ppyoloe_crn_l_alpha_largesize_80e_visdrone.yml 模型用起来。 代码没有修改直接用的是百度的https://aistudio.baidu.com/aistudio/ 下的开发平台 运行!python tools/train.py -c configs/visdrone/ppyoloe_crn_l_alpha_largesize_80e_visdrone.yml --use_vdl=True --vdl_log_dir=./visdrone/ --eval 命令时候报错。 报错如下: loading annotations into memory... Done (t=0.20s) creating index... index created! W0911 11:41:50.710125 4053 gpu_resources.cc:61] Please NOTE: device: 0, GPU Compute Capability: 7.0, Driver API Version: 11.2, Runtime API Version: 11.2 W0911 11:41:50.714862 4053 gpu_resources.cc:91] device: 0, cuDNN Version: 8.2. [09/11 11:41:53] ppdet.utils.checkpoint INFO: The shape [80] in pretrained weight yolo_head.pred_cls.0.bias is unmatched with the shape [3] in model yolo_head.pred_cls.0.bias. And the weight yolo_head.pred_cls.0.bias will not be loaded [09/11 11:41:53] ppdet.utils.checkpoint INFO: The shape [80, 768, 3, 3] in pretrained weight yolo_head.pred_cls.0.weight is unmatched with the shape [3, 768, 3, 3] in model yolo_head.pred_cls.0.weight. And the weight yolo_head.pred_cls.0.weight will not be loaded [09/11 11:41:53] ppdet.utils.checkpoint INFO: The shape [80] in pretrained weight yolo_head.pred_cls.1.bias is unmatched with the shape [3] in model yolo_head.pred_cls.1.bias. And the weight yolo_head.pred_cls.1.bias will not be loaded [09/11 11:41:53] ppdet.utils.checkpoint INFO: The shape [80, 384, 3, 3] in pretrained weight yolo_head.pred_cls.1.weight is unmatched with the shape [3, 384, 3, 3] in model yolo_head.pred_cls.1.weight. And the weight yolo_head.pred_cls.1.weight will not be loaded [09/11 11:41:53] ppdet.utils.checkpoint INFO: The shape [80] in pretrained weight yolo_head.pred_cls.2.bias is unmatched with the shape [3] in model yolo_head.pred_cls.2.bias. And the weight yolo_head.pred_cls.2.bias will not be loaded [09/11 11:41:53] ppdet.utils.checkpoint INFO: The shape [80, 192, 3, 3] in pretrained weight yolo_head.pred_cls.2.weight is unmatched with the shape [3, 192, 3, 3] in model yolo_head.pred_cls.2.weight. And the weight yolo_head.pred_cls.2.weight will not be loaded [09/11 11:41:53] ppdet.utils.checkpoint INFO: Finish loading model weights: models/ppyoloe_plus_crn_l_80e_coco.pdparams [09/11 11:41:55] ppdet.engine INFO: Epoch: [0] [ 0/5564] learning_rate: 0.000000 loss: 5.066185 loss_cls: 1.017579 loss_iou: 0.979015 loss_dfl: 3.202135 loss_l1: 6.590621 eta: 4 days, 4:15:48 batch_cost: 2.1624 data_cost: 0.0044 ips: 0.9249 images/s Error: /paddle/paddle/phi/kernels/gpu/one_hot_kernel.cu:38 Assertion p_in_data[idx] >= 0 && p_in_data[idx] < depth failed. Illegal index value, Input(input) value should be greater than or equal to 0, and less than depth [24276], but received [140098075152656]. Error: /paddle/paddle/phi/kernels/gpu/one_hot_kernel.cu:38 Assertion p_in_data[idx] >= 0 && p_in_data[idx] < depth failed. Illegal index value, Input(input) value should be greater than or equal to 0, and less than depth [24276], but received [140098075152656].
复现环境 Environment
项目“工业品表面缺陷检测”共享链接(有效期三天):https://aistudio.baidu.com/studio/project/partial/verify/4422217/8805b66ed7964a969d3721ed18a0bc2e
操作系统: Linux. paddlepaddle:2.2.2 paddledetectionc:2.5 python:3.7 cuda:10.2 cudnn:7.6 gcc:8.2
Bug描述确认 Bug description confirmation
是否愿意提交PR? Are you willing to submit a PR?