训练BUG

[cqray1990]

我觉得训练det_train.py, Acc 一直为0.5是个BUG，正常模型训练应该不会出现这种情况，这个很奇怪，能找到是什么原因？是否根数据集有关，用的同样是icdar

[fxwfzsxyq]

我觉得训练det_train.py, Acc 一直为0.5是个BUG，正常模型训练应该不会出现这种情况，这个很奇怪，能找到是什么原因？是否根数据集有关，用的同样是icdar

我没法复现你的问题，因为在我这一直是训练正常的 我这边每次训练可能收敛的epoch数可能不同但是基本都是会收敛的 不会一直维持在0.5 这个我之后可能会在看看 会不会和环境有关系

[cqray1990]

@fxwfzsxyq 梯度大概率没有传递

(22/1200/0/62) | loss_total:2.7230 | loss_l1:0.1161 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4977 | lr:0.00098348 (22/1200/20/62) | loss_total:2.7140 | loss_l1:0.1152 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4973 | lr:0.00098348 (22/1200/40/62) | loss_total:2.7159 | loss_l1:0.1154 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4971 | lr:0.00098348 (22/1200/60/62) | loss_total:2.7145 | loss_l1:0.1152 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4971 | lr:0.00098348 (23/1200/0/62) | loss_total:2.6985 | loss_l1:0.1136 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4967 | lr:0.00098273 (23/1200/20/62) | loss_total:2.7123 | loss_l1:0.1150 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4972 | lr:0.00098273 (23/1200/40/62) | loss_total:2.7131 | loss_l1:0.1151 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4971 | lr:0.00098273 (23/1200/60/62) | loss_total:2.7132 | loss_l1:0.1151 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4972 | lr:0.00098273 (24/1200/0/62) | loss_total:2.7270 | loss_l1:0.1165 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4977 | lr:0.00098198 (24/1200/20/62) | loss_total:2.7174 | loss_l1:0.1155 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4974 | lr:0.00098198 (24/1200/40/62) | loss_total:2.7160 | loss_l1:0.1154 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4974 | lr:0.00098198 (24/1200/60/62) | loss_total:2.7145 | loss_l1:0.1152 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4974 | lr:0.00098198 (25/1200/0/62) | loss_total:2.7199 | loss_l1:0.1158 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4959 | lr:0.00098123 (25/1200/20/62) | loss_total:2.7166 | loss_l1:0.1154 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4969 | lr:0.00098123 (25/1200/40/62) | loss_total:2.7170 | loss_l1:0.1155 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4971 | lr:0.00098123 (25/1200/60/62) | loss_total:2.7153 | loss_l1:0.1153 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4970 | lr:0.00098123 (26/1200/0/62) | loss_total:2.7044 | loss_l1:0.1142 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4974 | lr:0.00098048 (26/1200/20/62) | loss_total:2.7157 | loss_l1:0.1153 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4971 | lr:0.00098048 (26/1200/40/62) | loss_total:2.7159 | loss_l1:0.1154 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4970 | lr:0.00098048 (26/1200/60/62) | loss_total:2.7151 | loss_l1:0.1153 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4971 | lr:0.00098048 (27/1200/0/62) | loss_total:2.7194 | loss_l1:0.1157 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4978 | lr:0.00097973 (27/1200/20/62) | loss_total:2.7193 | loss_l1:0.1157 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4971 | lr:0.00097973 (27/1200/40/62) | loss_total:2.7151 | loss_l1:0.1153 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4972 | lr:0.00097973 (27/1200/60/62) | loss_total:2.7157 | loss_l1:0.1153 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4972 | lr:0.00097973 (28/1200/0/62) | loss_total:2.7286 | loss_l1:0.1166 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4953 | lr:0.00097898 (28/1200/20/62) | loss_total:2.7141 | loss_l1:0.1152 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4972 | lr:0.00097898 (28/1200/40/62) | loss_total:2.7144 | loss_l1:0.1152 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4972 | lr:0.00097898 (28/1200/60/62) | loss_total:2.7146 | loss_l1:0.1152 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4971 | lr:0.00097898 (29/1200/0/62) | loss_total:2.7100 | loss_l1:0.1148 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4975 | lr:0.00097822 (29/1200/20/62) | loss_total:2.7118 | loss_l1:0.1150 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4972 | lr:0.00097822 (29/1200/40/62) | loss_total:2.7134 | loss_l1:0.1151 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4973 | lr:0.00097822 (29/1200/60/62) | loss_total:2.7137 | loss_l1:0.1151 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4972 | lr:0.00097822 (30/1200/0/62) | loss_total:2.7176 | loss_l1:0.1155 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4974 | lr:0.00097747 (30/1200/20/62) | loss_total:2.7153 | loss_l1:0.1153 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4971 | lr:0.00097747 (30/1200/40/62) | loss_total:2.7140 | loss_l1:0.1152 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4972 | lr:0.00097747 (30/1200/60/62) | loss_total:2.7144 | loss_l1:0.1152 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4972 | lr:0.00097747 (31/1200/0/62) | loss_total:2.7246 | loss_l1:0.1162 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4962 | lr:0.00097672 (31/1200/20/62) | loss_total:2.7177 | loss_l1:0.1155 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4974 | lr:0.00097672 (31/1200/40/62) | loss_total:2.7146 | loss_l1:0.1152 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4974 | lr:0.00097672 (31/1200/60/62) | loss_total:2.7141 | loss_l1:0.1152 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4972 | lr:0.00097672 (32/1200/0/62) | loss_total:2.7100 | loss_l1:0.1148 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4970 | lr:0.00097597 (32/1200/20/62) | loss_total:2.7127 | loss_l1:0.1150 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4975 | lr:0.00097597 (32/1200/40/62) | loss_total:2.7115 | loss_l1:0.1149 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4974 | lr:0.00097597 (32/1200/60/62) | loss_total:2.7127 | loss_l1:0.1150 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4973 | lr:0.00097597 (33/1200/0/62) | loss_total:2.7220 | loss_l1:0.1160 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4961 | lr:0.00097522 (33/1200/20/62) | loss_total:2.7136 | loss_l1:0.1151 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4971 | lr:0.00097522 (33/1200/40/62) | loss_total:2.7141 | loss_l1:0.1152 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4972 | lr:0.00097522 (33/1200/60/62) | loss_total:2.7141 | loss_l1:0.1152 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4972 | lr:0.00097522 (34/1200/0/62) | loss_total:2.7160 | loss_l1:0.1154 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4968 | lr:0.00097446 (34/1200/20/62) | loss_total:2.7137 | loss_l1:0.1151 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4971 | lr:0.00097446 (34/1200/40/62) | loss_total:2.7140 | loss_l1:0.1152 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4971 | lr:0.00097446 (34/1200/60/62) | loss_total:2.7144 | loss_l1:0.1152 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4972 | lr:0.00097446 (35/1200/0/62) | loss_total:2.7132 | loss_l1:0.1151 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4978 | lr:0.00097371 (35/1200/20/62) | loss_total:2.7125 | loss_l1:0.1150 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4970 | lr:0.00097371 (35/1200/40/62) | loss_total:2.7131 | loss_l1:0.1151 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4971 | lr:0.00097371 (35/1200/60/62) | loss_total:2.7146 | loss_l1:0.1152 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4971 | lr:0.00097371 (36/1200/0/62) | loss_total:2.7217 | loss_l1:0.1159 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4944 | lr:0.00097296 (36/1200/20/62) | loss_total:2.7153 | loss_l1:0.1153 | loss_bce:0.5626 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4973 | lr:0.00097296 (36/1200/40/62) | loss_total:2.7143 | loss_l1:0.1152 | loss_bce:0.5625 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4971 | lr:0.00097296 (36/1200/60/62) | loss_total:2.7137 | loss_l1:0.1151 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4971 | lr:0.00097296 (37/1200/0/62) | loss_total:2.7194 | loss_l1:0.1157 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4976 | lr:0.00097221 (37/1200/20/62) | loss_total:2.7160 | loss_l1:0.1154 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4974 | lr:0.00097221 (37/1200/40/62) | loss_total:2.7161 | loss_l1:0.1154 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4970 | lr:0.00097221 (37/1200/60/62) | loss_total:2.7159 | loss_l1:0.1154 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4971 | lr:0.00097221 (38/1200/0/62) | loss_total:2.7030 | loss_l1:0.1141 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4976 | lr:0.00097145 (38/1200/20/62) | loss_total:2.7134 | loss_l1:0.1151 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4972 | lr:0.00097145 (38/1200/40/62) | loss_total:2.7144 | loss_l1:0.1152 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4973 | lr:0.00097145 (38/1200/60/62) | loss_total:2.7150 | loss_l1:0.1153 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4972 | lr:0.00097145 (39/1200/0/62) | loss_total:2.7080 | loss_l1:0.1146 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4980 | lr:0.00097070 (39/1200/20/62) | loss_total:2.7150 | loss_l1:0.1153 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4968 | lr:0.00097070 (39/1200/40/62) | loss_total:2.7139 | loss_l1:0.1152 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4970 | lr:0.00097070 (39/1200/60/62) | loss_total:2.7136 | loss_l1:0.1151 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4970 | lr:0.00097070 (40/1200/0/62) | loss_total:2.7081 | loss_l1:0.1146 | loss_bce:0.5623 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4965 | lr:0.00096995 (40/1200/20/62) | loss_total:2.7168 | loss_l1:0.1155 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4973 | lr:0.00096995 (40/1200/40/62) | loss_total:2.7155 | loss_l1:0.1153 | loss_bce:0.5624 | loss_thresh:1.0000 | ACC:0.5000 | IOU:0.4971 | lr:0.00096995