训练问题

[nyjshinibaba]

2024-02-19 16:34:22,237 - mmdet - INFO - Iter [4950/120000] lr: 4.000e-04, eta: 6:13:37, time: 0.202, data_time: 0.020, memory: 4061, loss_mask: 0.1048, loss_texture: 0.1240, iou: 0.7859, loss: 0.2288 2024-02-19 16:34:31,688 - mmdet - INFO - Saving checkpoint at 5000 iterations 2024-02-19 16:34:35,605 - mmdet - INFO - Exp name: segrefiner_lr.py 2024-02-19 16:34:35,606 - mmdet - INFO - Iter [5000/120000] lr: 4.000e-04, eta: 6:14:51, time: 0.267, data_time: 0.007, memory: 4061, loss_mask: 0.1186, loss_texture: 0.1323, iou: 0.7998, loss: 0.2509 2024-02-19 16:34:46,323 - mmdet - INFO - Iter [5050/120000] lr: 4.000e-04, eta: 6:15:03, time: 0.214, data_time: 0.032, memory: 4061, loss_mask: 0.1113, loss_texture: 0.1355, iou: 0.8468, loss: 0.2468 2024-02-19 16:34:55,565 - mmdet - INFO - Iter [5100/120000] lr: 4.000e-04, eta: 6:14:40, time: 0.185, data_time: 0.006, memory: 4061, loss_mask: 0.1199, loss_texture: 0.1308, iou: 0.8084, loss: 0.2507 2024-02-19 16:35:06,691 - mmdet - INFO - Iter [5150/120000] lr: 4.000e-04, eta: 6:15:01, time: 0.223, data_time: 0.043, memory: 4061, loss_mask: 0.1118, loss_texture: 0.1262, iou: 0.7809, loss: 0.2380 2024-02-19 16:35:16,616 - mmdet - INFO - Iter [5200/120000] lr: 4.000e-04, eta: 6:14:54, time: 0.198, data_time: 0.015, memory: 4061, loss_mask: 0.1292, loss_texture: 0.1344, iou: 0.7996, loss: 0.2636 2024-02-19 16:35:26,801 - mmdet - INFO - Iter [5250/120000] lr: 4.000e-04, eta: 6:14:52, time: 0.204, data_time: 0.016, memory: 4061, loss_mask: 0.1406, loss_texture: 0.1342, iou: 0.8005, loss: 0.2748 2024-02-19 16:35:37,887 - mmdet - INFO - Iter [5300/120000] lr: 4.000e-04, eta: 6:15:10, time: 0.222, data_time: 0.037, memory: 4061, loss_mask: 0.1334, loss_texture: 0.1322, iou: 0.8054, loss: 0.2656 2024-02-19 16:35:46,770 - mmdet - INFO - Iter [5350/120000] lr: 4.000e-04, eta: 6:14:41, time: 0.178, data_time: 0.017, memory: 4061, loss_mask: 0.1028, loss_texture: 0.1307, iou: 0.8518, loss: 0.2335 2024-02-19 16:35:57,319 - mmdet - INFO - Iter [5400/120000] lr: 4.000e-04, eta: 6:14:47, time: 0.211, data_time: 0.016, memory: 4061, loss_mask: 0.2681, loss_texture: 0.1454, iou: 0.6773, loss: 0.4135 2024-02-19 16:36:07,540 - mmdet - INFO - Iter [5450/120000] lr: 4.000e-04, eta: 6:14:45, time: 0.204, data_time: 0.010, memory: 4061, loss_mask: 0.4376, loss_texture: 0.1406, iou: 0.3416, loss: 0.5782 2024-02-19 16:36:17,684 - mmdet - INFO - Iter [5500/120000] lr: 4.000e-04, eta: 6:14:43, time: 0.203, data_time: 0.023, memory: 4061, loss_mask: 0.5540, loss_texture: 0.1434, iou: 0.0000, loss: 0.6974 2024-02-19 16:36:27,192 - mmdet - INFO - Iter [5550/120000] lr: 4.000e-04, eta: 6:14:26, time: 0.190, data_time: 0.009, memory: 4061, loss_mask: 0.6086, loss_texture: 0.1315, iou: 0.0000, loss: 0.7401 2024-02-19 16:36:38,381 - mmdet - INFO - Iter [5600/120000] lr: 4.000e-04, eta: 6:14:45, time: 0.224, data_time: 0.038, memory: 4061, loss_mask: 0.5943, loss_texture: 0.1409, iou: 0.0000, loss: 0.7352 2024-02-19 16:36:48,669 - mmdet - INFO - Iter [5650/120000] lr: 4.000e-04, eta: 6:14:44, time: 0.206, data_time: 0.028, memory: 4061, loss_mask: 0.5827, loss_texture: 0.1452, iou: 0.0000, loss: 0.7279 2024-02-19 16:36:57,981 - mmdet - INFO - Iter [5700/120000] lr: 4.000e-04, eta: 6:14:24, time: 0.186, data_time: 0.008, memory: 4061, loss_mask: 0.5287, loss_texture: 0.1281, iou: 0.0000, loss: 0.6568 2024-02-19 16:37:07,785 - mmdet - INFO - Iter [5750/120000] lr: 4.000e-04, eta: 6:14:13, time: 0.196, data_time: 0.017, memory: 4061, loss_mask: 0.5875, loss_texture: 0.1300, iou: 0.0000, loss: 0.7175 2024-02-19 16:37:17,714 - mmdet - INFO - Iter [5800/120000] lr: 4.000e-04, eta: 6:14:06, time: 0.199, data_time: 0.017, memory: 4061, loss_mask: 0.5624, loss_texture: 0.1306, iou: 0.0000, loss: 0.6930 2024-02-19 16:37:27,840 - mmdet - INFO - Iter [5850/120000] lr: 4.000e-04, eta: 6:14:02, time: 0.202, data_time: 0.024, memory: 4061, loss_mask: 0.5655, loss_texture: 0.1239, iou: 0.0000, loss: 0.6894 2024-02-19 16:37:40,109 - mmdet - INFO - Iter [5900/120000] lr: 4.000e-04, eta: 6:14:39, time: 0.245, data_time: 0.066, memory: 4061, loss_mask: 0.6541, loss_texture: 0.1476, iou: 0.0000, loss: 0.8016 2024-02-19 16:37:50,496 - mmdet - INFO - Iter [5950/120000] lr: 4.000e-04, eta: 6:14:39, time: 0.208, data_time: 0.030, memory: 4061, loss_mask: 0.5569, loss_texture: 0.1273, iou: 0.0000, loss: 0.6842 从5500iter开始iou变成0是为什么呢

[nyjshinibaba]

并且loss此时出现了增大的趋势 @MengyuWang826

[yanrihong]

我遇到了一样的问题，IOU会变成0 ，请问你解决了吗

[nyjshinibaba]

我遇到了一样的问题，IOU会变成0 ，请问你解决了吗

解决了，适当调整学习率即可@ @yanrihong