search

LiheYoung / UniMatch

[CVPR 2023] Revisiting Weak-to-Strong Consistency in Semi-Supervised Semantic Segmentation
https://arxiv.org/abs/2208.09910
MIT License
483 stars 60 forks source link

About the reproduction of results on Pascal dataset #13

Closed nysp78 closed 2 years ago

nysp78 commented 2 years ago

Hello, Would it be easy to reproduce the results for pascal dataset on a single gpu? What learning rate is required for training on a single gpu to reproduce the results for FixMatch / UniMatch?

Thanks

LiheYoung commented 2 years ago

Our total batch size is 8 (2 imgs / card x 4 cards) for both labeled and unlabeled images. The corresponding learning rate is 0.001. So if you only use a single card and the batch size is 8 / K, the learning rate should also be 0.001 / K. For example, if your batch size is 4 only with a single card, the learning rate then will be 0.0005.

nysp78 commented 2 years ago

I ran the unimatch with batch size 4 and initial lr 0.0005 on a single GPU for the 1/16 partition of pascal and I only took 75.50 mIOU. Τhere is any other trick I can do or perhaps the results can not be reproduced on a single GPU?

LiheYoung commented 2 years ago

Please detail which backbone and cropping size are adopted. And what the reported result is.

nysp78 commented 2 years ago

[2022-11-25 16:02:33,141][ INFO] {'backbone': 'resnet101', 'batch_size': 4, 'conf_thresh': 0.95, 'criterion': {'kwargs': {'ignore_index': 255}, 'name': 'CELoss'}, 'crop_size': 321, 'data_root': /path/to/pascal, 'dataset': 'pascal', 'dilations': [6, 12, 18], 'epochs': 80, 'lr': 0.0005, 'lr_multi': 10.0, 'multi_grid': False, 'nclass': 21, 'replace_stride_with_dilation': [False, False, True]}

[2022-11-25 16:02:34,849][ INFO] Total params: 59.5M

[2022-11-25 16:02:37,349][ INFO] ===========> Epoch: 0, LR: 0.0005, Previous best: 0.00 [2022-11-25 16:02:45,618][ INFO] Iters: 0, Total loss: 1.575, Loss x: 3.150, Loss s: 0.000, Loss w_fp: 0.000, Mask: 0.000 [2022-11-25 16:05:12,455][ INFO] Iters: 310, Total loss: 0.556, Loss x: 1.075, Loss s: 0.052, Loss w_fp: 0.021, Mask: 0.222 [2022-11-25 16:07:39,331][ INFO] Iters: 620, Total loss: 0.450, Loss x: 0.855, Loss s: 0.071, Loss w_fp: 0.019, Mask: 0.292 [2022-11-25 16:10:06,300][ INFO] Iters: 930, Total loss: 0.395, Loss x: 0.740, Loss s: 0.082, Loss w_fp: 0.018, Mask: 0.327 [2022-11-25 16:12:33,237][ INFO] Iters: 1240, Total loss: 0.361, Loss x: 0.668, Loss s: 0.090, Loss w_fp: 0.019, Mask: 0.366 [2022-11-25 16:14:58,762][ INFO] Iters: 1550, Total loss: 0.338, Loss x: 0.618, Loss s: 0.096, Loss w_fp: 0.019, Mask: 0.385 [2022-11-25 16:17:24,090][ INFO] Iters: 1860, Total loss: 0.318, Loss x: 0.576, Loss s: 0.102, Loss w_fp: 0.019, Mask: 0.404 [2022-11-25 16:19:49,541][ INFO] Iters: 2170, Total loss: 0.303, Loss x: 0.543, Loss s: 0.107, Loss w_fp: 0.020, Mask: 0.422 [2022-11-25 16:25:08,569][ INFO] Evaluation original >>>> meanIOU: 56.33

[2022-11-25 16:25:09,356][ INFO] ===========> Epoch: 1, LR: 0.0005, Previous best: 56.33 [2022-11-25 16:25:10,444][ INFO] Iters: 0, Total loss: 0.115, Loss x: 0.136, Loss s: 0.165, Loss w_fp: 0.022, Mask: 0.721 [2022-11-25 16:27:37,508][ INFO] Iters: 310, Total loss: 0.182, Loss x: 0.276, Loss s: 0.151, Loss w_fp: 0.024, Mask: 0.551 [2022-11-25 16:30:04,694][ INFO] Iters: 620, Total loss: 0.180, Loss x: 0.274, Loss s: 0.148, Loss w_fp: 0.022, Mask: 0.563 [2022-11-25 16:32:31,996][ INFO] Iters: 930, Total loss: 0.180, Loss x: 0.274, Loss s: 0.150, Loss w_fp: 0.022, Mask: 0.573 [2022-11-25 16:34:59,147][ INFO] Iters: 1240, Total loss: 0.176, Loss x: 0.265, Loss s: 0.151, Loss w_fp: 0.022, Mask: 0.584 [2022-11-25 16:37:26,400][ INFO] Iters: 1550, Total loss: 0.170, Loss x: 0.255, Loss s: 0.149, Loss w_fp: 0.022, Mask: 0.590 [2022-11-25 16:39:53,906][ INFO] Iters: 1860, Total loss: 0.169, Loss x: 0.252, Loss s: 0.150, Loss w_fp: 0.021, Mask: 0.595 [2022-11-25 16:42:21,170][ INFO] Iters: 2170, Total loss: 0.167, Loss x: 0.249, Loss s: 0.150, Loss w_fp: 0.021, Mask: 0.598 [2022-11-25 16:45:43,356][ INFO] Evaluation original >>>> meanIOU: 65.14

[2022-11-25 16:45:43,982][ INFO] ===========> Epoch: 2, LR: 0.0005, Previous best: 65.14 [2022-11-25 16:45:45,041][ INFO] Iters: 0, Total loss: 0.065, Loss x: 0.053, Loss s: 0.147, Loss w_fp: 0.008, Mask: 0.330 [2022-11-25 16:48:12,148][ INFO] Iters: 310, Total loss: 0.148, Loss x: 0.211, Loss s: 0.148, Loss w_fp: 0.020, Mask: 0.639 [2022-11-25 16:50:39,326][ INFO] Iters: 620, Total loss: 0.146, Loss x: 0.206, Loss s: 0.150, Loss w_fp: 0.021, Mask: 0.637 [2022-11-25 16:53:06,565][ INFO] Iters: 930, Total loss: 0.146, Loss x: 0.207, Loss s: 0.148, Loss w_fp: 0.020, Mask: 0.636 [2022-11-25 16:55:33,742][ INFO] Iters: 1240, Total loss: 0.142, Loss x: 0.201, Loss s: 0.148, Loss w_fp: 0.020, Mask: 0.640 [2022-11-25 16:58:00,975][ INFO] Iters: 1550, Total loss: 0.141, Loss x: 0.198, Loss s: 0.148, Loss w_fp: 0.020, Mask: 0.642 [2022-11-25 17:00:28,034][ INFO] Iters: 1860, Total loss: 0.139, Loss x: 0.194, Loss s: 0.146, Loss w_fp: 0.020, Mask: 0.646 [2022-11-25 17:02:55,229][ INFO] Iters: 2170, Total loss: 0.138, Loss x: 0.193, Loss s: 0.145, Loss w_fp: 0.019, Mask: 0.648 [2022-11-25 17:06:12,677][ INFO] Evaluation original >>>> meanIOU: 66.59

[2022-11-25 17:06:13,541][ INFO] ===========> Epoch: 3, LR: 0.0005, Previous best: 66.59 [2022-11-25 17:06:14,564][ INFO] Iters: 0, Total loss: 0.091, Loss x: 0.057, Loss s: 0.241, Loss w_fp: 0.009, Mask: 0.687 [2022-11-25 17:08:41,982][ INFO] Iters: 310, Total loss: 0.123, Loss x: 0.159, Loss s: 0.155, Loss w_fp: 0.019, Mask: 0.672 [2022-11-25 17:11:09,323][ INFO] Iters: 620, Total loss: 0.125, Loss x: 0.164, Loss s: 0.154, Loss w_fp: 0.018, Mask: 0.672 [2022-11-25 17:13:36,525][ INFO] Iters: 930, Total loss: 0.126, Loss x: 0.167, Loss s: 0.152, Loss w_fp: 0.018, Mask: 0.672 [2022-11-25 17:16:03,706][ INFO] Iters: 1240, Total loss: 0.126, Loss x: 0.169, Loss s: 0.149, Loss w_fp: 0.018, Mask: 0.669 [2022-11-25 17:18:30,888][ INFO] Iters: 1550, Total loss: 0.126, Loss x: 0.170, Loss s: 0.148, Loss w_fp: 0.018, Mask: 0.670 [2022-11-25 17:20:58,025][ INFO] Iters: 1860, Total loss: 0.125, Loss x: 0.168, Loss s: 0.147, Loss w_fp: 0.018, Mask: 0.670 [2022-11-25 17:23:25,015][ INFO] Iters: 2170, Total loss: 0.124, Loss x: 0.167, Loss s: 0.145, Loss w_fp: 0.018, Mask: 0.671 [2022-11-25 17:26:46,643][ INFO] Evaluation original >>>> meanIOU: 68.03

[2022-11-25 17:26:47,472][ INFO] ===========> Epoch: 4, LR: 0.0005, Previous best: 68.03 [2022-11-25 17:26:48,533][ INFO] Iters: 0, Total loss: 0.092, Loss x: 0.106, Loss s: 0.116, Loss w_fp: 0.039, Mask: 0.710 [2022-11-25 17:29:15,607][ INFO] Iters: 310, Total loss: 0.107, Loss x: 0.137, Loss s: 0.138, Loss w_fp: 0.015, Mask: 0.706 [2022-11-25 17:31:42,655][ INFO] Iters: 620, Total loss: 0.109, Loss x: 0.139, Loss s: 0.141, Loss w_fp: 0.016, Mask: 0.702 [2022-11-25 17:34:09,820][ INFO] Iters: 930, Total loss: 0.108, Loss x: 0.138, Loss s: 0.141, Loss w_fp: 0.016, Mask: 0.697 [2022-11-25 17:36:37,012][ INFO] Iters: 1240, Total loss: 0.107, Loss x: 0.137, Loss s: 0.138, Loss w_fp: 0.015, Mask: 0.699 [2022-11-25 17:39:04,242][ INFO] Iters: 1550, Total loss: 0.107, Loss x: 0.136, Loss s: 0.139, Loss w_fp: 0.015, Mask: 0.701 [2022-11-25 17:41:31,489][ INFO] Iters: 1860, Total loss: 0.107, Loss x: 0.137, Loss s: 0.138, Loss w_fp: 0.015, Mask: 0.703 [2022-11-25 17:43:58,650][ INFO] Iters: 2170, Total loss: 0.107, Loss x: 0.137, Loss s: 0.138, Loss w_fp: 0.015, Mask: 0.704 [2022-11-25 17:47:21,482][ INFO] Evaluation original >>>> meanIOU: 69.49

[2022-11-25 17:47:22,141][ INFO] ===========> Epoch: 5, LR: 0.0005, Previous best: 69.49 [2022-11-25 17:47:23,167][ INFO] Iters: 0, Total loss: 0.038, Loss x: 0.039, Loss s: 0.052, Loss w_fp: 0.022, Mask: 0.812 [2022-11-25 17:49:49,580][ INFO] Iters: 310, Total loss: 0.112, Loss x: 0.148, Loss s: 0.140, Loss w_fp: 0.015, Mask: 0.687 [2022-11-25 17:52:15,380][ INFO] Iters: 620, Total loss: 0.106, Loss x: 0.141, Loss s: 0.126, Loss w_fp: 0.014, Mask: 0.697 [2022-11-25 17:54:42,373][ INFO] Iters: 930, Total loss: 0.103, Loss x: 0.137, Loss s: 0.126, Loss w_fp: 0.014, Mask: 0.706 [2022-11-25 17:57:12,858][ INFO] Iters: 1240, Total loss: 0.102, Loss x: 0.134, Loss s: 0.124, Loss w_fp: 0.014, Mask: 0.708 [2022-11-25 17:59:44,193][ INFO] Iters: 1550, Total loss: 0.102, Loss x: 0.134, Loss s: 0.126, Loss w_fp: 0.014, Mask: 0.709 [2022-11-25 18:02:15,723][ INFO] Iters: 1860, Total loss: 0.102, Loss x: 0.134, Loss s: 0.128, Loss w_fp: 0.014, Mask: 0.708 [2022-11-25 18:04:47,196][ INFO] Iters: 2170, Total loss: 0.102, Loss x: 0.132, Loss s: 0.131, Loss w_fp: 0.014, Mask: 0.709 [2022-11-25 18:08:23,400][ INFO] Evaluation original >>>> meanIOU: 69.62

[2022-11-25 18:08:25,530][ INFO] ===========> Epoch: 6, LR: 0.0005, Previous best: 69.62 [2022-11-25 18:08:26,844][ INFO] Iters: 0, Total loss: 0.070, Loss x: 0.128, Loss s: 0.021, Loss w_fp: 0.001, Mask: 0.816 [2022-11-25 18:10:58,762][ INFO] Iters: 310, Total loss: 0.097, Loss x: 0.120, Loss s: 0.133, Loss w_fp: 0.013, Mask: 0.727 [2022-11-25 18:13:30,480][ INFO] Iters: 620, Total loss: 0.097, Loss x: 0.121, Loss s: 0.133, Loss w_fp: 0.013, Mask: 0.728 [2022-11-25 18:16:01,912][ INFO] Iters: 930, Total loss: 0.095, Loss x: 0.118, Loss s: 0.130, Loss w_fp: 0.014, Mask: 0.730 [2022-11-25 18:18:33,570][ INFO] Iters: 1240, Total loss: 0.094, Loss x: 0.117, Loss s: 0.131, Loss w_fp: 0.014, Mask: 0.730 [2022-11-25 18:21:05,225][ INFO] Iters: 1550, Total loss: 0.095, Loss x: 0.116, Loss s: 0.132, Loss w_fp: 0.014, Mask: 0.731 [2022-11-25 18:23:36,861][ INFO] Iters: 1860, Total loss: 0.095, Loss x: 0.116, Loss s: 0.134, Loss w_fp: 0.014, Mask: 0.733 [2022-11-25 18:26:08,254][ INFO] Iters: 2170, Total loss: 0.094, Loss x: 0.114, Loss s: 0.133, Loss w_fp: 0.014, Mask: 0.734 [2022-11-25 18:29:43,723][ INFO] Evaluation original >>>> meanIOU: 70.17

[2022-11-25 18:29:44,586][ INFO] ===========> Epoch: 7, LR: 0.0005, Previous best: 70.17 [2022-11-25 18:29:45,906][ INFO] Iters: 0, Total loss: 0.073, Loss x: 0.131, Loss s: 0.027, Loss w_fp: 0.005, Mask: 0.635 [2022-11-25 18:32:17,722][ INFO] Iters: 310, Total loss: 0.091, Loss x: 0.114, Loss s: 0.122, Loss w_fp: 0.013, Mask: 0.734 [2022-11-25 18:34:50,059][ INFO] Iters: 620, Total loss: 0.094, Loss x: 0.118, Loss s: 0.128, Loss w_fp: 0.013, Mask: 0.731 [2022-11-25 18:37:21,780][ INFO] Iters: 930, Total loss: 0.093, Loss x: 0.114, Loss s: 0.130, Loss w_fp: 0.013, Mask: 0.732 [2022-11-25 18:39:53,319][ INFO] Iters: 1240, Total loss: 0.092, Loss x: 0.114, Loss s: 0.128, Loss w_fp: 0.013, Mask: 0.734 [2022-11-25 18:42:25,440][ INFO] Iters: 1550, Total loss: 0.093, Loss x: 0.113, Loss s: 0.131, Loss w_fp: 0.013, Mask: 0.733 [2022-11-25 18:44:56,969][ INFO] Iters: 1860, Total loss: 0.092, Loss x: 0.112, Loss s: 0.130, Loss w_fp: 0.013, Mask: 0.733 [2022-11-25 18:47:28,275][ INFO] Iters: 2170, Total loss: 0.091, Loss x: 0.111, Loss s: 0.130, Loss w_fp: 0.013, Mask: 0.735 [2022-11-25 18:51:04,887][ INFO] Evaluation original >>>> meanIOU: 69.25

[2022-11-25 18:51:04,888][ INFO] ===========> Epoch: 8, LR: 0.0005, Previous best: 70.17 [2022-11-25 18:51:06,098][ INFO] Iters: 0, Total loss: 0.045, Loss x: 0.071, Loss s: 0.033, Loss w_fp: 0.006, Mask: 0.870 [2022-11-25 18:53:37,571][ INFO] Iters: 310, Total loss: 0.083, Loss x: 0.099, Loss s: 0.119, Loss w_fp: 0.014, Mask: 0.752 [2022-11-25 18:56:09,372][ INFO] Iters: 620, Total loss: 0.085, Loss x: 0.100, Loss s: 0.124, Loss w_fp: 0.014, Mask: 0.749 [2022-11-25 18:58:40,992][ INFO] Iters: 930, Total loss: 0.085, Loss x: 0.101, Loss s: 0.123, Loss w_fp: 0.014, Mask: 0.751 [2022-11-25 19:01:12,512][ INFO] Iters: 1240, Total loss: 0.085, Loss x: 0.102, Loss s: 0.123, Loss w_fp: 0.013, Mask: 0.749 [2022-11-25 19:03:43,991][ INFO] Iters: 1550, Total loss: 0.084, Loss x: 0.101, Loss s: 0.122, Loss w_fp: 0.013, Mask: 0.750 [2022-11-25 19:06:15,637][ INFO] Iters: 1860, Total loss: 0.083, Loss x: 0.100, Loss s: 0.120, Loss w_fp: 0.013, Mask: 0.751 [2022-11-25 19:08:46,753][ INFO] Iters: 2170, Total loss: 0.083, Loss x: 0.099, Loss s: 0.120, Loss w_fp: 0.013, Mask: 0.752 [2022-11-25 19:12:21,983][ INFO] Evaluation original >>>> meanIOU: 70.08

[2022-11-25 19:12:21,983][ INFO] ===========> Epoch: 9, LR: 0.0004, Previous best: 70.17 [2022-11-25 19:12:23,223][ INFO] Iters: 0, Total loss: 0.042, Loss x: 0.060, Loss s: 0.041, Loss w_fp: 0.006, Mask: 0.712 [2022-11-25 19:14:55,485][ INFO] Iters: 310, Total loss: 0.081, Loss x: 0.089, Loss s: 0.133, Loss w_fp: 0.014, Mask: 0.761 [2022-11-25 19:17:28,156][ INFO] Iters: 620, Total loss: 0.080, Loss x: 0.092, Loss s: 0.124, Loss w_fp: 0.013, Mask: 0.763 [2022-11-25 19:19:59,950][ INFO] Iters: 930, Total loss: 0.080, Loss x: 0.094, Loss s: 0.120, Loss w_fp: 0.013, Mask: 0.765 [2022-11-25 19:22:30,970][ INFO] Iters: 1240, Total loss: 0.080, Loss x: 0.094, Loss s: 0.119, Loss w_fp: 0.013, Mask: 0.764 [2022-11-25 19:25:03,080][ INFO] Iters: 1550, Total loss: 0.079, Loss x: 0.093, Loss s: 0.119, Loss w_fp: 0.012, Mask: 0.765 [2022-11-25 19:27:34,553][ INFO] Iters: 1860, Total loss: 0.079, Loss x: 0.093, Loss s: 0.118, Loss w_fp: 0.012, Mask: 0.763 [2022-11-25 19:30:05,906][ INFO] Iters: 2170, Total loss: 0.079, Loss x: 0.093, Loss s: 0.118, Loss w_fp: 0.012, Mask: 0.764 [2022-11-25 19:33:40,794][ INFO] Evaluation original >>>> meanIOU: 71.09

[2022-11-25 19:33:41,548][ INFO] ===========> Epoch: 10, LR: 0.0004, Previous best: 71.09 [2022-11-25 19:33:42,738][ INFO] Iters: 0, Total loss: 0.207, Loss x: 0.108, Loss s: 0.588, Loss w_fp: 0.026, Mask: 0.719 [2022-11-25 19:36:14,250][ INFO] Iters: 310, Total loss: 0.079, Loss x: 0.093, Loss s: 0.120, Loss w_fp: 0.011, Mask: 0.760 [2022-11-25 19:38:45,860][ INFO] Iters: 620, Total loss: 0.080, Loss x: 0.096, Loss s: 0.118, Loss w_fp: 0.011, Mask: 0.758 [2022-11-25 19:41:17,631][ INFO] Iters: 930, Total loss: 0.080, Loss x: 0.095, Loss s: 0.118, Loss w_fp: 0.011, Mask: 0.762 [2022-11-25 19:43:49,302][ INFO] Iters: 1240, Total loss: 0.080, Loss x: 0.096, Loss s: 0.116, Loss w_fp: 0.012, Mask: 0.761 [2022-11-25 19:46:21,250][ INFO] Iters: 1550, Total loss: 0.080, Loss x: 0.096, Loss s: 0.118, Loss w_fp: 0.012, Mask: 0.762 [2022-11-25 19:48:53,459][ INFO] Iters: 1860, Total loss: 0.080, Loss x: 0.095, Loss s: 0.118, Loss w_fp: 0.012, Mask: 0.760 [2022-11-25 19:51:25,058][ INFO] Iters: 2170, Total loss: 0.080, Loss x: 0.096, Loss s: 0.117, Loss w_fp: 0.012, Mask: 0.761 [2022-11-25 19:55:00,769][ INFO] Evaluation original >>>> meanIOU: 72.05

[2022-11-25 19:55:01,511][ INFO] ===========> Epoch: 11, LR: 0.0004, Previous best: 72.05 [2022-11-25 19:55:02,744][ INFO] Iters: 0, Total loss: 0.098, Loss x: 0.028, Loss s: 0.335, Loss w_fp: 0.003, Mask: 0.270 [2022-11-25 19:57:34,080][ INFO] Iters: 310, Total loss: 0.083, Loss x: 0.105, Loss s: 0.112, Loss w_fp: 0.011, Mask: 0.768 [2022-11-25 20:00:05,952][ INFO] Iters: 620, Total loss: 0.078, Loss x: 0.097, Loss s: 0.108, Loss w_fp: 0.011, Mask: 0.771 [2022-11-25 20:02:37,223][ INFO] Iters: 930, Total loss: 0.078, Loss x: 0.095, Loss s: 0.110, Loss w_fp: 0.011, Mask: 0.775 [2022-11-25 20:05:09,282][ INFO] Iters: 1240, Total loss: 0.077, Loss x: 0.093, Loss s: 0.112, Loss w_fp: 0.011, Mask: 0.773 [2022-11-25 20:07:41,667][ INFO] Iters: 1550, Total loss: 0.076, Loss x: 0.091, Loss s: 0.112, Loss w_fp: 0.011, Mask: 0.774 [2022-11-25 20:10:14,422][ INFO] Iters: 1860, Total loss: 0.077, Loss x: 0.090, Loss s: 0.116, Loss w_fp: 0.011, Mask: 0.774 [2022-11-25 20:12:46,237][ INFO] Iters: 2170, Total loss: 0.076, Loss x: 0.090, Loss s: 0.115, Loss w_fp: 0.011, Mask: 0.776 [2022-11-25 20:16:23,248][ INFO] Evaluation original >>>> meanIOU: 71.68

[2022-11-25 20:16:23,248][ INFO] ===========> Epoch: 12, LR: 0.0004, Previous best: 72.05 [2022-11-25 20:16:24,392][ INFO] Iters: 0, Total loss: 0.051, Loss x: 0.050, Loss s: 0.071, Loss w_fp: 0.034, Mask: 0.730 [2022-11-25 20:18:56,379][ INFO] Iters: 310, Total loss: 0.077, Loss x: 0.088, Loss s: 0.122, Loss w_fp: 0.011, Mask: 0.779 [2022-11-25 20:21:28,478][ INFO] Iters: 620, Total loss: 0.075, Loss x: 0.085, Loss s: 0.122, Loss w_fp: 0.011, Mask: 0.782 [2022-11-25 20:24:00,077][ INFO] Iters: 930, Total loss: 0.073, Loss x: 0.084, Loss s: 0.115, Loss w_fp: 0.011, Mask: 0.783 [2022-11-25 20:26:31,537][ INFO] Iters: 1240, Total loss: 0.074, Loss x: 0.084, Loss s: 0.117, Loss w_fp: 0.011, Mask: 0.782 [2022-11-25 20:29:03,227][ INFO] Iters: 1550, Total loss: 0.074, Loss x: 0.085, Loss s: 0.116, Loss w_fp: 0.011, Mask: 0.779 [2022-11-25 20:31:35,276][ INFO] Iters: 1860, Total loss: 0.074, Loss x: 0.085, Loss s: 0.115, Loss w_fp: 0.011, Mask: 0.779 [2022-11-25 20:34:06,577][ INFO] Iters: 2170, Total loss: 0.074, Loss x: 0.084, Loss s: 0.115, Loss w_fp: 0.011, Mask: 0.781 [2022-11-25 20:37:42,721][ INFO] Evaluation original >>>> meanIOU: 71.79

[2022-11-25 20:37:42,721][ INFO] ===========> Epoch: 13, LR: 0.0004, Previous best: 72.05 [2022-11-25 20:37:44,065][ INFO] Iters: 0, Total loss: 0.057, Loss x: 0.095, Loss s: 0.035, Loss w_fp: 0.004, Mask: 0.880 [2022-11-25 20:40:15,848][ INFO] Iters: 310, Total loss: 0.073, Loss x: 0.085, Loss s: 0.112, Loss w_fp: 0.009, Mask: 0.781 [2022-11-25 20:42:47,639][ INFO] Iters: 620, Total loss: 0.072, Loss x: 0.084, Loss s: 0.113, Loss w_fp: 0.010, Mask: 0.784 [2022-11-25 20:45:18,999][ INFO] Iters: 930, Total loss: 0.072, Loss x: 0.082, Loss s: 0.111, Loss w_fp: 0.010, Mask: 0.785 [2022-11-25 20:47:51,388][ INFO] Iters: 1240, Total loss: 0.072, Loss x: 0.083, Loss s: 0.112, Loss w_fp: 0.010, Mask: 0.785 [2022-11-25 20:50:23,184][ INFO] Iters: 1550, Total loss: 0.071, Loss x: 0.082, Loss s: 0.111, Loss w_fp: 0.011, Mask: 0.787 [2022-11-25 20:52:54,934][ INFO] Iters: 1860, Total loss: 0.071, Loss x: 0.083, Loss s: 0.109, Loss w_fp: 0.011, Mask: 0.789 [2022-11-25 20:55:26,701][ INFO] Iters: 2170, Total loss: 0.072, Loss x: 0.083, Loss s: 0.111, Loss w_fp: 0.010, Mask: 0.789 [2022-11-25 20:59:03,060][ INFO] Evaluation original >>>> meanIOU: 72.21

[2022-11-25 20:59:03,741][ INFO] ===========> Epoch: 14, LR: 0.0004, Previous best: 72.21 [2022-11-25 20:59:04,979][ INFO] Iters: 0, Total loss: 0.110, Loss x: 0.145, Loss s: 0.132, Loss w_fp: 0.016, Mask: 0.884 [2022-11-25 21:01:36,628][ INFO] Iters: 310, Total loss: 0.069, Loss x: 0.080, Loss s: 0.106, Loss w_fp: 0.011, Mask: 0.795 [2022-11-25 21:04:08,967][ INFO] Iters: 620, Total loss: 0.069, Loss x: 0.080, Loss s: 0.108, Loss w_fp: 0.011, Mask: 0.796 [2022-11-25 21:06:40,441][ INFO] Iters: 930, Total loss: 0.068, Loss x: 0.078, Loss s: 0.106, Loss w_fp: 0.010, Mask: 0.796 [2022-11-25 21:09:12,309][ INFO] Iters: 1240, Total loss: 0.069, Loss x: 0.080, Loss s: 0.106, Loss w_fp: 0.010, Mask: 0.794 [2022-11-25 21:11:44,643][ INFO] Iters: 1550, Total loss: 0.068, Loss x: 0.079, Loss s: 0.106, Loss w_fp: 0.010, Mask: 0.794 [2022-11-25 21:14:16,856][ INFO] Iters: 1860, Total loss: 0.068, Loss x: 0.078, Loss s: 0.107, Loss w_fp: 0.010, Mask: 0.793 [2022-11-25 21:16:48,650][ INFO] Iters: 2170, Total loss: 0.069, Loss x: 0.079, Loss s: 0.108, Loss w_fp: 0.010, Mask: 0.791 [2022-11-25 21:20:24,370][ INFO] Evaluation original >>>> meanIOU: 73.29

[2022-11-25 21:20:25,107][ INFO] ===========> Epoch: 15, LR: 0.0004, Previous best: 73.29 [2022-11-25 21:20:26,304][ INFO] Iters: 0, Total loss: 0.046, Loss x: 0.058, Loss s: 0.051, Loss w_fp: 0.018, Mask: 0.899 [2022-11-25 21:22:57,513][ INFO] Iters: 310, Total loss: 0.067, Loss x: 0.077, Loss s: 0.103, Loss w_fp: 0.009, Mask: 0.804 [2022-11-25 21:25:29,872][ INFO] Iters: 620, Total loss: 0.067, Loss x: 0.076, Loss s: 0.107, Loss w_fp: 0.010, Mask: 0.797 [2022-11-25 21:28:01,570][ INFO] Iters: 930, Total loss: 0.069, Loss x: 0.077, Loss s: 0.112, Loss w_fp: 0.010, Mask: 0.796 [2022-11-25 21:30:33,722][ INFO] Iters: 1240, Total loss: 0.069, Loss x: 0.078, Loss s: 0.112, Loss w_fp: 0.010, Mask: 0.793 [2022-11-25 21:33:05,826][ INFO] Iters: 1550, Total loss: 0.068, Loss x: 0.077, Loss s: 0.110, Loss w_fp: 0.010, Mask: 0.795 [2022-11-25 21:35:37,127][ INFO] Iters: 1860, Total loss: 0.068, Loss x: 0.077, Loss s: 0.111, Loss w_fp: 0.010, Mask: 0.795 [2022-11-25 21:38:09,178][ INFO] Iters: 2170, Total loss: 0.069, Loss x: 0.077, Loss s: 0.110, Loss w_fp: 0.009, Mask: 0.793 [2022-11-25 21:41:44,043][ INFO] Evaluation original >>>> meanIOU: 73.38

[2022-11-25 21:41:44,735][ INFO] ===========> Epoch: 16, LR: 0.0004, Previous best: 73.38 [2022-11-25 21:41:45,856][ INFO] Iters: 0, Total loss: 0.032, Loss x: 0.036, Loss s: 0.054, Loss w_fp: 0.004, Mask: 0.877 [2022-11-25 21:44:17,663][ INFO] Iters: 310, Total loss: 0.070, Loss x: 0.080, Loss s: 0.112, Loss w_fp: 0.009, Mask: 0.796 [2022-11-25 21:46:49,893][ INFO] Iters: 620, Total loss: 0.067, Loss x: 0.077, Loss s: 0.105, Loss w_fp: 0.011, Mask: 0.798 [2022-11-25 21:49:21,569][ INFO] Iters: 930, Total loss: 0.067, Loss x: 0.076, Loss s: 0.106, Loss w_fp: 0.010, Mask: 0.799 [2022-11-25 21:51:53,145][ INFO] Iters: 1240, Total loss: 0.067, Loss x: 0.076, Loss s: 0.105, Loss w_fp: 0.010, Mask: 0.798 [2022-11-25 21:54:24,692][ INFO] Iters: 1550, Total loss: 0.067, Loss x: 0.076, Loss s: 0.106, Loss w_fp: 0.010, Mask: 0.801 [2022-11-25 21:56:56,705][ INFO] Iters: 1860, Total loss: 0.066, Loss x: 0.075, Loss s: 0.106, Loss w_fp: 0.010, Mask: 0.801 [2022-11-25 21:59:28,836][ INFO] Iters: 2170, Total loss: 0.066, Loss x: 0.075, Loss s: 0.106, Loss w_fp: 0.010, Mask: 0.799 [2022-11-25 22:03:05,959][ INFO] Evaluation original >>>> meanIOU: 73.11

[2022-11-25 22:03:05,959][ INFO] ===========> Epoch: 17, LR: 0.0004, Previous best: 73.38 [2022-11-25 22:03:07,126][ INFO] Iters: 0, Total loss: 0.060, Loss x: 0.083, Loss s: 0.064, Loss w_fp: 0.012, Mask: 0.715 [2022-11-25 22:05:38,886][ INFO] Iters: 310, Total loss: 0.064, Loss x: 0.075, Loss s: 0.097, Loss w_fp: 0.010, Mask: 0.810 [2022-11-25 22:08:10,586][ INFO] Iters: 620, Total loss: 0.065, Loss x: 0.074, Loss s: 0.103, Loss w_fp: 0.010, Mask: 0.807 [2022-11-25 22:10:42,156][ INFO] Iters: 930, Total loss: 0.067, Loss x: 0.075, Loss s: 0.106, Loss w_fp: 0.010, Mask: 0.805 [2022-11-25 22:13:13,982][ INFO] Iters: 1240, Total loss: 0.066, Loss x: 0.074, Loss s: 0.106, Loss w_fp: 0.010, Mask: 0.805 [2022-11-25 22:15:46,311][ INFO] Iters: 1550, Total loss: 0.066, Loss x: 0.073, Loss s: 0.107, Loss w_fp: 0.010, Mask: 0.807 [2022-11-25 22:18:18,647][ INFO] Iters: 1860, Total loss: 0.066, Loss x: 0.073, Loss s: 0.107, Loss w_fp: 0.010, Mask: 0.807 [2022-11-25 22:20:50,422][ INFO] Iters: 2170, Total loss: 0.066, Loss x: 0.073, Loss s: 0.107, Loss w_fp: 0.010, Mask: 0.805 [2022-11-25 22:24:26,194][ INFO] Evaluation original >>>> meanIOU: 72.51

[2022-11-25 22:24:26,195][ INFO] ===========> Epoch: 18, LR: 0.0004, Previous best: 73.38 [2022-11-25 22:24:27,416][ INFO] Iters: 0, Total loss: 0.075, Loss x: 0.096, Loss s: 0.101, Loss w_fp: 0.009, Mask: 0.850 [2022-11-25 22:26:59,062][ INFO] Iters: 310, Total loss: 0.063, Loss x: 0.069, Loss s: 0.104, Loss w_fp: 0.010, Mask: 0.802 [2022-11-25 22:29:31,213][ INFO] Iters: 620, Total loss: 0.063, Loss x: 0.070, Loss s: 0.101, Loss w_fp: 0.010, Mask: 0.805 [2022-11-25 22:32:03,120][ INFO] Iters: 930, Total loss: 0.065, Loss x: 0.072, Loss s: 0.106, Loss w_fp: 0.010, Mask: 0.800 [2022-11-25 22:34:34,807][ INFO] Iters: 1240, Total loss: 0.065, Loss x: 0.073, Loss s: 0.106, Loss w_fp: 0.010, Mask: 0.802 [2022-11-25 22:37:06,460][ INFO] Iters: 1550, Total loss: 0.066, Loss x: 0.073, Loss s: 0.107, Loss w_fp: 0.010, Mask: 0.803 [2022-11-25 22:39:38,286][ INFO] Iters: 1860, Total loss: 0.066, Loss x: 0.075, Loss s: 0.107, Loss w_fp: 0.010, Mask: 0.802 [2022-11-25 22:42:10,421][ INFO] Iters: 2170, Total loss: 0.067, Loss x: 0.074, Loss s: 0.108, Loss w_fp: 0.010, Mask: 0.802 [2022-11-25 22:45:47,389][ INFO] Evaluation original >>>> meanIOU: 72.03

[2022-11-25 22:45:47,390][ INFO] ===========> Epoch: 19, LR: 0.0004, Previous best: 73.38 [2022-11-25 22:45:48,542][ INFO] Iters: 0, Total loss: 0.044, Loss x: 0.023, Loss s: 0.126, Loss w_fp: 0.006, Mask: 0.831 [2022-11-25 22:48:20,031][ INFO] Iters: 310, Total loss: 0.061, Loss x: 0.067, Loss s: 0.103, Loss w_fp: 0.009, Mask: 0.807 [2022-11-25 22:50:51,460][ INFO] Iters: 620, Total loss: 0.062, Loss x: 0.067, Loss s: 0.105, Loss w_fp: 0.009, Mask: 0.813 [2022-11-25 22:53:23,441][ INFO] Iters: 930, Total loss: 0.063, Loss x: 0.068, Loss s: 0.104, Loss w_fp: 0.009, Mask: 0.810 [2022-11-25 22:55:55,332][ INFO] Iters: 1240, Total loss: 0.063, Loss x: 0.068, Loss s: 0.105, Loss w_fp: 0.009, Mask: 0.810 [2022-11-25 22:58:27,054][ INFO] Iters: 1550, Total loss: 0.063, Loss x: 0.068, Loss s: 0.104, Loss w_fp: 0.009, Mask: 0.810 [2022-11-25 23:00:59,185][ INFO] Iters: 1860, Total loss: 0.063, Loss x: 0.069, Loss s: 0.104, Loss w_fp: 0.009, Mask: 0.810 [2022-11-25 23:03:30,841][ INFO] Iters: 2170, Total loss: 0.063, Loss x: 0.069, Loss s: 0.104, Loss w_fp: 0.009, Mask: 0.811 [2022-11-25 23:07:07,834][ INFO] Evaluation original >>>> meanIOU: 73.40

[2022-11-25 23:07:08,760][ INFO] ===========> Epoch: 20, LR: 0.0004, Previous best: 73.40 [2022-11-25 23:07:10,052][ INFO] Iters: 0, Total loss: 0.109, Loss x: 0.097, Loss s: 0.241, Loss w_fp: 0.002, Mask: 0.909 [2022-11-25 23:09:41,545][ INFO] Iters: 310, Total loss: 0.064, Loss x: 0.072, Loss s: 0.102, Loss w_fp: 0.010, Mask: 0.804 [2022-11-25 23:12:13,575][ INFO] Iters: 620, Total loss: 0.063, Loss x: 0.071, Loss s: 0.101, Loss w_fp: 0.009, Mask: 0.803 [2022-11-25 23:14:45,947][ INFO] Iters: 930, Total loss: 0.062, Loss x: 0.070, Loss s: 0.101, Loss w_fp: 0.009, Mask: 0.807 [2022-11-25 23:17:18,124][ INFO] Iters: 1240, Total loss: 0.062, Loss x: 0.070, Loss s: 0.099, Loss w_fp: 0.009, Mask: 0.808 [2022-11-25 23:19:50,623][ INFO] Iters: 1550, Total loss: 0.062, Loss x: 0.071, Loss s: 0.099, Loss w_fp: 0.009, Mask: 0.811 [2022-11-25 23:22:23,298][ INFO] Iters: 1860, Total loss: 0.063, Loss x: 0.071, Loss s: 0.099, Loss w_fp: 0.009, Mask: 0.812 [2022-11-25 23:24:55,918][ INFO] Iters: 2170, Total loss: 0.063, Loss x: 0.071, Loss s: 0.101, Loss w_fp: 0.009, Mask: 0.811 [2022-11-25 23:28:33,052][ INFO] Evaluation original >>>> meanIOU: 72.97

[2022-11-25 23:28:33,053][ INFO] ===========> Epoch: 21, LR: 0.0004, Previous best: 73.40 [2022-11-25 23:28:34,409][ INFO] Iters: 0, Total loss: 0.164, Loss x: 0.097, Loss s: 0.416, Loss w_fp: 0.048, Mask: 0.562 [2022-11-25 23:31:06,463][ INFO] Iters: 310, Total loss: 0.065, Loss x: 0.072, Loss s: 0.106, Loss w_fp: 0.008, Mask: 0.819 [2022-11-25 23:33:38,695][ INFO] Iters: 620, Total loss: 0.065, Loss x: 0.071, Loss s: 0.108, Loss w_fp: 0.009, Mask: 0.820 [2022-11-25 23:36:10,812][ INFO] Iters: 930, Total loss: 0.063, Loss x: 0.070, Loss s: 0.101, Loss w_fp: 0.009, Mask: 0.819 [2022-11-25 23:38:43,080][ INFO] Iters: 1240, Total loss: 0.063, Loss x: 0.070, Loss s: 0.103, Loss w_fp: 0.009, Mask: 0.817 [2022-11-25 23:41:14,832][ INFO] Iters: 1550, Total loss: 0.063, Loss x: 0.069, Loss s: 0.104, Loss w_fp: 0.009, Mask: 0.816 [2022-11-25 23:43:46,867][ INFO] Iters: 1860, Total loss: 0.062, Loss x: 0.069, Loss s: 0.104, Loss w_fp: 0.009, Mask: 0.817 [2022-11-25 23:46:18,692][ INFO] Iters: 2170, Total loss: 0.062, Loss x: 0.068, Loss s: 0.102, Loss w_fp: 0.009, Mask: 0.817 [2022-11-25 23:49:56,159][ INFO] Evaluation original >>>> meanIOU: 73.91

[2022-11-25 23:49:56,920][ INFO] ===========> Epoch: 22, LR: 0.0004, Previous best: 73.91 [2022-11-25 23:49:58,238][ INFO] Iters: 0, Total loss: 0.044, Loss x: 0.050, Loss s: 0.072, Loss w_fp: 0.005, Mask: 0.837 [2022-11-25 23:52:30,334][ INFO] Iters: 310, Total loss: 0.062, Loss x: 0.065, Loss s: 0.107, Loss w_fp: 0.010, Mask: 0.814 [2022-11-25 23:55:02,457][ INFO] Iters: 620, Total loss: 0.060, Loss x: 0.064, Loss s: 0.103, Loss w_fp: 0.009, Mask: 0.817 [2022-11-25 23:57:34,409][ INFO] Iters: 930, Total loss: 0.061, Loss x: 0.065, Loss s: 0.105, Loss w_fp: 0.009, Mask: 0.819 [2022-11-26 00:00:06,086][ INFO] Iters: 1240, Total loss: 0.061, Loss x: 0.065, Loss s: 0.105, Loss w_fp: 0.009, Mask: 0.817 [2022-11-26 00:02:37,744][ INFO] Iters: 1550, Total loss: 0.062, Loss x: 0.066, Loss s: 0.105, Loss w_fp: 0.009, Mask: 0.818 [2022-11-26 00:05:09,704][ INFO] Iters: 1860, Total loss: 0.061, Loss x: 0.066, Loss s: 0.104, Loss w_fp: 0.009, Mask: 0.817 [2022-11-26 00:07:42,003][ INFO] Iters: 2170, Total loss: 0.061, Loss x: 0.066, Loss s: 0.103, Loss w_fp: 0.009, Mask: 0.816 [2022-11-26 00:11:18,139][ INFO] Evaluation original >>>> meanIOU: 73.62

[2022-11-26 00:11:18,139][ INFO] ===========> Epoch: 23, LR: 0.0004, Previous best: 73.91 [2022-11-26 00:11:19,442][ INFO] Iters: 0, Total loss: 0.109, Loss x: 0.082, Loss s: 0.261, Loss w_fp: 0.013, Mask: 0.752 [2022-11-26 00:13:51,170][ INFO] Iters: 310, Total loss: 0.058, Loss x: 0.066, Loss s: 0.091, Loss w_fp: 0.008, Mask: 0.813 [2022-11-26 00:16:23,112][ INFO] Iters: 620, Total loss: 0.057, Loss x: 0.064, Loss s: 0.093, Loss w_fp: 0.008, Mask: 0.820 [2022-11-26 00:18:55,130][ INFO] Iters: 930, Total loss: 0.058, Loss x: 0.063, Loss s: 0.095, Loss w_fp: 0.009, Mask: 0.818 [2022-11-26 00:21:26,712][ INFO] Iters: 1240, Total loss: 0.058, Loss x: 0.064, Loss s: 0.095, Loss w_fp: 0.008, Mask: 0.820 [2022-11-26 00:23:58,878][ INFO] Iters: 1550, Total loss: 0.058, Loss x: 0.065, Loss s: 0.096, Loss w_fp: 0.009, Mask: 0.820 [2022-11-26 00:26:26,518][ INFO] Iters: 1860, Total loss: 0.059, Loss x: 0.065, Loss s: 0.097, Loss w_fp: 0.009, Mask: 0.821 [2022-11-26 00:28:53,696][ INFO] Iters: 2170, Total loss: 0.059, Loss x: 0.065, Loss s: 0.097, Loss w_fp: 0.009, Mask: 0.822 [2022-11-26 00:32:15,014][ INFO] Evaluation original >>>> meanIOU: 73.14

[2022-11-26 00:32:15,015][ INFO] ===========> Epoch: 24, LR: 0.0004, Previous best: 73.91 [2022-11-26 00:32:16,136][ INFO] Iters: 0, Total loss: 0.022, Loss x: 0.030, Loss s: 0.024, Loss w_fp: 0.003, Mask: 0.900 [2022-11-26 00:34:43,617][ INFO] Iters: 310, Total loss: 0.056, Loss x: 0.064, Loss s: 0.087, Loss w_fp: 0.008, Mask: 0.829 [2022-11-26 00:37:15,260][ INFO] Iters: 620, Total loss: 0.056, Loss x: 0.063, Loss s: 0.089, Loss w_fp: 0.008, Mask: 0.828 [2022-11-26 00:39:46,748][ INFO] Iters: 930, Total loss: 0.057, Loss x: 0.063, Loss s: 0.094, Loss w_fp: 0.008, Mask: 0.826 [2022-11-26 00:42:18,556][ INFO] Iters: 1240, Total loss: 0.057, Loss x: 0.063, Loss s: 0.095, Loss w_fp: 0.008, Mask: 0.825 [2022-11-26 00:44:50,817][ INFO] Iters: 1550, Total loss: 0.058, Loss x: 0.063, Loss s: 0.098, Loss w_fp: 0.008, Mask: 0.825 [2022-11-26 00:47:22,182][ INFO] Iters: 1860, Total loss: 0.058, Loss x: 0.063, Loss s: 0.097, Loss w_fp: 0.008, Mask: 0.825 [2022-11-26 00:49:53,805][ INFO] Iters: 2170, Total loss: 0.058, Loss x: 0.063, Loss s: 0.097, Loss w_fp: 0.008, Mask: 0.824 [2022-11-26 00:53:30,153][ INFO] Evaluation original >>>> meanIOU: 72.66

[2022-11-26 00:53:30,153][ INFO] ===========> Epoch: 25, LR: 0.0004, Previous best: 73.91 [2022-11-26 00:53:31,353][ INFO] Iters: 0, Total loss: 0.062, Loss x: 0.113, Loss s: 0.018, Loss w_fp: 0.006, Mask: 0.870 [2022-11-26 00:56:02,955][ INFO] Iters: 310, Total loss: 0.059, Loss x: 0.063, Loss s: 0.102, Loss w_fp: 0.009, Mask: 0.825 [2022-11-26 00:58:34,438][ INFO] Iters: 620, Total loss: 0.060, Loss x: 0.063, Loss s: 0.104, Loss w_fp: 0.009, Mask: 0.820 [2022-11-26 01:01:06,570][ INFO] Iters: 930, Total loss: 0.059, Loss x: 0.064, Loss s: 0.101, Loss w_fp: 0.009, Mask: 0.822 [2022-11-26 01:03:37,871][ INFO] Iters: 1240, Total loss: 0.059, Loss x: 0.064, Loss s: 0.099, Loss w_fp: 0.009, Mask: 0.823 [2022-11-26 01:06:10,063][ INFO] Iters: 1550, Total loss: 0.058, Loss x: 0.063, Loss s: 0.098, Loss w_fp: 0.008, Mask: 0.824 [2022-11-26 01:08:41,765][ INFO] Iters: 1860, Total loss: 0.058, Loss x: 0.064, Loss s: 0.098, Loss w_fp: 0.009, Mask: 0.824 [2022-11-26 01:11:13,881][ INFO] Iters: 2170, Total loss: 0.059, Loss x: 0.064, Loss s: 0.098, Loss w_fp: 0.009, Mask: 0.824 [2022-11-26 01:14:49,221][ INFO] Evaluation original >>>> meanIOU: 74.33

[2022-11-26 01:14:49,940][ INFO] ===========> Epoch: 26, LR: 0.0004, Previous best: 74.33 [2022-11-26 01:14:51,106][ INFO] Iters: 0, Total loss: 0.052, Loss x: 0.058, Loss s: 0.075, Loss w_fp: 0.018, Mask: 0.844 [2022-11-26 01:17:22,892][ INFO] Iters: 310, Total loss: 0.056, Loss x: 0.062, Loss s: 0.091, Loss w_fp: 0.008, Mask: 0.832 [2022-11-26 01:19:55,890][ INFO] Iters: 620, Total loss: 0.055, Loss x: 0.061, Loss s: 0.091, Loss w_fp: 0.009, Mask: 0.832 [2022-11-26 01:22:27,307][ INFO] Iters: 930, Total loss: 0.057, Loss x: 0.061, Loss s: 0.097, Loss w_fp: 0.009, Mask: 0.833 [2022-11-26 01:24:59,476][ INFO] Iters: 1240, Total loss: 0.057, Loss x: 0.061, Loss s: 0.097, Loss w_fp: 0.009, Mask: 0.832 [2022-11-26 01:27:31,186][ INFO] Iters: 1550, Total loss: 0.057, Loss x: 0.061, Loss s: 0.097, Loss w_fp: 0.008, Mask: 0.830 [2022-11-26 01:30:03,458][ INFO] Iters: 1860, Total loss: 0.057, Loss x: 0.061, Loss s: 0.097, Loss w_fp: 0.008, Mask: 0.830 [2022-11-26 01:32:35,568][ INFO] Iters: 2170, Total loss: 0.057, Loss x: 0.061, Loss s: 0.097, Loss w_fp: 0.009, Mask: 0.831 [2022-11-26 01:36:12,444][ INFO] Evaluation original >>>> meanIOU: 73.00

[2022-11-26 01:36:12,445][ INFO] ===========> Epoch: 27, LR: 0.0003, Previous best: 74.33 [2022-11-26 01:36:13,783][ INFO] Iters: 0, Total loss: 0.070, Loss x: 0.092, Loss s: 0.091, Loss w_fp: 0.006, Mask: 0.827 [2022-11-26 01:38:45,911][ INFO] Iters: 310, Total loss: 0.064, Loss x: 0.065, Loss s: 0.115, Loss w_fp: 0.010, Mask: 0.821 [2022-11-26 01:41:17,162][ INFO] Iters: 620, Total loss: 0.062, Loss x: 0.065, Loss s: 0.108, Loss w_fp: 0.009, Mask: 0.823 [2022-11-26 01:43:49,374][ INFO] Iters: 930, Total loss: 0.061, Loss x: 0.064, Loss s: 0.106, Loss w_fp: 0.009, Mask: 0.823 [2022-11-26 01:46:21,581][ INFO] Iters: 1240, Total loss: 0.060, Loss x: 0.063, Loss s: 0.103, Loss w_fp: 0.009, Mask: 0.824 [2022-11-26 01:48:52,891][ INFO] Iters: 1550, Total loss: 0.059, Loss x: 0.063, Loss s: 0.102, Loss w_fp: 0.009, Mask: 0.825 [2022-11-26 01:51:24,832][ INFO] Iters: 1860, Total loss: 0.059, Loss x: 0.063, Loss s: 0.101, Loss w_fp: 0.009, Mask: 0.826 [2022-11-26 01:53:56,355][ INFO] Iters: 2170, Total loss: 0.059, Loss x: 0.063, Loss s: 0.101, Loss w_fp: 0.009, Mask: 0.827 [2022-11-26 01:57:24,997][ INFO] Evaluation original >>>> meanIOU: 75.16

[2022-11-26 01:57:25,717][ INFO] ===========> Epoch: 28, LR: 0.0003, Previous best: 75.16 [2022-11-26 01:57:26,886][ INFO] Iters: 0, Total loss: 0.075, Loss x: 0.039, Loss s: 0.217, Loss w_fp: 0.003, Mask: 0.723 [2022-11-26 01:59:54,536][ INFO] Iters: 310, Total loss: 0.056, Loss x: 0.060, Loss s: 0.095, Loss w_fp: 0.008, Mask: 0.833 [2022-11-26 02:02:22,276][ INFO] Iters: 620, Total loss: 0.058, Loss x: 0.061, Loss s: 0.100, Loss w_fp: 0.008, Mask: 0.834 [2022-11-26 02:04:53,495][ INFO] Iters: 930, Total loss: 0.058, Loss x: 0.061, Loss s: 0.102, Loss w_fp: 0.008, Mask: 0.832 [2022-11-26 02:07:25,745][ INFO] Iters: 1240, Total loss: 0.057, Loss x: 0.061, Loss s: 0.100, Loss w_fp: 0.008, Mask: 0.831 [2022-11-26 02:09:58,255][ INFO] Iters: 1550, Total loss: 0.057, Loss x: 0.061, Loss s: 0.099, Loss w_fp: 0.008, Mask: 0.831 [2022-11-26 02:12:29,832][ INFO] Iters: 1860, Total loss: 0.057, Loss x: 0.061, Loss s: 0.097, Loss w_fp: 0.008, Mask: 0.833 [2022-11-26 02:15:01,580][ INFO] Iters: 2170, Total loss: 0.057, Loss x: 0.061, Loss s: 0.097, Loss w_fp: 0.008, Mask: 0.832 [2022-11-26 02:18:38,219][ INFO] Evaluation original >>>> meanIOU: 74.21

[2022-11-26 02:18:38,219][ INFO] ===========> Epoch: 29, LR: 0.0003, Previous best: 75.16 [2022-11-26 02:18:39,527][ INFO] Iters: 0, Total loss: 0.039, Loss x: 0.040, Loss s: 0.070, Loss w_fp: 0.008, Mask: 0.750 [2022-11-26 02:21:11,341][ INFO] Iters: 310, Total loss: 0.055, Loss x: 0.061, Loss s: 0.091, Loss w_fp: 0.009, Mask: 0.833 [2022-11-26 02:23:42,798][ INFO] Iters: 620, Total loss: 0.056, Loss x: 0.060, Loss s: 0.094, Loss w_fp: 0.008, Mask: 0.832 [2022-11-26 02:26:14,431][ INFO] Iters: 930, Total loss: 0.055, Loss x: 0.059, Loss s: 0.093, Loss w_fp: 0.008, Mask: 0.833 [2022-11-26 02:28:46,564][ INFO] Iters: 1240, Total loss: 0.055, Loss x: 0.059, Loss s: 0.092, Loss w_fp: 0.008, Mask: 0.836 [2022-11-26 02:31:18,480][ INFO] Iters: 1550, Total loss: 0.055, Loss x: 0.059, Loss s: 0.094, Loss w_fp: 0.008, Mask: 0.833 [2022-11-26 02:33:50,321][ INFO] Iters: 1860, Total loss: 0.054, Loss x: 0.058, Loss s: 0.093, Loss w_fp: 0.008, Mask: 0.835 [2022-11-26 02:36:22,296][ INFO] Iters: 2170, Total loss: 0.055, Loss x: 0.059, Loss s: 0.093, Loss w_fp: 0.008, Mask: 0.834 [2022-11-26 02:39:58,409][ INFO] Evaluation original >>>> meanIOU: 75.50

[2022-11-26 02:39:59,175][ INFO] ===========> Epoch: 30, LR: 0.0003, Previous best: 75.50 [2022-11-26 02:40:00,395][ INFO] Iters: 0, Total loss: 0.050, Loss x: 0.060, Loss s: 0.073, Loss w_fp: 0.007, Mask: 0.738 [2022-11-26 02:42:32,158][ INFO] Iters: 310, Total loss: 0.056, Loss x: 0.059, Loss s: 0.097, Loss w_fp: 0.007, Mask: 0.836 [2022-11-26 02:45:04,883][ INFO] Iters: 620, Total loss: 0.055, Loss x: 0.058, Loss s: 0.095, Loss w_fp: 0.008, Mask: 0.838 [2022-11-26 02:47:37,097][ INFO] Iters: 930, Total loss: 0.054, Loss x: 0.058, Loss s: 0.094, Loss w_fp: 0.007, Mask: 0.838 [2022-11-26 02:50:08,985][ INFO] Iters: 1240, Total loss: 0.056, Loss x: 0.061, Loss s: 0.094, Loss w_fp: 0.007, Mask: 0.834 [2022-11-26 02:52:40,988][ INFO] Iters: 1550, Total loss: 0.057, Loss x: 0.063, Loss s: 0.096, Loss w_fp: 0.008, Mask: 0.831 [2022-11-26 02:55:12,729][ INFO] Iters: 1860, Total loss: 0.058, Loss x: 0.064, Loss s: 0.095, Loss w_fp: 0.008, Mask: 0.828 [2022-11-26 02:57:44,575][ INFO] Iters: 2170, Total loss: 0.057, Loss x: 0.063, Loss s: 0.095, Loss w_fp: 0.008, Mask: 0.829 [2022-11-26 03:01:19,901][ INFO] Evaluation original >>>> meanIOU: 73.61

[2022-11-26 03:01:19,901][ INFO] ===========> Epoch: 31, LR: 0.0003, Previous best: 75.50 [2022-11-26 03:01:21,159][ INFO] Iters: 0, Total loss: 0.050, Loss x: 0.049, Loss s: 0.099, Loss w_fp: 0.005, Mask: 0.923 [2022-11-26 03:03:52,176][ INFO] Iters: 310, Total loss: 0.056, Loss x: 0.059, Loss s: 0.097, Loss w_fp: 0.008, Mask: 0.829 [2022-11-26 03:06:23,637][ INFO] Iters: 620, Total loss: 0.056, Loss x: 0.061, Loss s: 0.095, Loss w_fp: 0.008, Mask: 0.830 [2022-11-26 03:08:55,391][ INFO] Iters: 930, Total loss: 0.056, Loss x: 0.061, Loss s: 0.094, Loss w_fp: 0.008, Mask: 0.831 [2022-11-26 03:11:26,776][ INFO] Iters: 1240, Total loss: 0.056, Loss x: 0.061, Loss s: 0.094, Loss w_fp: 0.008, Mask: 0.833 [2022-11-26 03:13:58,162][ INFO] Iters: 1550, Total loss: 0.057, Loss x: 0.061, Loss s: 0.096, Loss w_fp: 0.008, Mask: 0.832 [2022-11-26 03:16:29,520][ INFO] Iters: 1860, Total loss: 0.057, Loss x: 0.061, Loss s: 0.096, Loss w_fp: 0.008, Mask: 0.833 [2022-11-26 03:19:01,022][ INFO] Iters: 2170, Total loss: 0.056, Loss x: 0.061, Loss s: 0.094, Loss w_fp: 0.008, Mask: 0.834 [2022-11-26 03:22:35,723][ INFO] Evaluation original >>>> meanIOU: 74.38

[2022-11-26 03:22:35,723][ INFO] ===========> Epoch: 32, LR: 0.0003, Previous best: 75.50 [2022-11-26 03:22:36,886][ INFO] Iters: 0, Total loss: 0.066, Loss x: 0.079, Loss s: 0.097, Loss w_fp: 0.009, Mask: 0.688 [2022-11-26 03:25:04,079][ INFO] Iters: 310, Total loss: 0.053, Loss x: 0.058, Loss s: 0.088, Loss w_fp: 0.008, Mask: 0.832 [2022-11-26 03:27:31,214][ INFO] Iters: 620, Total loss: 0.053, Loss x: 0.057, Loss s: 0.090, Loss w_fp: 0.008, Mask: 0.833 [2022-11-26 03:29:58,622][ INFO] Iters: 930, Total loss: 0.053, Loss x: 0.058, Loss s: 0.090, Loss w_fp: 0.008, Mask: 0.834 [2022-11-26 03:32:30,543][ INFO] Iters: 1240, Total loss: 0.054, Loss x: 0.057, Loss s: 0.093, Loss w_fp: 0.008, Mask: 0.836 [2022-11-26 03:35:02,917][ INFO] Iters: 1550, Total loss: 0.054, Loss x: 0.057, Loss s: 0.093, Loss w_fp: 0.008, Mask: 0.838 [2022-11-26 03:37:35,553][ INFO] Iters: 1860, Total loss: 0.054, Loss x: 0.057, Loss s: 0.092, Loss w_fp: 0.008, Mask: 0.839 [2022-11-26 03:40:07,923][ INFO] Iters: 2170, Total loss: 0.053, Loss x: 0.057, Loss s: 0.092, Loss w_fp: 0.008, Mask: 0.839 [2022-11-26 03:43:45,001][ INFO] Evaluation original >>>> meanIOU: 73.81

[2022-11-26 03:43:45,001][ INFO] ===========> Epoch: 33, LR: 0.0003, Previous best: 75.50 [2022-11-26 03:43:46,280][ INFO] Iters: 0, Total loss: 0.046, Loss x: 0.062, Loss s: 0.056, Loss w_fp: 0.004, Mask: 0.851 [2022-11-26 03:46:18,370][ INFO] Iters: 310, Total loss: 0.049, Loss x: 0.053, Loss s: 0.084, Loss w_fp: 0.008, Mask: 0.847 [2022-11-26 03:48:50,543][ INFO] Iters: 620, Total loss: 0.050, Loss x: 0.055, Loss s: 0.083, Loss w_fp: 0.007, Mask: 0.848 [2022-11-26 03:51:22,944][ INFO] Iters: 930, Total loss: 0.051, Loss x: 0.055, Loss s: 0.086, Loss w_fp: 0.007, Mask: 0.845 [2022-11-26 03:53:55,383][ INFO] Iters: 1240, Total loss: 0.051, Loss x: 0.055, Loss s: 0.086, Loss w_fp: 0.007, Mask: 0.844 [2022-11-26 03:56:27,481][ INFO] Iters: 1550, Total loss: 0.052, Loss x: 0.055, Loss s: 0.089, Loss w_fp: 0.008, Mask: 0.843 [2022-11-26 03:58:59,236][ INFO] Iters: 1860, Total loss: 0.052, Loss x: 0.055, Loss s: 0.089, Loss w_fp: 0.007, Mask: 0.844 [2022-11-26 04:01:31,274][ INFO] Iters: 2170, Total loss: 0.052, Loss x: 0.056, Loss s: 0.089, Loss w_fp: 0.007, Mask: 0.844 [2022-11-26 04:05:08,973][ INFO] Evaluation original >>>> meanIOU: 73.42

[2022-11-26 04:05:08,974][ INFO] ===========> Epoch: 34, LR: 0.0003, Previous best: 75.50 [2022-11-26 04:05:10,274][ INFO] Iters: 0, Total loss: 0.017, Loss x: 0.023, Loss s: 0.019, Loss w_fp: 0.005, Mask: 0.923 [2022-11-26 04:07:42,168][ INFO] Iters: 310, Total loss: 0.055, Loss x: 0.059, Loss s: 0.092, Loss w_fp: 0.009, Mask: 0.834 [2022-11-26 04:10:14,693][ INFO] Iters: 620, Total loss: 0.055, Loss x: 0.060, Loss s: 0.092, Loss w_fp: 0.008, Mask: 0.833 [2022-11-26 04:12:46,611][ INFO] Iters: 930, Total loss: 0.054, Loss x: 0.059, Loss s: 0.089, Loss w_fp: 0.008, Mask: 0.837 [2022-11-26 04:15:18,449][ INFO] Iters: 1240, Total loss: 0.054, Loss x: 0.058, Loss s: 0.091, Loss w_fp: 0.007, Mask: 0.837 [2022-11-26 04:17:50,212][ INFO] Iters: 1550, Total loss: 0.053, Loss x: 0.058, Loss s: 0.091, Loss w_fp: 0.007, Mask: 0.838 [2022-11-26 04:20:22,630][ INFO] Iters: 1860, Total loss: 0.054, Loss x: 0.057, Loss s: 0.092, Loss w_fp: 0.007, Mask: 0.838 [2022-11-26 04:22:54,465][ INFO] Iters: 2170, Total loss: 0.054, Loss x: 0.057, Loss s: 0.093, Loss w_fp: 0.008, Mask: 0.837 [2022-11-26 04:26:31,838][ INFO] Evaluation original >>>> meanIOU: 73.96

[2022-11-26 04:26:31,838][ INFO] ===========> Epoch: 35, LR: 0.0003, Previous best: 75.50 [2022-11-26 04:26:33,074][ INFO] Iters: 0, Total loss: 0.068, Loss x: 0.064, Loss s: 0.129, Loss w_fp: 0.016, Mask: 0.818 [2022-11-26 04:29:05,422][ INFO] Iters: 310, Total loss: 0.052, Loss x: 0.056, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.841 [2022-11-26 04:31:37,662][ INFO] Iters: 620, Total loss: 0.053, Loss x: 0.058, Loss s: 0.087, Loss w_fp: 0.007, Mask: 0.839 [2022-11-26 04:34:10,203][ INFO] Iters: 930, Total loss: 0.053, Loss x: 0.057, Loss s: 0.089, Loss w_fp: 0.007, Mask: 0.840 [2022-11-26 04:36:42,221][ INFO] Iters: 1240, Total loss: 0.052, Loss x: 0.056, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.841 [2022-11-26 04:39:14,643][ INFO] Iters: 1550, Total loss: 0.052, Loss x: 0.056, Loss s: 0.089, Loss w_fp: 0.007, Mask: 0.842 [2022-11-26 04:41:46,537][ INFO] Iters: 1860, Total loss: 0.052, Loss x: 0.056, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.843 [2022-11-26 04:44:19,157][ INFO] Iters: 2170, Total loss: 0.051, Loss x: 0.055, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.843 [2022-11-26 04:47:55,833][ INFO] Evaluation original >>>> meanIOU: 74.02

[2022-11-26 04:47:55,834][ INFO] ===========> Epoch: 36, LR: 0.0003, Previous best: 75.50 [2022-11-26 04:47:56,947][ INFO] Iters: 0, Total loss: 0.043, Loss x: 0.053, Loss s: 0.064, Loss w_fp: 0.004, Mask: 0.848 [2022-11-26 04:50:29,061][ INFO] Iters: 310, Total loss: 0.052, Loss x: 0.056, Loss s: 0.087, Loss w_fp: 0.008, Mask: 0.841 [2022-11-26 04:52:56,854][ INFO] Iters: 620, Total loss: 0.052, Loss x: 0.054, Loss s: 0.090, Loss w_fp: 0.008, Mask: 0.841 [2022-11-26 04:55:24,589][ INFO] Iters: 930, Total loss: 0.052, Loss x: 0.055, Loss s: 0.089, Loss w_fp: 0.008, Mask: 0.841 [2022-11-26 04:57:52,093][ INFO] Iters: 1240, Total loss: 0.052, Loss x: 0.054, Loss s: 0.090, Loss w_fp: 0.007, Mask: 0.842 [2022-11-26 05:00:19,320][ INFO] Iters: 1550, Total loss: 0.051, Loss x: 0.054, Loss s: 0.089, Loss w_fp: 0.007, Mask: 0.843 [2022-11-26 05:02:46,526][ INFO] Iters: 1860, Total loss: 0.051, Loss x: 0.054, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.843 [2022-11-26 05:05:13,866][ INFO] Iters: 2170, Total loss: 0.051, Loss x: 0.055, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.844 [2022-11-26 05:08:39,956][ INFO] Evaluation original >>>> meanIOU: 74.34

[2022-11-26 05:08:39,956][ INFO] ===========> Epoch: 37, LR: 0.0003, Previous best: 75.50 [2022-11-26 05:08:41,038][ INFO] Iters: 0, Total loss: 0.055, Loss x: 0.042, Loss s: 0.135, Loss w_fp: 0.002, Mask: 0.713 [2022-11-26 05:11:08,250][ INFO] Iters: 310, Total loss: 0.050, Loss x: 0.054, Loss s: 0.081, Loss w_fp: 0.008, Mask: 0.837 [2022-11-26 05:13:35,415][ INFO] Iters: 620, Total loss: 0.051, Loss x: 0.053, Loss s: 0.088, Loss w_fp: 0.008, Mask: 0.841 [2022-11-26 05:16:02,777][ INFO] Iters: 930, Total loss: 0.051, Loss x: 0.053, Loss s: 0.091, Loss w_fp: 0.008, Mask: 0.845 [2022-11-26 05:18:29,924][ INFO] Iters: 1240, Total loss: 0.051, Loss x: 0.053, Loss s: 0.091, Loss w_fp: 0.008, Mask: 0.847 [2022-11-26 05:20:57,195][ INFO] Iters: 1550, Total loss: 0.051, Loss x: 0.053, Loss s: 0.091, Loss w_fp: 0.008, Mask: 0.846 [2022-11-26 05:23:24,363][ INFO] Iters: 1860, Total loss: 0.052, Loss x: 0.054, Loss s: 0.091, Loss w_fp: 0.008, Mask: 0.844 [2022-11-26 05:25:51,538][ INFO] Iters: 2170, Total loss: 0.052, Loss x: 0.055, Loss s: 0.090, Loss w_fp: 0.008, Mask: 0.844 [2022-11-26 05:29:15,612][ INFO] Evaluation original >>>> meanIOU: 74.90

[2022-11-26 05:29:15,613][ INFO] ===========> Epoch: 38, LR: 0.0003, Previous best: 75.50 [2022-11-26 05:29:16,730][ INFO] Iters: 0, Total loss: 0.072, Loss x: 0.073, Loss s: 0.138, Loss w_fp: 0.003, Mask: 0.870 [2022-11-26 05:31:43,928][ INFO] Iters: 310, Total loss: 0.050, Loss x: 0.052, Loss s: 0.090, Loss w_fp: 0.007, Mask: 0.841 [2022-11-26 05:34:11,015][ INFO] Iters: 620, Total loss: 0.051, Loss x: 0.054, Loss s: 0.087, Loss w_fp: 0.007, Mask: 0.841 [2022-11-26 05:36:38,046][ INFO] Iters: 930, Total loss: 0.051, Loss x: 0.054, Loss s: 0.089, Loss w_fp: 0.007, Mask: 0.842 [2022-11-26 05:39:05,319][ INFO] Iters: 1240, Total loss: 0.051, Loss x: 0.053, Loss s: 0.091, Loss w_fp: 0.008, Mask: 0.843 [2022-11-26 05:41:32,492][ INFO] Iters: 1550, Total loss: 0.051, Loss x: 0.053, Loss s: 0.089, Loss w_fp: 0.007, Mask: 0.844 [2022-11-26 05:43:59,655][ INFO] Iters: 1860, Total loss: 0.051, Loss x: 0.054, Loss s: 0.089, Loss w_fp: 0.007, Mask: 0.846 [2022-11-26 05:46:26,853][ INFO] Iters: 2170, Total loss: 0.051, Loss x: 0.053, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.848 [2022-11-26 05:49:50,750][ INFO] Evaluation original >>>> meanIOU: 74.73

[2022-11-26 05:49:50,750][ INFO] ===========> Epoch: 39, LR: 0.0003, Previous best: 75.50 [2022-11-26 05:49:51,794][ INFO] Iters: 0, Total loss: 0.037, Loss x: 0.023, Loss s: 0.093, Loss w_fp: 0.010, Mask: 0.905 [2022-11-26 05:52:19,005][ INFO] Iters: 310, Total loss: 0.048, Loss x: 0.052, Loss s: 0.083, Loss w_fp: 0.007, Mask: 0.846 [2022-11-26 05:54:46,391][ INFO] Iters: 620, Total loss: 0.051, Loss x: 0.053, Loss s: 0.089, Loss w_fp: 0.008, Mask: 0.844 [2022-11-26 05:57:13,646][ INFO] Iters: 930, Total loss: 0.051, Loss x: 0.053, Loss s: 0.091, Loss w_fp: 0.008, Mask: 0.844 [2022-11-26 05:59:41,046][ INFO] Iters: 1240, Total loss: 0.051, Loss x: 0.054, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.849 [2022-11-26 06:02:08,181][ INFO] Iters: 1550, Total loss: 0.051, Loss x: 0.053, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.850 [2022-11-26 06:04:35,452][ INFO] Iters: 1860, Total loss: 0.051, Loss x: 0.053, Loss s: 0.089, Loss w_fp: 0.007, Mask: 0.849 [2022-11-26 06:07:02,763][ INFO] Iters: 2170, Total loss: 0.051, Loss x: 0.054, Loss s: 0.089, Loss w_fp: 0.007, Mask: 0.849 [2022-11-26 06:10:26,666][ INFO] Evaluation original >>>> meanIOU: 74.32

[2022-11-26 06:10:26,666][ INFO] ===========> Epoch: 40, LR: 0.0003, Previous best: 75.50 [2022-11-26 06:10:27,756][ INFO] Iters: 0, Total loss: 0.117, Loss x: 0.091, Loss s: 0.276, Loss w_fp: 0.011, Mask: 0.765 [2022-11-26 06:12:54,928][ INFO] Iters: 310, Total loss: 0.050, Loss x: 0.052, Loss s: 0.087, Loss w_fp: 0.009, Mask: 0.846 [2022-11-26 06:15:22,432][ INFO] Iters: 620, Total loss: 0.050, Loss x: 0.052, Loss s: 0.086, Loss w_fp: 0.009, Mask: 0.846 [2022-11-26 06:17:49,693][ INFO] Iters: 930, Total loss: 0.049, Loss x: 0.051, Loss s: 0.087, Loss w_fp: 0.008, Mask: 0.847 [2022-11-26 06:20:17,024][ INFO] Iters: 1240, Total loss: 0.050, Loss x: 0.052, Loss s: 0.090, Loss w_fp: 0.008, Mask: 0.847 [2022-11-26 06:22:44,320][ INFO] Iters: 1550, Total loss: 0.050, Loss x: 0.052, Loss s: 0.088, Loss w_fp: 0.008, Mask: 0.849 [2022-11-26 06:25:11,534][ INFO] Iters: 1860, Total loss: 0.050, Loss x: 0.052, Loss s: 0.089, Loss w_fp: 0.008, Mask: 0.848 [2022-11-26 06:27:38,850][ INFO] Iters: 2170, Total loss: 0.050, Loss x: 0.052, Loss s: 0.089, Loss w_fp: 0.008, Mask: 0.849 [2022-11-26 06:31:02,729][ INFO] Evaluation original >>>> meanIOU: 74.43

[2022-11-26 06:31:02,729][ INFO] ===========> Epoch: 41, LR: 0.0003, Previous best: 75.50 [2022-11-26 06:31:03,893][ INFO] Iters: 0, Total loss: 0.033, Loss x: 0.041, Loss s: 0.050, Loss w_fp: 0.001, Mask: 0.920 [2022-11-26 06:33:31,166][ INFO] Iters: 310, Total loss: 0.053, Loss x: 0.055, Loss s: 0.093, Loss w_fp: 0.008, Mask: 0.849 [2022-11-26 06:35:58,148][ INFO] Iters: 620, Total loss: 0.051, Loss x: 0.053, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.846 [2022-11-26 06:38:24,089][ INFO] Iters: 930, Total loss: 0.050, Loss x: 0.053, Loss s: 0.086, Loss w_fp: 0.007, Mask: 0.848 [2022-11-26 06:40:49,982][ INFO] Iters: 1240, Total loss: 0.049, Loss x: 0.053, Loss s: 0.086, Loss w_fp: 0.007, Mask: 0.850 [2022-11-26 06:43:15,828][ INFO] Iters: 1550, Total loss: 0.049, Loss x: 0.052, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.851 [2022-11-26 06:45:41,294][ INFO] Iters: 1860, Total loss: 0.050, Loss x: 0.053, Loss s: 0.086, Loss w_fp: 0.007, Mask: 0.850 [2022-11-26 06:48:06,862][ INFO] Iters: 2170, Total loss: 0.049, Loss x: 0.052, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.852 [2022-11-26 06:51:17,615][ INFO] Evaluation original >>>> meanIOU: 74.23

[2022-11-26 06:51:17,615][ INFO] ===========> Epoch: 42, LR: 0.0003, Previous best: 75.50 [2022-11-26 06:51:18,636][ INFO] Iters: 0, Total loss: 0.082, Loss x: 0.122, Loss s: 0.080, Loss w_fp: 0.003, Mask: 0.934 [2022-11-26 06:53:44,099][ INFO] Iters: 310, Total loss: 0.046, Loss x: 0.051, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.860 [2022-11-26 06:56:09,596][ INFO] Iters: 620, Total loss: 0.047, Loss x: 0.050, Loss s: 0.080, Loss w_fp: 0.006, Mask: 0.861 [2022-11-26 06:58:35,110][ INFO] Iters: 930, Total loss: 0.047, Loss x: 0.050, Loss s: 0.083, Loss w_fp: 0.007, Mask: 0.857 [2022-11-26 07:01:00,495][ INFO] Iters: 1240, Total loss: 0.048, Loss x: 0.050, Loss s: 0.083, Loss w_fp: 0.007, Mask: 0.855 [2022-11-26 07:03:25,899][ INFO] Iters: 1550, Total loss: 0.048, Loss x: 0.051, Loss s: 0.084, Loss w_fp: 0.007, Mask: 0.857 [2022-11-26 07:05:51,458][ INFO] Iters: 1860, Total loss: 0.048, Loss x: 0.051, Loss s: 0.084, Loss w_fp: 0.007, Mask: 0.857 [2022-11-26 07:08:16,978][ INFO] Iters: 2170, Total loss: 0.048, Loss x: 0.051, Loss s: 0.083, Loss w_fp: 0.007, Mask: 0.857 [2022-11-26 07:11:28,024][ INFO] Evaluation original >>>> meanIOU: 73.96

[2022-11-26 07:11:28,024][ INFO] ===========> Epoch: 43, LR: 0.0002, Previous best: 75.50 [2022-11-26 07:11:28,965][ INFO] Iters: 0, Total loss: 0.066, Loss x: 0.053, Loss s: 0.147, Loss w_fp: 0.008, Mask: 0.662 [2022-11-26 07:13:54,466][ INFO] Iters: 310, Total loss: 0.050, Loss x: 0.052, Loss s: 0.087, Loss w_fp: 0.008, Mask: 0.853 [2022-11-26 07:16:19,959][ INFO] Iters: 620, Total loss: 0.049, Loss x: 0.052, Loss s: 0.083, Loss w_fp: 0.007, Mask: 0.855 [2022-11-26 07:18:45,415][ INFO] Iters: 930, Total loss: 0.049, Loss x: 0.052, Loss s: 0.083, Loss w_fp: 0.007, Mask: 0.856 [2022-11-26 07:21:10,911][ INFO] Iters: 1240, Total loss: 0.049, Loss x: 0.051, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.856 [2022-11-26 07:23:36,492][ INFO] Iters: 1550, Total loss: 0.049, Loss x: 0.052, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.856 [2022-11-26 07:26:02,107][ INFO] Iters: 1860, Total loss: 0.049, Loss x: 0.052, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.854 [2022-11-26 07:28:27,663][ INFO] Iters: 2170, Total loss: 0.049, Loss x: 0.052, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.853 [2022-11-26 07:31:38,971][ INFO] Evaluation original >>>> meanIOU: 73.28

[2022-11-26 07:31:38,971][ INFO] ===========> Epoch: 44, LR: 0.0002, Previous best: 75.50 [2022-11-26 07:31:39,927][ INFO] Iters: 0, Total loss: 0.028, Loss x: 0.043, Loss s: 0.023, Loss w_fp: 0.003, Mask: 0.564 [2022-11-26 07:34:05,276][ INFO] Iters: 310, Total loss: 0.051, Loss x: 0.052, Loss s: 0.092, Loss w_fp: 0.007, Mask: 0.847 [2022-11-26 07:36:30,847][ INFO] Iters: 620, Total loss: 0.050, Loss x: 0.051, Loss s: 0.092, Loss w_fp: 0.007, Mask: 0.848 [2022-11-26 07:38:56,425][ INFO] Iters: 930, Total loss: 0.049, Loss x: 0.051, Loss s: 0.087, Loss w_fp: 0.007, Mask: 0.852 [2022-11-26 07:41:21,879][ INFO] Iters: 1240, Total loss: 0.048, Loss x: 0.051, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.852 [2022-11-26 07:43:47,336][ INFO] Iters: 1550, Total loss: 0.049, Loss x: 0.050, Loss s: 0.087, Loss w_fp: 0.007, Mask: 0.852 [2022-11-26 07:46:12,760][ INFO] Iters: 1860, Total loss: 0.049, Loss x: 0.051, Loss s: 0.087, Loss w_fp: 0.007, Mask: 0.853 [2022-11-26 07:48:38,283][ INFO] Iters: 2170, Total loss: 0.049, Loss x: 0.051, Loss s: 0.086, Loss w_fp: 0.007, Mask: 0.853 [2022-11-26 07:51:50,323][ INFO] Evaluation original >>>> meanIOU: 73.96

[2022-11-26 07:51:50,323][ INFO] ===========> Epoch: 45, LR: 0.0002, Previous best: 75.50 [2022-11-26 07:51:51,311][ INFO] Iters: 0, Total loss: 0.048, Loss x: 0.041, Loss s: 0.105, Loss w_fp: 0.005, Mask: 0.853 [2022-11-26 07:54:16,848][ INFO] Iters: 310, Total loss: 0.049, Loss x: 0.051, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.856 [2022-11-26 07:56:42,407][ INFO] Iters: 620, Total loss: 0.048, Loss x: 0.050, Loss s: 0.087, Loss w_fp: 0.007, Mask: 0.860 [2022-11-26 07:59:07,930][ INFO] Iters: 930, Total loss: 0.049, Loss x: 0.050, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.860 [2022-11-26 08:01:33,449][ INFO] Iters: 1240, Total loss: 0.049, Loss x: 0.050, Loss s: 0.087, Loss w_fp: 0.007, Mask: 0.860 [2022-11-26 08:03:59,016][ INFO] Iters: 1550, Total loss: 0.049, Loss x: 0.050, Loss s: 0.087, Loss w_fp: 0.007, Mask: 0.859 [2022-11-26 08:06:24,576][ INFO] Iters: 1860, Total loss: 0.049, Loss x: 0.050, Loss s: 0.087, Loss w_fp: 0.008, Mask: 0.858 [2022-11-26 08:08:50,084][ INFO] Iters: 2170, Total loss: 0.048, Loss x: 0.050, Loss s: 0.086, Loss w_fp: 0.007, Mask: 0.858 [2022-11-26 08:12:01,618][ INFO] Evaluation original >>>> meanIOU: 73.92

[2022-11-26 08:12:01,619][ INFO] ===========> Epoch: 46, LR: 0.0002, Previous best: 75.50 [2022-11-26 08:12:02,567][ INFO] Iters: 0, Total loss: 0.041, Loss x: 0.043, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.894 [2022-11-26 08:14:28,025][ INFO] Iters: 310, Total loss: 0.048, Loss x: 0.049, Loss s: 0.087, Loss w_fp: 0.007, Mask: 0.861 [2022-11-26 08:16:53,576][ INFO] Iters: 620, Total loss: 0.048, Loss x: 0.049, Loss s: 0.086, Loss w_fp: 0.007, Mask: 0.859 [2022-11-26 08:19:19,101][ INFO] Iters: 930, Total loss: 0.048, Loss x: 0.049, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.859 [2022-11-26 08:21:44,638][ INFO] Iters: 1240, Total loss: 0.047, Loss x: 0.049, Loss s: 0.083, Loss w_fp: 0.007, Mask: 0.859 [2022-11-26 08:24:10,203][ INFO] Iters: 1550, Total loss: 0.047, Loss x: 0.049, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.859 [2022-11-26 08:26:35,756][ INFO] Iters: 1860, Total loss: 0.047, Loss x: 0.049, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.860 [2022-11-26 08:29:01,371][ INFO] Iters: 2170, Total loss: 0.047, Loss x: 0.050, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.858 [2022-11-26 08:32:11,983][ INFO] Evaluation original >>>> meanIOU: 75.14

[2022-11-26 08:32:11,983][ INFO] ===========> Epoch: 47, LR: 0.0002, Previous best: 75.50 [2022-11-26 08:32:12,975][ INFO] Iters: 0, Total loss: 0.058, Loss x: 0.101, Loss s: 0.024, Loss w_fp: 0.004, Mask: 0.937 [2022-11-26 08:34:38,364][ INFO] Iters: 310, Total loss: 0.047, Loss x: 0.050, Loss s: 0.080, Loss w_fp: 0.006, Mask: 0.860 [2022-11-26 08:37:03,967][ INFO] Iters: 620, Total loss: 0.048, Loss x: 0.050, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.859 [2022-11-26 08:39:29,659][ INFO] Iters: 930, Total loss: 0.047, Loss x: 0.050, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.859 [2022-11-26 08:41:55,223][ INFO] Iters: 1240, Total loss: 0.047, Loss x: 0.050, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.860 [2022-11-26 08:44:20,985][ INFO] Iters: 1550, Total loss: 0.047, Loss x: 0.049, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.860 [2022-11-26 08:46:46,526][ INFO] Iters: 1860, Total loss: 0.048, Loss x: 0.050, Loss s: 0.083, Loss w_fp: 0.007, Mask: 0.860 [2022-11-26 08:49:12,020][ INFO] Iters: 2170, Total loss: 0.047, Loss x: 0.050, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.858 [2022-11-26 08:52:22,672][ INFO] Evaluation original >>>> meanIOU: 74.65

[2022-11-26 08:52:22,672][ INFO] ===========> Epoch: 48, LR: 0.0002, Previous best: 75.50 [2022-11-26 08:52:23,697][ INFO] Iters: 0, Total loss: 0.048, Loss x: 0.050, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.909 [2022-11-26 08:54:49,490][ INFO] Iters: 310, Total loss: 0.046, Loss x: 0.050, Loss s: 0.078, Loss w_fp: 0.006, Mask: 0.864 [2022-11-26 08:57:15,084][ INFO] Iters: 620, Total loss: 0.046, Loss x: 0.050, Loss s: 0.076, Loss w_fp: 0.007, Mask: 0.863 [2022-11-26 08:59:40,743][ INFO] Iters: 930, Total loss: 0.048, Loss x: 0.052, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.857 [2022-11-26 09:02:06,297][ INFO] Iters: 1240, Total loss: 0.049, Loss x: 0.051, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.855 [2022-11-26 09:04:32,002][ INFO] Iters: 1550, Total loss: 0.048, Loss x: 0.051, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.854 [2022-11-26 09:06:57,657][ INFO] Iters: 1860, Total loss: 0.048, Loss x: 0.050, Loss s: 0.086, Loss w_fp: 0.007, Mask: 0.854 [2022-11-26 09:09:23,299][ INFO] Iters: 2170, Total loss: 0.049, Loss x: 0.050, Loss s: 0.086, Loss w_fp: 0.007, Mask: 0.855 [2022-11-26 09:12:35,224][ INFO] Evaluation original >>>> meanIOU: 74.26

[2022-11-26 09:12:35,225][ INFO] ===========> Epoch: 49, LR: 0.0002, Previous best: 75.50 [2022-11-26 09:12:36,191][ INFO] Iters: 0, Total loss: 0.043, Loss x: 0.026, Loss s: 0.118, Loss w_fp: 0.003, Mask: 0.927 [2022-11-26 09:15:01,760][ INFO] Iters: 310, Total loss: 0.046, Loss x: 0.049, Loss s: 0.081, Loss w_fp: 0.006, Mask: 0.862 [2022-11-26 09:17:27,337][ INFO] Iters: 620, Total loss: 0.047, Loss x: 0.049, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.861 [2022-11-26 09:19:52,931][ INFO] Iters: 930, Total loss: 0.047, Loss x: 0.049, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.861 [2022-11-26 09:22:18,609][ INFO] Iters: 1240, Total loss: 0.047, Loss x: 0.049, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.861 [2022-11-26 09:24:44,144][ INFO] Iters: 1550, Total loss: 0.047, Loss x: 0.049, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.861 [2022-11-26 09:27:09,714][ INFO] Iters: 1860, Total loss: 0.047, Loss x: 0.049, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.861 [2022-11-26 09:29:35,470][ INFO] Iters: 2170, Total loss: 0.047, Loss x: 0.049, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.863 [2022-11-26 09:32:47,801][ INFO] Evaluation original >>>> meanIOU: 74.28

[2022-11-26 09:32:47,801][ INFO] ===========> Epoch: 50, LR: 0.0002, Previous best: 75.50 [2022-11-26 09:32:48,807][ INFO] Iters: 0, Total loss: 0.040, Loss x: 0.063, Loss s: 0.032, Loss w_fp: 0.001, Mask: 0.884 [2022-11-26 09:35:14,531][ INFO] Iters: 310, Total loss: 0.046, Loss x: 0.049, Loss s: 0.079, Loss w_fp: 0.007, Mask: 0.868 [2022-11-26 09:37:40,257][ INFO] Iters: 620, Total loss: 0.046, Loss x: 0.048, Loss s: 0.083, Loss w_fp: 0.007, Mask: 0.862 [2022-11-26 09:40:06,319][ INFO] Iters: 930, Total loss: 0.046, Loss x: 0.047, Loss s: 0.083, Loss w_fp: 0.007, Mask: 0.861 [2022-11-26 09:42:32,331][ INFO] Iters: 1240, Total loss: 0.047, Loss x: 0.048, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.860 [2022-11-26 09:44:58,249][ INFO] Iters: 1550, Total loss: 0.047, Loss x: 0.048, Loss s: 0.085, Loss w_fp: 0.007, Mask: 0.860 [2022-11-26 09:47:24,310][ INFO] Iters: 1860, Total loss: 0.046, Loss x: 0.047, Loss s: 0.084, Loss w_fp: 0.007, Mask: 0.860 [2022-11-26 09:49:50,410][ INFO] Iters: 2170, Total loss: 0.047, Loss x: 0.047, Loss s: 0.086, Loss w_fp: 0.007, Mask: 0.860 [2022-11-26 09:53:03,597][ INFO] Evaluation original >>>> meanIOU: 74.61

[2022-11-26 09:53:03,597][ INFO] ===========> Epoch: 51, LR: 0.0002, Previous best: 75.50 [2022-11-26 09:53:04,569][ INFO] Iters: 0, Total loss: 0.028, Loss x: 0.046, Loss s: 0.016, Loss w_fp: 0.003, Mask: 0.708 [2022-11-26 09:55:30,218][ INFO] Iters: 310, Total loss: 0.048, Loss x: 0.049, Loss s: 0.087, Loss w_fp: 0.007, Mask: 0.858 [2022-11-26 09:57:56,160][ INFO] Iters: 620, Total loss: 0.049, Loss x: 0.050, Loss s: 0.088, Loss w_fp: 0.007, Mask: 0.855 [2022-11-26 10:00:21,889][ INFO] Iters: 930, Total loss: 0.048, Loss x: 0.049, Loss s: 0.087, Loss w_fp: 0.007, Mask: 0.857 [2022-11-26 10:02:47,769][ INFO] Iters: 1240, Total loss: 0.047, Loss x: 0.048, Loss s: 0.084, Loss w_fp: 0.007, Mask: 0.857 [2022-11-26 10:05:13,434][ INFO] Iters: 1550, Total loss: 0.046, Loss x: 0.048, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.859 [2022-11-26 10:07:39,109][ INFO] Iters: 1860, Total loss: 0.046, Loss x: 0.047, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.861 [2022-11-26 10:10:04,923][ INFO] Iters: 2170, Total loss: 0.046, Loss x: 0.048, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.861 [2022-11-26 10:13:17,711][ INFO] Evaluation original >>>> meanIOU: 74.48

[2022-11-26 10:13:17,711][ INFO] ===========> Epoch: 52, LR: 0.0002, Previous best: 75.50 [2022-11-26 10:13:18,711][ INFO] Iters: 0, Total loss: 0.050, Loss x: 0.068, Loss s: 0.056, Loss w_fp: 0.008, Mask: 0.751 [2022-11-26 10:15:44,407][ INFO] Iters: 310, Total loss: 0.045, Loss x: 0.048, Loss s: 0.077, Loss w_fp: 0.006, Mask: 0.869 [2022-11-26 10:18:10,372][ INFO] Iters: 620, Total loss: 0.046, Loss x: 0.048, Loss s: 0.081, Loss w_fp: 0.006, Mask: 0.862 [2022-11-26 10:20:36,209][ INFO] Iters: 930, Total loss: 0.046, Loss x: 0.048, Loss s: 0.082, Loss w_fp: 0.006, Mask: 0.862 [2022-11-26 10:23:01,994][ INFO] Iters: 1240, Total loss: 0.046, Loss x: 0.048, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.861 [2022-11-26 10:25:27,779][ INFO] Iters: 1550, Total loss: 0.046, Loss x: 0.048, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.861 [2022-11-26 10:27:53,873][ INFO] Iters: 1860, Total loss: 0.047, Loss x: 0.049, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.862 [2022-11-26 10:30:19,787][ INFO] Iters: 2170, Total loss: 0.046, Loss x: 0.049, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.863 [2022-11-26 10:33:32,393][ INFO] Evaluation original >>>> meanIOU: 74.49

[2022-11-26 10:33:32,393][ INFO] ===========> Epoch: 53, LR: 0.0002, Previous best: 75.50 [2022-11-26 10:33:33,332][ INFO] Iters: 0, Total loss: 0.026, Loss x: 0.028, Loss s: 0.044, Loss w_fp: 0.003, Mask: 0.903 [2022-11-26 10:35:58,714][ INFO] Iters: 310, Total loss: 0.045, Loss x: 0.048, Loss s: 0.078, Loss w_fp: 0.006, Mask: 0.869 [2022-11-26 10:38:24,337][ INFO] Iters: 620, Total loss: 0.045, Loss x: 0.047, Loss s: 0.080, Loss w_fp: 0.006, Mask: 0.868 [2022-11-26 10:40:50,150][ INFO] Iters: 930, Total loss: 0.045, Loss x: 0.046, Loss s: 0.081, Loss w_fp: 0.006, Mask: 0.865 [2022-11-26 10:43:16,044][ INFO] Iters: 1240, Total loss: 0.045, Loss x: 0.047, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.865 [2022-11-26 10:45:41,914][ INFO] Iters: 1550, Total loss: 0.045, Loss x: 0.047, Loss s: 0.080, Loss w_fp: 0.007, Mask: 0.865 [2022-11-26 10:48:07,904][ INFO] Iters: 1860, Total loss: 0.045, Loss x: 0.047, Loss s: 0.079, Loss w_fp: 0.007, Mask: 0.864 [2022-11-26 10:50:33,652][ INFO] Iters: 2170, Total loss: 0.045, Loss x: 0.047, Loss s: 0.079, Loss w_fp: 0.007, Mask: 0.864 [2022-11-26 10:53:45,375][ INFO] Evaluation original >>>> meanIOU: 74.35

[2022-11-26 10:53:45,375][ INFO] ===========> Epoch: 54, LR: 0.0002, Previous best: 75.50 [2022-11-26 10:53:46,323][ INFO] Iters: 0, Total loss: 0.032, Loss x: 0.040, Loss s: 0.045, Loss w_fp: 0.002, Mask: 0.881 [2022-11-26 10:56:11,853][ INFO] Iters: 310, Total loss: 0.045, Loss x: 0.048, Loss s: 0.077, Loss w_fp: 0.007, Mask: 0.862 [2022-11-26 10:58:37,966][ INFO] Iters: 620, Total loss: 0.045, Loss x: 0.047, Loss s: 0.079, Loss w_fp: 0.007, Mask: 0.862 [2022-11-26 11:01:04,017][ INFO] Iters: 930, Total loss: 0.045, Loss x: 0.047, Loss s: 0.080, Loss w_fp: 0.007, Mask: 0.865 [2022-11-26 11:03:30,321][ INFO] Iters: 1240, Total loss: 0.044, Loss x: 0.046, Loss s: 0.078, Loss w_fp: 0.007, Mask: 0.866 [2022-11-26 11:05:56,510][ INFO] Iters: 1550, Total loss: 0.044, Loss x: 0.046, Loss s: 0.078, Loss w_fp: 0.007, Mask: 0.867 [2022-11-26 11:08:22,646][ INFO] Iters: 1860, Total loss: 0.045, Loss x: 0.047, Loss s: 0.080, Loss w_fp: 0.007, Mask: 0.865 [2022-11-26 11:10:48,504][ INFO] Iters: 2170, Total loss: 0.045, Loss x: 0.047, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.864 [2022-11-26 11:14:01,102][ INFO] Evaluation original >>>> meanIOU: 74.29

[2022-11-26 11:14:01,103][ INFO] ===========> Epoch: 55, LR: 0.0002, Previous best: 75.50 [2022-11-26 11:14:02,100][ INFO] Iters: 0, Total loss: 0.025, Loss x: 0.034, Loss s: 0.030, Loss w_fp: 0.001, Mask: 0.925 [2022-11-26 11:16:27,599][ INFO] Iters: 310, Total loss: 0.045, Loss x: 0.048, Loss s: 0.077, Loss w_fp: 0.006, Mask: 0.861 [2022-11-26 11:18:53,285][ INFO] Iters: 620, Total loss: 0.045, Loss x: 0.048, Loss s: 0.076, Loss w_fp: 0.006, Mask: 0.861 [2022-11-26 11:21:18,824][ INFO] Iters: 930, Total loss: 0.045, Loss x: 0.047, Loss s: 0.078, Loss w_fp: 0.007, Mask: 0.862 [2022-11-26 11:23:44,518][ INFO] Iters: 1240, Total loss: 0.044, Loss x: 0.047, Loss s: 0.076, Loss w_fp: 0.007, Mask: 0.863 [2022-11-26 11:26:10,227][ INFO] Iters: 1550, Total loss: 0.045, Loss x: 0.047, Loss s: 0.079, Loss w_fp: 0.007, Mask: 0.862 [2022-11-26 11:28:35,855][ INFO] Iters: 1860, Total loss: 0.045, Loss x: 0.047, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.863 [2022-11-26 11:31:01,421][ INFO] Iters: 2170, Total loss: 0.045, Loss x: 0.047, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.864 [2022-11-26 11:34:13,650][ INFO] Evaluation original >>>> meanIOU: 74.33

[2022-11-26 11:34:13,650][ INFO] ===========> Epoch: 56, LR: 0.0002, Previous best: 75.50 [2022-11-26 11:34:14,687][ INFO] Iters: 0, Total loss: 0.035, Loss x: 0.037, Loss s: 0.061, Loss w_fp: 0.005, Mask: 0.893 [2022-11-26 11:36:40,506][ INFO] Iters: 310, Total loss: 0.046, Loss x: 0.048, Loss s: 0.083, Loss w_fp: 0.006, Mask: 0.865 [2022-11-26 11:39:06,332][ INFO] Iters: 620, Total loss: 0.045, Loss x: 0.047, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.865 [2022-11-26 11:41:32,073][ INFO] Iters: 930, Total loss: 0.045, Loss x: 0.046, Loss s: 0.083, Loss w_fp: 0.006, Mask: 0.866 [2022-11-26 11:43:57,916][ INFO] Iters: 1240, Total loss: 0.045, Loss x: 0.046, Loss s: 0.082, Loss w_fp: 0.007, Mask: 0.867 [2022-11-26 11:46:23,908][ INFO] Iters: 1550, Total loss: 0.045, Loss x: 0.046, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.867 [2022-11-26 11:48:49,777][ INFO] Iters: 1860, Total loss: 0.045, Loss x: 0.046, Loss s: 0.080, Loss w_fp: 0.007, Mask: 0.867 [2022-11-26 11:51:15,520][ INFO] Iters: 2170, Total loss: 0.045, Loss x: 0.046, Loss s: 0.080, Loss w_fp: 0.007, Mask: 0.868 [2022-11-26 11:54:27,968][ INFO] Evaluation original >>>> meanIOU: 73.50

[2022-11-26 11:54:27,968][ INFO] ===========> Epoch: 57, LR: 0.0002, Previous best: 75.50 [2022-11-26 11:54:28,900][ INFO] Iters: 0, Total loss: 0.038, Loss x: 0.045, Loss s: 0.058, Loss w_fp: 0.002, Mask: 0.883 [2022-11-26 11:56:54,730][ INFO] Iters: 310, Total loss: 0.044, Loss x: 0.046, Loss s: 0.078, Loss w_fp: 0.005, Mask: 0.863 [2022-11-26 11:59:20,621][ INFO] Iters: 620, Total loss: 0.045, Loss x: 0.046, Loss s: 0.081, Loss w_fp: 0.006, Mask: 0.859 [2022-11-26 12:01:46,475][ INFO] Iters: 930, Total loss: 0.045, Loss x: 0.046, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.860 [2022-11-26 12:04:12,280][ INFO] Iters: 1240, Total loss: 0.044, Loss x: 0.046, Loss s: 0.080, Loss w_fp: 0.006, Mask: 0.863 [2022-11-26 12:06:38,034][ INFO] Iters: 1550, Total loss: 0.045, Loss x: 0.046, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.865 [2022-11-26 12:09:03,909][ INFO] Iters: 1860, Total loss: 0.044, Loss x: 0.046, Loss s: 0.080, Loss w_fp: 0.007, Mask: 0.865 [2022-11-26 12:11:30,059][ INFO] Iters: 2170, Total loss: 0.044, Loss x: 0.046, Loss s: 0.079, Loss w_fp: 0.007, Mask: 0.865 [2022-11-26 12:14:42,647][ INFO] Evaluation original >>>> meanIOU: 74.24

[2022-11-26 12:14:42,647][ INFO] ===========> Epoch: 58, LR: 0.0002, Previous best: 75.50 [2022-11-26 12:14:43,629][ INFO] Iters: 0, Total loss: 0.038, Loss x: 0.054, Loss s: 0.042, Loss w_fp: 0.003, Mask: 0.819 [2022-11-26 12:17:09,267][ INFO] Iters: 310, Total loss: 0.043, Loss x: 0.045, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.869 [2022-11-26 12:19:35,127][ INFO] Iters: 620, Total loss: 0.043, Loss x: 0.046, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.869 [2022-11-26 12:22:01,166][ INFO] Iters: 930, Total loss: 0.044, Loss x: 0.046, Loss s: 0.077, Loss w_fp: 0.006, Mask: 0.870 [2022-11-26 12:24:27,059][ INFO] Iters: 1240, Total loss: 0.044, Loss x: 0.046, Loss s: 0.077, Loss w_fp: 0.006, Mask: 0.871 [2022-11-26 12:26:53,013][ INFO] Iters: 1550, Total loss: 0.044, Loss x: 0.046, Loss s: 0.077, Loss w_fp: 0.006, Mask: 0.871 [2022-11-26 12:29:19,101][ INFO] Iters: 1860, Total loss: 0.044, Loss x: 0.046, Loss s: 0.077, Loss w_fp: 0.006, Mask: 0.871 [2022-11-26 12:31:45,082][ INFO] Iters: 2170, Total loss: 0.044, Loss x: 0.046, Loss s: 0.078, Loss w_fp: 0.007, Mask: 0.870 [2022-11-26 12:34:56,832][ INFO] Evaluation original >>>> meanIOU: 75.20

[2022-11-26 12:34:56,832][ INFO] ===========> Epoch: 59, LR: 0.0002, Previous best: 75.50 [2022-11-26 12:34:57,782][ INFO] Iters: 0, Total loss: 0.026, Loss x: 0.026, Loss s: 0.041, Loss w_fp: 0.012, Mask: 0.897 [2022-11-26 12:37:23,437][ INFO] Iters: 310, Total loss: 0.042, Loss x: 0.043, Loss s: 0.077, Loss w_fp: 0.007, Mask: 0.874 [2022-11-26 12:39:49,310][ INFO] Iters: 620, Total loss: 0.043, Loss x: 0.044, Loss s: 0.077, Loss w_fp: 0.007, Mask: 0.872 [2022-11-26 12:42:15,185][ INFO] Iters: 930, Total loss: 0.043, Loss x: 0.045, Loss s: 0.077, Loss w_fp: 0.006, Mask: 0.872 [2022-11-26 12:44:41,006][ INFO] Iters: 1240, Total loss: 0.043, Loss x: 0.044, Loss s: 0.078, Loss w_fp: 0.006, Mask: 0.871 [2022-11-26 12:47:06,909][ INFO] Iters: 1550, Total loss: 0.043, Loss x: 0.045, Loss s: 0.078, Loss w_fp: 0.006, Mask: 0.870 [2022-11-26 12:49:32,478][ INFO] Iters: 1860, Total loss: 0.044, Loss x: 0.045, Loss s: 0.079, Loss w_fp: 0.007, Mask: 0.870 [2022-11-26 12:51:57,936][ INFO] Iters: 2170, Total loss: 0.044, Loss x: 0.045, Loss s: 0.079, Loss w_fp: 0.007, Mask: 0.869 [2022-11-26 12:55:09,624][ INFO] Evaluation original >>>> meanIOU: 75.18

[2022-11-26 12:55:09,624][ INFO] ===========> Epoch: 60, LR: 0.0001, Previous best: 75.50 [2022-11-26 12:55:10,612][ INFO] Iters: 0, Total loss: 0.045, Loss x: 0.033, Loss s: 0.108, Loss w_fp: 0.006, Mask: 0.899 [2022-11-26 12:57:36,220][ INFO] Iters: 310, Total loss: 0.042, Loss x: 0.044, Loss s: 0.072, Loss w_fp: 0.006, Mask: 0.862 [2022-11-26 13:00:02,062][ INFO] Iters: 620, Total loss: 0.042, Loss x: 0.045, Loss s: 0.072, Loss w_fp: 0.006, Mask: 0.866 [2022-11-26 13:02:27,916][ INFO] Iters: 930, Total loss: 0.042, Loss x: 0.045, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.866 [2022-11-26 13:04:53,793][ INFO] Iters: 1240, Total loss: 0.042, Loss x: 0.044, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.868 [2022-11-26 13:07:19,528][ INFO] Iters: 1550, Total loss: 0.043, Loss x: 0.045, Loss s: 0.074, Loss w_fp: 0.007, Mask: 0.868 [2022-11-26 13:09:45,282][ INFO] Iters: 1860, Total loss: 0.043, Loss x: 0.045, Loss s: 0.074, Loss w_fp: 0.007, Mask: 0.869 [2022-11-26 13:12:11,231][ INFO] Iters: 2170, Total loss: 0.042, Loss x: 0.045, Loss s: 0.074, Loss w_fp: 0.007, Mask: 0.870 [2022-11-26 13:15:22,930][ INFO] Evaluation original >>>> meanIOU: 73.15

[2022-11-26 13:15:22,931][ INFO] ===========> Epoch: 61, LR: 0.0001, Previous best: 75.50 [2022-11-26 13:15:23,886][ INFO] Iters: 0, Total loss: 0.044, Loss x: 0.063, Loss s: 0.046, Loss w_fp: 0.004, Mask: 0.843 [2022-11-26 13:17:49,647][ INFO] Iters: 310, Total loss: 0.046, Loss x: 0.046, Loss s: 0.087, Loss w_fp: 0.007, Mask: 0.869 [2022-11-26 13:20:15,348][ INFO] Iters: 620, Total loss: 0.045, Loss x: 0.046, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.868 [2022-11-26 13:22:41,402][ INFO] Iters: 930, Total loss: 0.045, Loss x: 0.046, Loss s: 0.080, Loss w_fp: 0.007, Mask: 0.867 [2022-11-26 13:25:06,971][ INFO] Iters: 1240, Total loss: 0.044, Loss x: 0.046, Loss s: 0.078, Loss w_fp: 0.006, Mask: 0.868 [2022-11-26 13:27:33,111][ INFO] Iters: 1550, Total loss: 0.044, Loss x: 0.045, Loss s: 0.079, Loss w_fp: 0.006, Mask: 0.869 [2022-11-26 13:29:58,903][ INFO] Iters: 1860, Total loss: 0.043, Loss x: 0.045, Loss s: 0.077, Loss w_fp: 0.007, Mask: 0.870 [2022-11-26 13:32:24,713][ INFO] Iters: 2170, Total loss: 0.043, Loss x: 0.045, Loss s: 0.077, Loss w_fp: 0.007, Mask: 0.871 [2022-11-26 13:35:36,523][ INFO] Evaluation original >>>> meanIOU: 74.47

[2022-11-26 13:35:36,523][ INFO] ===========> Epoch: 62, LR: 0.0001, Previous best: 75.50 [2022-11-26 13:35:37,471][ INFO] Iters: 0, Total loss: 0.014, Loss x: 0.004, Loss s: 0.034, Loss w_fp: 0.013, Mask: 0.875 [2022-11-26 13:38:03,005][ INFO] Iters: 310, Total loss: 0.043, Loss x: 0.046, Loss s: 0.074, Loss w_fp: 0.007, Mask: 0.869 [2022-11-26 13:40:28,712][ INFO] Iters: 620, Total loss: 0.044, Loss x: 0.045, Loss s: 0.080, Loss w_fp: 0.007, Mask: 0.865 [2022-11-26 13:42:54,664][ INFO] Iters: 930, Total loss: 0.044, Loss x: 0.045, Loss s: 0.079, Loss w_fp: 0.007, Mask: 0.870 [2022-11-26 13:45:20,332][ INFO] Iters: 1240, Total loss: 0.043, Loss x: 0.044, Loss s: 0.077, Loss w_fp: 0.007, Mask: 0.871 [2022-11-26 13:47:46,133][ INFO] Iters: 1550, Total loss: 0.043, Loss x: 0.044, Loss s: 0.078, Loss w_fp: 0.007, Mask: 0.870 [2022-11-26 13:50:11,919][ INFO] Iters: 1860, Total loss: 0.043, Loss x: 0.044, Loss s: 0.078, Loss w_fp: 0.007, Mask: 0.871 [2022-11-26 13:52:37,632][ INFO] Iters: 2170, Total loss: 0.043, Loss x: 0.044, Loss s: 0.078, Loss w_fp: 0.007, Mask: 0.870 [2022-11-26 13:55:49,728][ INFO] Evaluation original >>>> meanIOU: 74.41

[2022-11-26 13:55:49,728][ INFO] ===========> Epoch: 63, LR: 0.0001, Previous best: 75.50 [2022-11-26 13:55:50,670][ INFO] Iters: 0, Total loss: 0.029, Loss x: 0.051, Loss s: 0.015, Loss w_fp: 0.001, Mask: 0.958 [2022-11-26 13:58:16,224][ INFO] Iters: 310, Total loss: 0.042, Loss x: 0.044, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.876 [2022-11-26 14:00:42,107][ INFO] Iters: 620, Total loss: 0.043, Loss x: 0.045, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.873 [2022-11-26 14:03:07,846][ INFO] Iters: 930, Total loss: 0.043, Loss x: 0.045, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.871 [2022-11-26 14:05:33,621][ INFO] Iters: 1240, Total loss: 0.043, Loss x: 0.045, Loss s: 0.076, Loss w_fp: 0.006, Mask: 0.869 [2022-11-26 14:07:59,267][ INFO] Iters: 1550, Total loss: 0.043, Loss x: 0.045, Loss s: 0.076, Loss w_fp: 0.007, Mask: 0.870 [2022-11-26 14:10:24,949][ INFO] Iters: 1860, Total loss: 0.043, Loss x: 0.044, Loss s: 0.076, Loss w_fp: 0.007, Mask: 0.871 [2022-11-26 14:12:50,610][ INFO] Iters: 2170, Total loss: 0.043, Loss x: 0.045, Loss s: 0.075, Loss w_fp: 0.007, Mask: 0.870 [2022-11-26 14:16:01,629][ INFO] Evaluation original >>>> meanIOU: 74.38

[2022-11-26 14:16:01,630][ INFO] ===========> Epoch: 64, LR: 0.0001, Previous best: 75.50 [2022-11-26 14:16:02,649][ INFO] Iters: 0, Total loss: 0.043, Loss x: 0.064, Loss s: 0.031, Loss w_fp: 0.011, Mask: 0.900 [2022-11-26 14:18:28,307][ INFO] Iters: 310, Total loss: 0.042, Loss x: 0.043, Loss s: 0.074, Loss w_fp: 0.007, Mask: 0.869 [2022-11-26 14:20:54,204][ INFO] Iters: 620, Total loss: 0.041, Loss x: 0.043, Loss s: 0.071, Loss w_fp: 0.007, Mask: 0.874 [2022-11-26 14:23:20,072][ INFO] Iters: 930, Total loss: 0.041, Loss x: 0.044, Loss s: 0.072, Loss w_fp: 0.006, Mask: 0.873 [2022-11-26 14:25:45,909][ INFO] Iters: 1240, Total loss: 0.042, Loss x: 0.044, Loss s: 0.072, Loss w_fp: 0.006, Mask: 0.872 [2022-11-26 14:28:11,749][ INFO] Iters: 1550, Total loss: 0.042, Loss x: 0.043, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.873 [2022-11-26 14:30:37,476][ INFO] Iters: 1860, Total loss: 0.042, Loss x: 0.044, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.872 [2022-11-26 14:33:03,438][ INFO] Iters: 2170, Total loss: 0.042, Loss x: 0.043, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.873 [2022-11-26 14:36:15,417][ INFO] Evaluation original >>>> meanIOU: 74.00

[2022-11-26 14:36:15,418][ INFO] ===========> Epoch: 65, LR: 0.0001, Previous best: 75.50 [2022-11-26 14:36:16,383][ INFO] Iters: 0, Total loss: 0.120, Loss x: 0.051, Loss s: 0.358, Loss w_fp: 0.017, Mask: 0.739 [2022-11-26 14:38:41,883][ INFO] Iters: 310, Total loss: 0.045, Loss x: 0.045, Loss s: 0.083, Loss w_fp: 0.007, Mask: 0.867 [2022-11-26 14:41:07,715][ INFO] Iters: 620, Total loss: 0.044, Loss x: 0.044, Loss s: 0.081, Loss w_fp: 0.007, Mask: 0.869 [2022-11-26 14:43:33,642][ INFO] Iters: 930, Total loss: 0.043, Loss x: 0.044, Loss s: 0.078, Loss w_fp: 0.007, Mask: 0.872 [2022-11-26 14:45:59,682][ INFO] Iters: 1240, Total loss: 0.042, Loss x: 0.044, Loss s: 0.076, Loss w_fp: 0.007, Mask: 0.872 [2022-11-26 14:48:25,603][ INFO] Iters: 1550, Total loss: 0.042, Loss x: 0.044, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.873 [2022-11-26 14:50:51,235][ INFO] Iters: 1860, Total loss: 0.042, Loss x: 0.044, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.874 [2022-11-26 14:53:16,984][ INFO] Iters: 2170, Total loss: 0.042, Loss x: 0.044, Loss s: 0.073, Loss w_fp: 0.007, Mask: 0.874 [2022-11-26 14:56:29,377][ INFO] Evaluation original >>>> meanIOU: 73.41

[2022-11-26 14:56:29,377][ INFO] ===========> Epoch: 66, LR: 0.0001, Previous best: 75.50 [2022-11-26 14:56:30,308][ INFO] Iters: 0, Total loss: 0.037, Loss x: 0.030, Loss s: 0.085, Loss w_fp: 0.004, Mask: 0.910 [2022-11-26 14:58:56,270][ INFO] Iters: 310, Total loss: 0.039, Loss x: 0.042, Loss s: 0.068, Loss w_fp: 0.006, Mask: 0.874 [2022-11-26 15:01:22,247][ INFO] Iters: 620, Total loss: 0.040, Loss x: 0.042, Loss s: 0.071, Loss w_fp: 0.006, Mask: 0.875 [2022-11-26 15:03:48,455][ INFO] Iters: 930, Total loss: 0.042, Loss x: 0.042, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.873 [2022-11-26 15:06:14,616][ INFO] Iters: 1240, Total loss: 0.042, Loss x: 0.042, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.874 [2022-11-26 15:08:40,501][ INFO] Iters: 1550, Total loss: 0.042, Loss x: 0.043, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.875 [2022-11-26 15:11:06,328][ INFO] Iters: 1860, Total loss: 0.042, Loss x: 0.042, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.875 [2022-11-26 15:13:32,193][ INFO] Iters: 2170, Total loss: 0.042, Loss x: 0.043, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.875 [2022-11-26 15:16:45,312][ INFO] Evaluation original >>>> meanIOU: 74.90

[2022-11-26 15:16:45,312][ INFO] ===========> Epoch: 67, LR: 0.0001, Previous best: 75.50 [2022-11-26 15:16:46,296][ INFO] Iters: 0, Total loss: 0.028, Loss x: 0.037, Loss s: 0.030, Loss w_fp: 0.009, Mask: 0.900 [2022-11-26 15:19:12,017][ INFO] Iters: 310, Total loss: 0.043, Loss x: 0.045, Loss s: 0.077, Loss w_fp: 0.006, Mask: 0.883 [2022-11-26 15:21:37,881][ INFO] Iters: 620, Total loss: 0.042, Loss x: 0.044, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.874 [2022-11-26 15:24:03,621][ INFO] Iters: 930, Total loss: 0.042, Loss x: 0.043, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.874 [2022-11-26 15:26:29,487][ INFO] Iters: 1240, Total loss: 0.041, Loss x: 0.043, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.875 [2022-11-26 15:28:55,299][ INFO] Iters: 1550, Total loss: 0.041, Loss x: 0.043, Loss s: 0.074, Loss w_fp: 0.007, Mask: 0.875 [2022-11-26 15:31:20,999][ INFO] Iters: 1860, Total loss: 0.042, Loss x: 0.043, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.874 [2022-11-26 15:33:46,605][ INFO] Iters: 2170, Total loss: 0.042, Loss x: 0.043, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.874 [2022-11-26 15:36:58,628][ INFO] Evaluation original >>>> meanIOU: 74.40

[2022-11-26 15:36:58,628][ INFO] ===========> Epoch: 68, LR: 0.0001, Previous best: 75.50 [2022-11-26 15:36:59,587][ INFO] Iters: 0, Total loss: 0.036, Loss x: 0.031, Loss s: 0.078, Loss w_fp: 0.003, Mask: 0.893 [2022-11-26 15:39:25,359][ INFO] Iters: 310, Total loss: 0.043, Loss x: 0.043, Loss s: 0.078, Loss w_fp: 0.006, Mask: 0.873 [2022-11-26 15:41:51,302][ INFO] Iters: 620, Total loss: 0.044, Loss x: 0.043, Loss s: 0.082, Loss w_fp: 0.006, Mask: 0.871 [2022-11-26 15:44:17,272][ INFO] Iters: 930, Total loss: 0.043, Loss x: 0.044, Loss s: 0.077, Loss w_fp: 0.006, Mask: 0.874 [2022-11-26 15:46:43,027][ INFO] Iters: 1240, Total loss: 0.043, Loss x: 0.043, Loss s: 0.077, Loss w_fp: 0.006, Mask: 0.875 [2022-11-26 15:49:08,844][ INFO] Iters: 1550, Total loss: 0.043, Loss x: 0.043, Loss s: 0.079, Loss w_fp: 0.006, Mask: 0.874 [2022-11-26 15:51:34,646][ INFO] Iters: 1860, Total loss: 0.043, Loss x: 0.043, Loss s: 0.077, Loss w_fp: 0.007, Mask: 0.873 [2022-11-26 15:54:00,462][ INFO] Iters: 2170, Total loss: 0.043, Loss x: 0.043, Loss s: 0.078, Loss w_fp: 0.007, Mask: 0.874 [2022-11-26 15:57:12,666][ INFO] Evaluation original >>>> meanIOU: 73.46

[2022-11-26 15:57:12,667][ INFO] ===========> Epoch: 69, LR: 0.0001, Previous best: 75.50 [2022-11-26 15:57:13,584][ INFO] Iters: 0, Total loss: 0.020, Loss x: 0.019, Loss s: 0.040, Loss w_fp: 0.002, Mask: 0.936 [2022-11-26 15:59:39,474][ INFO] Iters: 310, Total loss: 0.042, Loss x: 0.042, Loss s: 0.078, Loss w_fp: 0.007, Mask: 0.883 [2022-11-26 16:02:05,623][ INFO] Iters: 620, Total loss: 0.042, Loss x: 0.043, Loss s: 0.075, Loss w_fp: 0.007, Mask: 0.880 [2022-11-26 16:04:31,799][ INFO] Iters: 930, Total loss: 0.042, Loss x: 0.043, Loss s: 0.077, Loss w_fp: 0.007, Mask: 0.879 [2022-11-26 16:06:57,971][ INFO] Iters: 1240, Total loss: 0.042, Loss x: 0.043, Loss s: 0.077, Loss w_fp: 0.006, Mask: 0.878 [2022-11-26 16:09:23,926][ INFO] Iters: 1550, Total loss: 0.042, Loss x: 0.043, Loss s: 0.077, Loss w_fp: 0.007, Mask: 0.877 [2022-11-26 16:11:49,954][ INFO] Iters: 1860, Total loss: 0.042, Loss x: 0.043, Loss s: 0.076, Loss w_fp: 0.007, Mask: 0.876 [2022-11-26 16:14:16,064][ INFO] Iters: 2170, Total loss: 0.042, Loss x: 0.043, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.875 [2022-11-26 16:17:27,339][ INFO] Evaluation original >>>> meanIOU: 74.04

[2022-11-26 16:17:27,339][ INFO] ===========> Epoch: 70, LR: 0.0001, Previous best: 75.50 [2022-11-26 16:17:28,286][ INFO] Iters: 0, Total loss: 0.067, Loss x: 0.055, Loss s: 0.154, Loss w_fp: 0.004, Mask: 0.897 [2022-11-26 16:19:53,816][ INFO] Iters: 310, Total loss: 0.042, Loss x: 0.044, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.877 [2022-11-26 16:22:19,542][ INFO] Iters: 620, Total loss: 0.041, Loss x: 0.043, Loss s: 0.072, Loss w_fp: 0.006, Mask: 0.875 [2022-11-26 16:24:45,129][ INFO] Iters: 930, Total loss: 0.041, Loss x: 0.042, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.874 [2022-11-26 16:27:10,740][ INFO] Iters: 1240, Total loss: 0.041, Loss x: 0.042, Loss s: 0.074, Loss w_fp: 0.007, Mask: 0.875 [2022-11-26 16:29:36,433][ INFO] Iters: 1550, Total loss: 0.042, Loss x: 0.042, Loss s: 0.076, Loss w_fp: 0.007, Mask: 0.873 [2022-11-26 16:32:02,092][ INFO] Iters: 1860, Total loss: 0.041, Loss x: 0.042, Loss s: 0.075, Loss w_fp: 0.007, Mask: 0.872 [2022-11-26 16:34:27,946][ INFO] Iters: 2170, Total loss: 0.042, Loss x: 0.042, Loss s: 0.075, Loss w_fp: 0.007, Mask: 0.872 [2022-11-26 16:37:40,700][ INFO] Evaluation original >>>> meanIOU: 74.45

[2022-11-26 16:37:40,700][ INFO] ===========> Epoch: 71, LR: 0.0001, Previous best: 75.50 [2022-11-26 16:37:41,687][ INFO] Iters: 0, Total loss: 0.034, Loss x: 0.042, Loss s: 0.050, Loss w_fp: 0.003, Mask: 0.958 [2022-11-26 16:40:07,409][ INFO] Iters: 310, Total loss: 0.042, Loss x: 0.040, Loss s: 0.079, Loss w_fp: 0.007, Mask: 0.872 [2022-11-26 16:42:33,304][ INFO] Iters: 620, Total loss: 0.041, Loss x: 0.041, Loss s: 0.075, Loss w_fp: 0.007, Mask: 0.875 [2022-11-26 16:44:59,122][ INFO] Iters: 930, Total loss: 0.041, Loss x: 0.042, Loss s: 0.075, Loss w_fp: 0.007, Mask: 0.874 [2022-11-26 16:47:25,134][ INFO] Iters: 1240, Total loss: 0.041, Loss x: 0.042, Loss s: 0.074, Loss w_fp: 0.007, Mask: 0.874 [2022-11-26 16:49:51,268][ INFO] Iters: 1550, Total loss: 0.041, Loss x: 0.042, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.876 [2022-11-26 16:52:17,082][ INFO] Iters: 1860, Total loss: 0.041, Loss x: 0.042, Loss s: 0.073, Loss w_fp: 0.007, Mask: 0.876 [2022-11-26 16:54:42,751][ INFO] Iters: 2170, Total loss: 0.041, Loss x: 0.042, Loss s: 0.074, Loss w_fp: 0.007, Mask: 0.876 [2022-11-26 16:57:54,588][ INFO] Evaluation original >>>> meanIOU: 74.28

[2022-11-26 16:57:54,589][ INFO] ===========> Epoch: 72, LR: 0.0001, Previous best: 75.50 [2022-11-26 16:57:55,606][ INFO] Iters: 0, Total loss: 0.021, Loss x: 0.024, Loss s: 0.031, Loss w_fp: 0.003, Mask: 0.906 [2022-11-26 17:00:21,296][ INFO] Iters: 310, Total loss: 0.041, Loss x: 0.042, Loss s: 0.074, Loss w_fp: 0.007, Mask: 0.873 [2022-11-26 17:02:47,255][ INFO] Iters: 620, Total loss: 0.040, Loss x: 0.042, Loss s: 0.072, Loss w_fp: 0.006, Mask: 0.872 [2022-11-26 17:05:13,095][ INFO] Iters: 930, Total loss: 0.041, Loss x: 0.042, Loss s: 0.072, Loss w_fp: 0.006, Mask: 0.872 [2022-11-26 17:07:38,972][ INFO] Iters: 1240, Total loss: 0.040, Loss x: 0.042, Loss s: 0.071, Loss w_fp: 0.007, Mask: 0.872 [2022-11-26 17:10:04,906][ INFO] Iters: 1550, Total loss: 0.040, Loss x: 0.042, Loss s: 0.071, Loss w_fp: 0.007, Mask: 0.874 [2022-11-26 17:12:30,715][ INFO] Iters: 1860, Total loss: 0.041, Loss x: 0.042, Loss s: 0.071, Loss w_fp: 0.007, Mask: 0.875 [2022-11-26 17:14:56,443][ INFO] Iters: 2170, Total loss: 0.041, Loss x: 0.042, Loss s: 0.071, Loss w_fp: 0.007, Mask: 0.875 [2022-11-26 17:18:08,370][ INFO] Evaluation original >>>> meanIOU: 73.72

[2022-11-26 17:18:08,370][ INFO] ===========> Epoch: 73, LR: 0.0001, Previous best: 75.50 [2022-11-26 17:18:09,303][ INFO] Iters: 0, Total loss: 0.036, Loss x: 0.055, Loss s: 0.021, Loss w_fp: 0.012, Mask: 0.858 [2022-11-26 17:20:35,162][ INFO] Iters: 310, Total loss: 0.042, Loss x: 0.043, Loss s: 0.074, Loss w_fp: 0.007, Mask: 0.869 [2022-11-26 17:23:01,109][ INFO] Iters: 620, Total loss: 0.041, Loss x: 0.042, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.874 [2022-11-26 17:25:27,178][ INFO] Iters: 930, Total loss: 0.041, Loss x: 0.043, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.874 [2022-11-26 17:27:53,151][ INFO] Iters: 1240, Total loss: 0.042, Loss x: 0.043, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.875 [2022-11-26 17:30:19,070][ INFO] Iters: 1550, Total loss: 0.041, Loss x: 0.042, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.875 [2022-11-26 17:32:44,909][ INFO] Iters: 1860, Total loss: 0.041, Loss x: 0.042, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.876 [2022-11-26 17:35:11,114][ INFO] Iters: 2170, Total loss: 0.041, Loss x: 0.042, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.876 [2022-11-26 17:38:25,497][ INFO] Evaluation original >>>> meanIOU: 75.23

[2022-11-26 17:38:25,498][ INFO] ===========> Epoch: 74, LR: 0.0000, Previous best: 75.50 [2022-11-26 17:38:26,570][ INFO] Iters: 0, Total loss: 0.027, Loss x: 0.040, Loss s: 0.012, Loss w_fp: 0.017, Mask: 0.965 [2022-11-26 17:40:52,452][ INFO] Iters: 310, Total loss: 0.039, Loss x: 0.040, Loss s: 0.070, Loss w_fp: 0.006, Mask: 0.879 [2022-11-26 17:43:18,544][ INFO] Iters: 620, Total loss: 0.040, Loss x: 0.042, Loss s: 0.072, Loss w_fp: 0.006, Mask: 0.877 [2022-11-26 17:45:44,546][ INFO] Iters: 930, Total loss: 0.040, Loss x: 0.042, Loss s: 0.072, Loss w_fp: 0.006, Mask: 0.877 [2022-11-26 17:48:10,223][ INFO] Iters: 1240, Total loss: 0.041, Loss x: 0.042, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.877 [2022-11-26 17:50:36,061][ INFO] Iters: 1550, Total loss: 0.041, Loss x: 0.042, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.877 [2022-11-26 17:53:01,906][ INFO] Iters: 1860, Total loss: 0.041, Loss x: 0.042, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.878 [2022-11-26 17:55:27,752][ INFO] Iters: 2170, Total loss: 0.041, Loss x: 0.041, Loss s: 0.074, Loss w_fp: 0.007, Mask: 0.878 [2022-11-26 17:58:40,788][ INFO] Evaluation original >>>> meanIOU: 74.61

[2022-11-26 17:58:40,789][ INFO] ===========> Epoch: 75, LR: 0.0000, Previous best: 75.50 [2022-11-26 17:58:41,776][ INFO] Iters: 0, Total loss: 0.044, Loss x: 0.064, Loss s: 0.040, Loss w_fp: 0.007, Mask: 0.950 [2022-11-26 18:01:07,456][ INFO] Iters: 310, Total loss: 0.039, Loss x: 0.042, Loss s: 0.067, Loss w_fp: 0.006, Mask: 0.885 [2022-11-26 18:03:33,128][ INFO] Iters: 620, Total loss: 0.039, Loss x: 0.041, Loss s: 0.067, Loss w_fp: 0.006, Mask: 0.881 [2022-11-26 18:05:59,445][ INFO] Iters: 930, Total loss: 0.040, Loss x: 0.042, Loss s: 0.070, Loss w_fp: 0.006, Mask: 0.880 [2022-11-26 18:08:25,245][ INFO] Iters: 1240, Total loss: 0.041, Loss x: 0.042, Loss s: 0.072, Loss w_fp: 0.006, Mask: 0.878 [2022-11-26 18:10:51,085][ INFO] Iters: 1550, Total loss: 0.041, Loss x: 0.042, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.879 [2022-11-26 18:13:16,942][ INFO] Iters: 1860, Total loss: 0.041, Loss x: 0.041, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.878 [2022-11-26 18:15:42,945][ INFO] Iters: 2170, Total loss: 0.040, Loss x: 0.041, Loss s: 0.072, Loss w_fp: 0.006, Mask: 0.879 [2022-11-26 18:18:56,812][ INFO] Evaluation original >>>> meanIOU: 73.65

[2022-11-26 18:18:56,813][ INFO] ===========> Epoch: 76, LR: 0.0000, Previous best: 75.50 [2022-11-26 18:18:57,858][ INFO] Iters: 0, Total loss: 0.019, Loss x: 0.028, Loss s: 0.017, Loss w_fp: 0.002, Mask: 0.947 [2022-11-26 18:21:23,727][ INFO] Iters: 310, Total loss: 0.038, Loss x: 0.040, Loss s: 0.065, Loss w_fp: 0.006, Mask: 0.883 [2022-11-26 18:23:49,527][ INFO] Iters: 620, Total loss: 0.039, Loss x: 0.041, Loss s: 0.068, Loss w_fp: 0.006, Mask: 0.883 [2022-11-26 18:26:15,655][ INFO] Iters: 930, Total loss: 0.040, Loss x: 0.041, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.881 [2022-11-26 18:28:41,375][ INFO] Iters: 1240, Total loss: 0.040, Loss x: 0.041, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.881 [2022-11-26 18:31:07,247][ INFO] Iters: 1550, Total loss: 0.040, Loss x: 0.041, Loss s: 0.072, Loss w_fp: 0.006, Mask: 0.880 [2022-11-26 18:33:32,964][ INFO] Iters: 1860, Total loss: 0.040, Loss x: 0.041, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.880 [2022-11-26 18:35:58,774][ INFO] Iters: 2170, Total loss: 0.040, Loss x: 0.041, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.879 [2022-11-26 18:39:12,715][ INFO] Evaluation original >>>> meanIOU: 74.40

[2022-11-26 18:39:12,715][ INFO] ===========> Epoch: 77, LR: 0.0000, Previous best: 75.50 [2022-11-26 18:39:13,694][ INFO] Iters: 0, Total loss: 0.023, Loss x: 0.024, Loss s: 0.040, Loss w_fp: 0.005, Mask: 0.893 [2022-11-26 18:41:39,716][ INFO] Iters: 310, Total loss: 0.039, Loss x: 0.040, Loss s: 0.068, Loss w_fp: 0.006, Mask: 0.879 [2022-11-26 18:44:05,647][ INFO] Iters: 620, Total loss: 0.040, Loss x: 0.041, Loss s: 0.071, Loss w_fp: 0.006, Mask: 0.878 [2022-11-26 18:46:31,521][ INFO] Iters: 930, Total loss: 0.040, Loss x: 0.041, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.877 [2022-11-26 18:48:57,314][ INFO] Iters: 1240, Total loss: 0.041, Loss x: 0.042, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.877 [2022-11-26 18:51:23,071][ INFO] Iters: 1550, Total loss: 0.041, Loss x: 0.042, Loss s: 0.075, Loss w_fp: 0.006, Mask: 0.877 [2022-11-26 18:53:48,921][ INFO] Iters: 1860, Total loss: 0.041, Loss x: 0.042, Loss s: 0.074, Loss w_fp: 0.006, Mask: 0.878 [2022-11-26 18:56:14,890][ INFO] Iters: 2170, Total loss: 0.041, Loss x: 0.041, Loss s: 0.073, Loss w_fp: 0.006, Mask: 0.878 [2022-11-26 18:59:28,298][ INFO] Evaluation original >>>> meanIOU: 74.88

[2022-11-26 18:59:28,298][ INFO] ===========> Epoch: 78, LR: 0.0000, Previous best: 75.50 [2022-11-26 18:59:29,233][ INFO] Iters: 0, Total loss: 0.042, Loss x: 0.070, Loss s: 0.027, Loss w_fp: 0.002, Mask: 0.941 [2022-11-26 19:01:55,023][ INFO] Iters: 310, Total loss: 0.040, Loss x: 0.042, Loss s: 0.071, Loss w_fp: 0.007, Mask: 0.876 [2022-11-26 19:04:21,088][ INFO] Iters: 620, Total loss: 0.042, Loss x: 0.042, Loss s: 0.076, Loss w_fp: 0.007, Mask: 0.877 [2022-11-26 19:06:46,743][ INFO] Iters: 930, Total loss: 0.041, Loss x: 0.042, Loss s: 0.075, Loss w_fp: 0.007, Mask: 0.876 [2022-11-26 19:09:12,492][ INFO] Iters: 1240, Total loss: 0.042, Loss x: 0.042, Loss s: 0.077, Loss w_fp: 0.007, Mask: 0.876 [2022-11-26 19:11:38,280][ INFO] Iters: 1550, Total loss: 0.041, Loss x: 0.042, Loss s: 0.075, Loss w_fp: 0.007, Mask: 0.878 [2022-11-26 19:14:04,024][ INFO] Iters: 1860, Total loss: 0.041, Loss x: 0.041, Loss s: 0.075, Loss w_fp: 0.007, Mask: 0.879

LiheYoung commented 2 years ago

Since the batch size is reduced by 2 times, I think it can be expected that the performance drops by 1%. You can try to use two GPUs, or reduce the cropping size from 321 to 281 and maintain the batch size as 8 if GPU memory is allowed.

jerrywyn commented 2 years ago

I ran the unimatch with batch size 4 and initial lr 0.0005 on a single GPU for the 1/16 partition of pascal and I took 78.63 mIOU without other change except eliminated distributed code. The best result is trained in epoch 38. This is a bit of a huge difference from your result of 75.50？

nysp78 commented 2 years ago

@jerrywyn Which part of the distributed code do you cross out? Also, what crop size do you use?

Thanks