search

czczup / ViT-Adapter

[ICLR 2023 Spotlight] Vision Transformer Adapter for Dense Predictions
https://arxiv.org/abs/2205.08534
Apache License 2.0
1.26k stars 139 forks source link

Poor performance on custom dataset. #155

Closed lianzheng-research closed 12 months ago

lianzheng-research commented 12 months ago

First, thank you for your excellent work! I have trained UperNet-DeiT-Adapter-Tiny-512-160k on ADE20k dataset and got promising performance. Now I'm training BEiT-Adapter on my custom dataset. I have ~50k semantic segmentation dataset of four categories (containing background class). I found the training loss and the acc_seg are really, and the decode.acc_seg maintains around 25.0. Here is my logs.

2023-11-22 01:50:52,670 - mmcv - INFO - Reducer buckets have been rebuilt in this iteration.
2023-11-22 01:51:14,253 - mmseg - INFO - Iter [50/160000]       lr: 1.006e-06, eta: 20:05:50, time: 0.452, data_time: 0.011, memory: 3738, decode.loss_ce: 0.2976, decode.acc_seg: 8.1025, aux.loss_ce: 0.1137, aux.acc_seg: 11.0681, loss: 0.4113
2023-11-22 01:51:36,294 - mmseg - INFO - Iter [100/160000]      lr: 2.032e-06, eta: 19:50:05, time: 0.441, data_time: 0.005, memory: 3738, decode.loss_ce: 0.2539, decode.acc_seg: 14.8696, aux.loss_ce: 0.1075, aux.acc_seg: 12.0448, loss: 0.3614
2023-11-22 01:51:58,046 - mmseg - INFO - Iter [150/160000]      lr: 3.057e-06, eta: 19:39:25, time: 0.435, data_time: 0.005, memory: 3738, decode.loss_ce: 0.2183, decode.acc_seg: 19.7279, aux.loss_ce: 0.1082, aux.acc_seg: 17.1957, loss: 0.3265
2023-11-22 01:52:19,835 - mmseg - INFO - Iter [200/160000]      lr: 4.081e-06, eta: 19:34:29, time: 0.436, data_time: 0.005, memory: 3738, decode.loss_ce: 0.1698, decode.acc_seg: 19.2952, aux.loss_ce: 0.0947, aux.acc_seg: 19.0775, loss: 0.2645
2023-11-22 01:52:41,727 - mmseg - INFO - Iter [250/160000]      lr: 5.105e-06, eta: 19:32:26, time: 0.438, data_time: 0.005, memory: 3738, decode.loss_ce: 0.1544, decode.acc_seg: 20.4510, aux.loss_ce: 0.0919, aux.acc_seg: 20.3115, loss: 0.2463
2023-11-22 01:53:03,520 - mmseg - INFO - Iter [300/160000]      lr: 6.128e-06, eta: 19:30:04, time: 0.436, data_time: 0.005, memory: 3738, decode.loss_ce: 0.1230, decode.acc_seg: 21.5389, aux.loss_ce: 0.0795, aux.acc_seg: 20.2106, loss: 0.2025
2023-11-22 01:53:25,342 - mmseg - INFO - Iter [350/160000]      lr: 7.151e-06, eta: 19:28:30, time: 0.436, data_time: 0.005, memory: 3738, decode.loss_ce: 0.1187, decode.acc_seg: 22.1860, aux.loss_ce: 0.0731, aux.acc_seg: 21.6865, loss: 0.1918
2023-11-22 01:53:47,357 - mmseg - INFO - Iter [400/160000]      lr: 8.173e-06, eta: 19:28:31, time: 0.440, data_time: 0.005, memory: 3738, decode.loss_ce: 0.1026, decode.acc_seg: 23.1645, aux.loss_ce: 0.0651, aux.acc_seg: 22.0624, loss: 0.1677
2023-11-22 01:54:09,344 - mmseg - INFO - Iter [450/160000]      lr: 9.194e-06, eta: 19:28:15, time: 0.440, data_time: 0.005, memory: 3738, decode.loss_ce: 0.1041, decode.acc_seg: 23.1843, aux.loss_ce: 0.0623, aux.acc_seg: 22.1687, loss: 0.1664
2023-11-22 01:54:31,278 - mmseg - INFO - Iter [500/160000]      lr: 1.021e-05, eta: 19:27:44, time: 0.439, data_time: 0.005, memory: 3738, decode.loss_ce: 0.0884, decode.acc_seg: 23.6145, aux.loss_ce: 0.0502, aux.acc_seg: 22.9369, loss: 0.1386
2023-11-22 01:54:53,196 - mmseg - INFO - Iter [550/160000]      lr: 1.123e-05, eta: 19:27:05, time: 0.438, data_time: 0.005, memory: 3738, decode.loss_ce: 0.0692, decode.acc_seg: 22.7350, aux.loss_ce: 0.0447, aux.acc_seg: 21.6501, loss: 0.1139
2023-11-22 01:55:15,227 - mmseg - INFO - Iter [600/160000]      lr: 1.225e-05, eta: 19:27:05, time: 0.441, data_time: 0.005, memory: 3738, decode.loss_ce: 0.0680, decode.acc_seg: 25.5586, aux.loss_ce: 0.0427, aux.acc_seg: 25.0341, loss: 0.1108
2023-11-22 01:55:37,057 - mmseg - INFO - Iter [650/160000]      lr: 1.327e-05, eta: 19:26:10, time: 0.437, data_time: 0.005, memory: 3738, decode.loss_ce: 0.0680, decode.acc_seg: 23.5253, aux.loss_ce: 0.0389, aux.acc_seg: 22.9765, loss: 0.1069
2023-11-22 01:55:58,692 - mmseg - INFO - Iter [700/160000]      lr: 1.429e-05, eta: 19:24:35, time: 0.433, data_time: 0.005, memory: 3738, decode.loss_ce: 0.0504, decode.acc_seg: 23.1398, aux.loss_ce: 0.0321, aux.acc_seg: 22.6663, loss: 0.0824
2023-11-22 01:56:20,878 - mmseg - INFO - Iter [750/160000]      lr: 1.531e-05, eta: 19:25:06, time: 0.444, data_time: 0.006, memory: 3738, decode.loss_ce: 0.0394, decode.acc_seg: 23.2774, aux.loss_ce: 0.0264, aux.acc_seg: 23.0125, loss: 0.0658
2023-11-22 01:56:42,816 - mmseg - INFO - Iter [800/160000]      lr: 1.632e-05, eta: 19:24:39, time: 0.438, data_time: 0.005, memory: 3738, decode.loss_ce: 0.0421, decode.acc_seg: 24.7753, aux.loss_ce: 0.0267, aux.acc_seg: 24.7051, loss: 0.0688
2023-11-22 01:57:04,684 - mmseg - INFO - Iter [850/160000]      lr: 1.734e-05, eta: 19:24:05, time: 0.438, data_time: 0.006, memory: 3738, decode.loss_ce: 0.0534, decode.acc_seg: 23.8046, aux.loss_ce: 0.0278, aux.acc_seg: 23.5948, loss: 0.0813
2023-11-22 01:57:26,567 - mmseg - INFO - Iter [900/160000]      lr: 1.836e-05, eta: 19:23:33, time: 0.438, data_time: 0.005, memory: 3738, decode.loss_ce: 0.0482, decode.acc_seg: 24.6686, aux.loss_ce: 0.0242, aux.acc_seg: 24.6322, loss: 0.0724
2023-11-22 01:57:48,439 - mmseg - INFO - Iter [950/160000]      lr: 1.937e-05, eta: 19:22:58, time: 0.437, data_time: 0.005, memory: 3738, decode.loss_ce: 0.0411, decode.acc_seg: 25.7160, aux.loss_ce: 0.0230, aux.acc_seg: 25.6236, loss: 0.0641
2023-11-22 01:58:09,990 - mmseg - INFO - Saving checkpoint at 1000 iterations
2023-11-22 01:58:11,721 - mmseg - INFO - Exp name: upernet_deit_tiny_512_160k_ade20k_ss.py
2023-11-22 01:58:11,721 - mmseg - INFO - Iter [1000/160000]     lr: 2.039e-05, eta: 19:26:12, time: 0.466, data_time: 0.005, memory: 3738, decode.loss_ce: 0.0332, decode.acc_seg: 26.0492, aux.loss_ce: 0.0204, aux.acc_seg: 25.9117, loss: 0.0536
2023-11-22 01:58:33,570 - mmseg - INFO - Iter [1050/160000]     lr: 2.140e-05, eta: 19:25:22, time: 0.437, data_time: 0.005, memory: 3738, decode.loss_ce: 0.0221, decode.acc_seg: 25.5939, aux.loss_ce: 0.0135, aux.acc_seg: 25.5504, loss: 0.0356
2023-11-22 01:58:55,552 - mmseg - INFO - Iter [1100/160000]     lr: 2.241e-05, eta: 19:25:02, time: 0.440, data_time: 0.006, memory: 3738, decode.loss_ce: 0.0237, decode.acc_seg: 24.3181, aux.loss_ce: 0.0137, aux.acc_seg: 24.2925, loss: 0.0373
2023-11-22 01:59:17,516 - mmseg - INFO - Iter [1150/160000]     lr: 2.342e-05, eta: 19:24:35, time: 0.439, data_time: 0.005, memory: 3738, decode.loss_ce: 0.0208, decode.acc_seg: 22.9327, aux.loss_ce: 0.0116, aux.acc_seg: 23.0078, loss: 0.0323
2023-11-22 01:59:39,441 - mmseg - INFO - Iter [1200/160000]     lr: 2.444e-05, eta: 19:24:04, time: 0.438, data_time: 0.005, memory: 3738, decode.loss_ce: 0.0172, decode.acc_seg: 26.6480, aux.loss_ce: 0.0111, aux.acc_seg: 26.6639, loss: 0.0282
2023-11-22 02:00:01,310 - mmseg - INFO - Iter [1250/160000]     lr: 2.545e-05, eta: 19:23:27, time: 0.437, data_time: 0.005, memory: 3738, decode.loss_ce: 0.0284, decode.acc_seg: 26.1277, aux.loss_ce: 0.0145, aux.acc_seg: 26.0905, loss: 0.0429
2023-11-22 02:00:23,544 - mmseg - INFO - Iter [1300/160000]     lr: 2.646e-05, eta: 19:23:35, time: 0.445, data_time: 0.005, memory: 3738, decode.loss_ce: 0.0170, decode.acc_seg: 24.3332, aux.loss_ce: 0.0085, aux.acc_seg: 24.4463, loss: 0.0255
2023-11-22 02:00:45,317 - mmseg - INFO - Iter [1350/160000]     lr: 2.747e-05, eta: 19:22:47, time: 0.435, data_time: 0.005, memory: 3738, decode.loss_ce: 0.0231, decode.acc_seg: 24.9752, aux.loss_ce: 0.0119, aux.acc_seg: 24.8936, loss: 0.0350
2023-11-22 02:01:07,096 - mmseg - INFO - Iter [1400/160000]     lr: 2.848e-05, eta: 19:21:59, time: 0.435, data_time: 0.005, memory: 3738, decode.loss_ce: 0.0445, decode.acc_seg: 26.3366, aux.loss_ce: 0.0167, aux.acc_seg: 26.5222, loss: 0.0612
2023-11-22 02:01:28,965 - mmseg - INFO - Iter [1450/160000]     lr: 2.948e-05, eta: 19:21:27, time: 0.438, data_time: 0.005, memory: 3738, decode.loss_ce: 0.0229, decode.acc_seg: 26.0669, aux.loss_ce: 0.0105, aux.acc_seg: 26.0061, loss: 0.0334
2023-11-22 02:01:50,879 - mmseg - INFO - Iter [1500/160000]     lr: 3.049e-05, eta: 19:20:58, time: 0.438, data_time: 0.005, memory: 3738, decode.loss_ce: 0.0136, decode.acc_seg: 24.4322, aux.loss_ce: 0.0072, aux.acc_seg: 24.4574, loss: 0.0208
2023-11-22 02:02:12,735 - mmseg - INFO - Iter [1550/160000]     lr: 3.050e-05, eta: 19:20:22, time: 0.437, data_time: 0.005, memory: 3738, decode.loss_ce: 0.0141, decode.acc_seg: 26.6601, aux.loss_ce: 0.0077, aux.acc_seg: 26.6608, loss: 0.0218
2023-11-22 02:02:34,824 - mmseg - INFO - Iter [1600/160000]     lr: 3.049e-05, eta: 19:20:13, time: 0.442, data_time: 0.006, memory: 3738, decode.loss_ce: 0.0186, decode.acc_seg: 27.4805, aux.loss_ce: 0.0095, aux.acc_seg: 27.4137, loss: 0.0281
2023-11-22 02:02:56,589 - mmseg - INFO - Iter [1650/160000]     lr: 3.048e-05, eta: 19:19:32, time: 0.435, data_time: 0.005, memory: 3738, decode.loss_ce: 0.0344, decode.acc_seg: 22.8601, aux.loss_ce: 0.0143, aux.acc_seg: 23.1057, loss: 0.0487
2023-11-22 02:03:18,389 - mmseg - INFO - Iter [1700/160000]     lr: 3.047e-05, eta: 19:18:54, time: 0.436, data_time: 0.005, memory: 3738, decode.loss_ce: 0.0498, decode.acc_seg: 23.2622, aux.loss_ce: 0.0185, aux.acc_seg: 23.3313, loss: 0.0684
2023-11-22 02:03:40,140 - mmseg - INFO - Iter [1750/160000]     lr: 3.046e-05, eta: 19:18:13, time: 0.435, data_time: 0.005, memory: 3738, decode.loss_ce: 0.0398, decode.acc_seg: 24.1014, aux.loss_ce: 0.0148, aux.acc_seg: 24.1411, loss: 0.0546
2023-11-22 02:04:01,936 - mmseg - INFO - Iter [1800/160000]     lr: 3.045e-05, eta: 19:17:34, time: 0.435, data_time: 0.005, memory: 3738, decode.loss_ce: 0.0263, decode.acc_seg: 25.6600, aux.loss_ce: 0.0109, aux.acc_seg: 25.6873, loss: 0.0372
2023-11-22 02:04:23,975 - mmseg - INFO - Iter [1850/160000]     lr: 3.044e-05, eta: 19:17:20, time: 0.441, data_time: 0.006, memory: 3738, decode.loss_ce: 0.0205, decode.acc_seg: 23.7300, aux.loss_ce: 0.0093, aux.acc_seg: 23.7323, loss: 0.0298
2023-11-22 02:04:45,692 - mmseg - INFO - Iter [1900/160000]     lr: 3.043e-05, eta: 19:16:40, time: 0.435, data_time: 0.005, memory: 3738, decode.loss_ce: 0.0283, decode.acc_seg: 25.0150, aux.loss_ce: 0.0122, aux.acc_seg: 25.0241, loss: 0.0405
2023-11-22 02:05:07,559 - mmseg - INFO - Iter [1950/160000]     lr: 3.043e-05, eta: 19:16:11, time: 0.437, data_time: 0.005, memory: 3738, decode.loss_ce: 0.0093, decode.acc_seg: 24.8424, aux.loss_ce: 0.0047, aux.acc_seg: 24.8262, loss: 0.0141
2023-11-22 02:05:29,185 - mmseg - INFO - Saving checkpoint at 2000 iterations
2023-11-22 02:05:30,911 - mmseg - INFO - Exp name: upernet_deit_tiny_512_160k_ade20k_ss.py

Could you please give me some advices? Thank you very much!

22ema commented 1 month ago

Hello. Do you solve this problem?