MzeroMiko
commented
3 weeks ago what is the performance in iter16000 (the first val)?
Closed ShixuanGu closed 3 weeks ago
MzeroMiko
commented
3 weeks ago what is the performance in iter16000 (the first val)?
ShixuanGu
commented
3 weeks ago mIoU 37.58
06/03 14:14:49 - mmengine - INFO - Exp name: ft_cs_20240603_132845
06/03 14:14:49 - mmengine - INFO - Iter(train) [ 16000/160000] base_lr: 5.4511e-05 lr: 5.4511e-05 eta: 6:48:42 time: 0.1667 data_time: 0.0044 memory: 4749 loss: 0.9094 decode.loss_ce: 0.6152 decode.acc_seg: 66.1648 aux.loss_ce: 0.2941 aux.acc_seg: 53.7722
06/03 14:14:49 - mmengine - INFO - Saving checkpoint at 16000 iterations
06/03 14:16:25 - mmengine - INFO - Iter(val) [ 50/500] eta: 0:14:06 time: 1.0836 data_time: 0.0013 memory: 75471
06/03 14:17:15 - mmengine - INFO - Iter(val) [100/500] eta: 0:09:38 time: 0.1367 data_time: 0.0011 memory: 77083
06/03 14:17:32 - mmengine - INFO - Iter(val) [150/500] eta: 0:06:16 time: 0.0334 data_time: 0.0011 memory: 34455
06/03 14:18:26 - mmengine - INFO - Iter(val) [200/500] eta: 0:05:22 time: 0.1571 data_time: 0.0012 memory: 40060
06/03 14:18:43 - mmengine - INFO - Iter(val) [250/500] eta: 0:03:52 time: 0.7341 data_time: 0.0012 memory: 40062
06/03 14:19:02 - mmengine - INFO - Iter(val) [300/500] eta: 0:02:47 time: 0.0386 data_time: 0.0014 memory: 47781
06/03 14:19:09 - mmengine - INFO - Iter(val) [350/500] eta: 0:01:50 time: 0.2811 data_time: 0.0011 memory: 34475
06/03 14:19:26 - mmengine - INFO - Iter(val) [400/500] eta: 0:01:08 time: 0.0399 data_time: 0.0011 memory: 34489
06/03 14:19:45 - mmengine - INFO - Iter(val) [450/500] eta: 0:00:32 time: 0.0403 data_time: 0.0010 memory: 34493
06/03 14:20:04 - mmengine - INFO - Iter(val) [500/500] eta: 0:00:00 time: 1.0236 data_time: 0.0012 memory: 75480
06/03 14:20:32 - mmengine - INFO - per class results:
06/03 14:20:32 - mmengine - INFO -
+---------------------+-------+-------+
| Class | IoU | Acc |
+---------------------+-------+-------+
| wall | 72.83 | 83.26 |
| building | 81.3 | 93.35 |
| sky | 93.21 | 96.06 |
| floor | 76.68 | 88.14 |
| tree | 72.36 | 86.45 |
| ceiling | 79.62 | 92.17 |
| road | 77.48 | 80.44 |
| bed | 82.66 | 93.21 |
| windowpane | 54.61 | 75.22 |
| grass | 65.96 | 91.97 |
| cabinet | 52.56 | 66.04 |
| sidewalk | 57.92 | 82.39 |
| person | 74.97 | 89.43 |
| earth | 33.84 | 50.0 |
| door | 41.28 | 58.23 |
| table | 46.26 | 60.65 |
| mountain | 56.43 | 67.45 |
| plant | 48.14 | 56.47 |
| curtain | 65.85 | 81.23 |
| chair | 44.18 | 64.53 |
| car | 77.69 | 92.62 |
| water | 44.2 | 60.58 |
| painting | 62.7 | 80.76 |
| sofa | 54.97 | 67.94 |
| shelf | 35.64 | 52.44 |
| house | 37.07 | 45.02 |
| sea | 58.67 | 94.26 |
| mirror | 53.26 | 62.2 |
| rug | 50.76 | 57.57 |
| field | 21.77 | 31.77 |
| armchair | 30.76 | 57.84 |
| seat | 54.03 | 81.86 |
| fence | 36.04 | 47.7 |
| desk | 35.64 | 55.0 |
| rock | 46.21 | 72.08 |
| wardrobe | 42.04 | 58.84 |
| lamp | 48.55 | 69.36 |
| bathtub | 62.5 | 80.45 |
| railing | 30.97 | 53.26 |
| cushion | 43.49 | 60.93 |
| base | 16.67 | 22.73 |
| box | 16.59 | 19.55 |
| column | 43.83 | 55.47 |
| signboard | 32.14 | 42.42 |
| chest of drawers | 30.77 | 67.42 |
| counter | 24.13 | 34.15 |
| sand | 40.05 | 47.52 |
| sink | 59.37 | 64.33 |
| skyscraper | 53.94 | 60.92 |
| fireplace | 63.67 | 87.17 |
| refrigerator | 46.93 | 81.95 |
| grandstand | 47.55 | 67.01 |
| path | 22.48 | 36.4 |
| stairs | 32.35 | 40.92 |
| runway | 70.21 | 96.12 |
| case | 42.58 | 64.85 |
| pool table | 84.39 | 96.34 |
| pillow | 41.0 | 53.35 |
| screen door | 55.23 | 67.14 |
| stairway | 31.12 | 36.82 |
| river | 15.85 | 30.39 |
| bridge | 53.68 | 71.03 |
| bookcase | 31.93 | 45.69 |
| blind | 13.16 | 13.84 |
| coffee table | 42.48 | 82.19 |
| toilet | 76.66 | 88.33 |
| flower | 27.52 | 44.97 |
| book | 45.66 | 63.32 |
| hill | 5.1 | 6.94 |
| bench | 35.59 | 52.27 |
| countertop | 45.7 | 64.69 |
| stove | 54.31 | 81.51 |
| palm | 37.65 | 44.18 |
| kitchen island | 23.4 | 46.47 |
| computer | 65.78 | 85.77 |
| swivel chair | 34.23 | 53.39 |
| boat | 35.14 | 48.52 |
| bar | 29.98 | 50.07 |
| arcade machine | 43.42 | 51.7 |
| hovel | 47.34 | 80.74 |
| bus | 75.44 | 87.73 |
| towel | 47.09 | 55.34 |
| light | 36.51 | 43.89 |
| truck | 25.66 | 33.57 |
| tower | 12.96 | 15.76 |
| chandelier | 51.56 | 69.33 |
| awning | 9.87 | 13.87 |
| streetlight | 13.82 | 16.75 |
| booth | 32.93 | 35.33 |
| television receiver | 60.6 | 74.37 |
| airplane | 42.04 | 64.07 |
| dirt track | 3.96 | 12.2 |
| apparel | 12.62 | 21.14 |
| pole | 11.48 | 13.18 |
| land | 0.03 | 0.04 |
| bannister | 0.0 | 0.0 |
| escalator | 23.69 | 25.57 |
| ottoman | 16.67 | 17.56 |
| bottle | 29.2 | 59.57 |
| buffet | 33.85 | 43.35 |
| poster | 1.71 | 1.74 |
| stage | 0.03 | 0.03 |
| van | 19.24 | 25.25 |
| ship | 22.0 | 40.64 |
| fountain | 7.74 | 7.77 |
| conveyer belt | 41.21 | 65.97 |
| canopy | 11.4 | 18.57 |
| washer | 57.59 | 61.74 |
| plaything | 20.83 | 35.82 |
| swimming pool | 30.05 | 66.81 |
| stool | 16.5 | 20.21 |
| barrel | 50.84 | 65.64 |
| basket | 14.88 | 18.44 |
| waterfall | 48.3 | 52.43 |
| tent | 84.62 | 99.58 |
| bag | 0.14 | 0.14 |
| minibike | 38.45 | 47.0 |
| cradle | 60.99 | 83.33 |
| oven | 7.47 | 9.52 |
| ball | 40.9 | 64.2 |
| food | 50.44 | 58.98 |
| step | 0.17 | 0.17 |
| tank | 23.24 | 28.95 |
| trade name | 17.97 | 19.15 |
| microwave | 36.08 | 40.11 |
| pot | 30.84 | 33.93 |
| animal | 52.01 | 55.92 |
| bicycle | 44.17 | 77.74 |
| lake | 0.0 | 0.0 |
| dishwasher | 39.9 | 43.49 |
| screen | 63.04 | 82.44 |
| blanket | 0.0 | 0.0 |
| sculpture | 38.93 | 41.1 |
| hood | 37.46 | 45.77 |
| sconce | 3.74 | 3.79 |
| vase | 24.83 | 35.48 |
| traffic light | 16.38 | 25.55 |
| tray | 0.0 | 0.0 |
| ashcan | 35.33 | 45.91 |
| fan | 26.01 | 30.0 |
| pier | 23.5 | 24.92 |
| crt screen | 0.0 | 0.0 |
| plate | 40.71 | 61.43 |
| monitor | 3.16 | 3.2 |
| bulletin board | 32.23 | 35.5 |
| shower | 0.0 | 0.0 |
| radiator | 31.76 | 32.45 |
| glass | 2.23 | 2.28 |
| clock | 5.57 | 5.95 |
| flag | 17.32 | 18.43 |
+---------------------+-------+-------+
06/03 14:20:32 - mmengine - INFO - Iter(val) [500/500] aAcc: 79.1100 mIoU: 37.5800 mAcc: 49.5200 data_time: 0.0019 time: 0.6260
MzeroMiko
commented
3 weeks ago Can you check that if you have load the checkpoint pretrained with classification correctly? I checked the corresponding log in iter16000, and found that the mIoU is 42.05 rather than 37.58.
ShixuanGu
commented
3 weeks ago yea i think so, here's the log:
Successfully load ckpt /n/holylfs05/LABS/pfister_lab/Lab/coxfs01/pfister_lab2/Lab/shixuan/VMamba_block/VMamba/pretrained_model/vssm1_tiny_0230s_ckpt_epoch_264.pth
Successfully load ckpt /n/holylfs05/LABS/pfister_lab/Lab/coxfs01/pfister_lab2/Lab/shixuan/VMamba_block/VMamba/pretrained_model/vssm1_tiny_0230s_ckpt_epoch_264.pth
Successfully load ckpt /n/holylfs05/LABS/pfister_lab/Lab/coxfs01/pfister_lab2/Lab/shixuan/VMamba_block/VMamba/pretrained_model/vssm1_tiny_0230s_ckpt_epoch_264.pth
Successfully load ckpt /n/holylfs05/LABS/pfister_lab/Lab/coxfs01/pfister_lab2/Lab/shixuan/VMamba_block/VMamba/pretrained_model/vssm1_tiny_0230s_ckpt_epoch_264.pth
_IncompatibleKeys(missing_keys=['outnorm0.weight', 'outnorm0.bias', 'outnorm1.weight', 'outnorm1.bias', 'outnorm2.weight', 'outnorm2.bias', 'outnorm3.weight', 'outnorm3.bias'], unexpected_keys=['classifier.norm.weight', 'classifier.norm.bias', 'classifier.head.weight', 'classifier.head.bias'])
_IncompatibleKeys(missing_keys=['outnorm0.weight', 'outnorm0.bias', 'outnorm1.weight', 'outnorm1.bias', 'outnorm2.weight', 'outnorm2.bias', 'outnorm3.weight', 'outnorm3.bias'], unexpected_keys=['classifier.norm.weight', 'classifier.norm.bias', 'classifier.head.weight', 'classifier.head.bias'])
_IncompatibleKeys(missing_keys=['outnorm0.weight', 'outnorm0.bias', 'outnorm1.weight', 'outnorm1.bias', 'outnorm2.weight', 'outnorm2.bias', 'outnorm3.weight', 'outnorm3.bias'], unexpected_keys=['classifier.norm.weight', 'classifier.norm.bias', 'classifier.head.weight', 'classifier.head.bias'])
_IncompatibleKeys(missing_keys=['outnorm0.weight', 'outnorm0.bias', 'outnorm1.weight', 'outnorm1.bias', 'outnorm2.weight', 'outnorm2.bias', 'outnorm3.weight', 'outnorm3.bias'], unexpected_keys=['classifier.norm.weight', 'classifier.norm.bias', 'classifier.head.weight', 'classifier.head.bias'])
/n/home02/shixuang/miniconda3/envs/vm/lib/python3.12/site-packages/mmseg/models/builder.py:36: UserWarning: build_loss would be deprecated soon, please use mmseg.registry.MODELS.build()
warnings.warn('build_loss would be deprecated soon, please use '
/n/home02/shixuang/miniconda3/envs/vm/lib/python3.12/site-packages/mmseg/models/builder.py:36: UserWarning: build_loss would be deprecated soon, please use mmseg.registry.MODELS.build()
warnings.warn('build_loss would be deprecated soon, please use '
/n/home02/shixuang/miniconda3/envs/vm/lib/python3.12/site-packages/mmseg/models/builder.py:36: UserWarning: build_loss would be deprecated soon, please use mmseg.registry.MODELS.build()
warnings.warn('build_loss would be deprecated soon, please use '
/n/home02/shixuang/miniconda3/envs/vm/lib/python3.12/site-packages/mmseg/models/builder.py:36: UserWarning: build_loss would be deprecated soon, please use mmseg.registry.MODELS.build()
warnings.warn('build_loss would be deprecated soon, please use '
/n/home02/shixuang/miniconda3/envs/vm/lib/python3.12/site-packages/mmseg/models/losses/cross_entropy_loss.py:250: UserWarning: Default avg_non_ignore is False, if you would like to ignore the certain label and average loss over non-ignore labels, which is the same with PyTorch official cross_entropy, set avg_non_ignore=True.
warnings.warn(
/n/home02/shixuang/miniconda3/envs/vm/lib/python3.12/site-packages/mmseg/models/losses/cross_entropy_loss.py:250: UserWarning: Default avg_non_ignore is False, if you would like to ignore the certain label and average loss over non-ignore labels, which is the same with PyTorch official cross_entropy, set avg_non_ignore=True.
warnings.warn(
/n/home02/shixuang/miniconda3/envs/vm/lib/python3.12/site-packages/mmseg/models/losses/cross_entropy_loss.py:250: UserWarning: Default avg_non_ignore is False, if you would like to ignore the certain label and average loss over non-ignore labels, which is the same with PyTorch official cross_entropy, set avg_non_ignore=True.
warnings.warn(
/n/home02/shixuang/miniconda3/envs/vm/lib/python3.12/site-packages/mmseg/models/losses/cross_entropy_loss.py:250: UserWarning: Default avg_non_ignore is False, if you would like to ignore the certain label and average loss over non-ignore labels, which is the same with PyTorch official cross_entropy, set avg_non_ignore=True.
warnings.warn(
/n/home02/shixuang/miniconda3/envs/vm/lib/python3.12/site-packages/mmseg/engine/hooks/visualization_hook.py:60: UserWarning: The draw is False, it means that the hook for visualization will not take effect. The results will NOT be visualized or stored.
warnings.warn('The draw is False, it means that the '
/n/home02/shixuang/miniconda3/envs/vm/lib/python3.12/site-packages/mmseg/engine/hooks/visualization_hook.py:60: UserWarning: The draw is False, it means that the hook for visualization will not take effect. The results will NOT be visualized or stored.
warnings.warn('The draw is False, it means that the '
/n/home02/shixuang/miniconda3/envs/vm/lib/python3.12/site-packages/mmseg/engine/hooks/visualization_hook.py:60: UserWarning: The draw is False, it means that the hook for visualization will not take effect. The results will NOT be visualized or stored.
warnings.warn('The draw is False, it means that the '
/n/home02/shixuang/miniconda3/envs/vm/lib/python3.12/site-packages/mmseg/engine/hooks/visualization_hook.py:60: UserWarning: The draw is False, it means that the hook for visualization will not take effect. The results will NOT be visualized or stored.
warnings.warn('The draw is False, it means that the '
06/03 13:29:22 - mmengine - INFO - Hooks will be executed in the following order:
before_run:
(VERY_HIGH ) RuntimeInfoHook
(BELOW_NORMAL) LoggerHook
before_train:
(VERY_HIGH ) RuntimeInfoHook
(NORMAL ) IterTimerHook
(VERY_LOW ) CheckpointHook
before_train_epoch:
(VERY_HIGH ) RuntimeInfoHook
(NORMAL ) IterTimerHook
(NORMAL ) DistSamplerSeedHook
before_train_iter:
(VERY_HIGH ) RuntimeInfoHook
(NORMAL ) IterTimerHook
after_train_iter:
(VERY_HIGH ) RuntimeInfoHook
(NORMAL ) IterTimerHook
(NORMAL ) SegVisualizationHook
(BELOW_NORMAL) LoggerHook
(LOW ) ParamSchedulerHook
(VERY_LOW ) CheckpointHook
after_train_epoch:
(NORMAL ) IterTimerHook
(LOW ) ParamSchedulerHook
(VERY_LOW ) CheckpointHook
before_val: (VERY_HIGH ) RuntimeInfoHook
before_val_epoch: (NORMAL ) IterTimerHook
before_val_iter: (NORMAL ) IterTimerHook
after_val_iter:
(NORMAL ) IterTimerHook
(NORMAL ) SegVisualizationHook
(BELOW_NORMAL) LoggerHook
after_val_epoch:
(VERY_HIGH ) RuntimeInfoHook
(NORMAL ) IterTimerHook
(BELOW_NORMAL) LoggerHook
(LOW ) ParamSchedulerHook
(VERY_LOW ) CheckpointHook
after_val: (VERY_HIGH ) RuntimeInfoHook
after_train:
(VERY_HIGH ) RuntimeInfoHook
(VERY_LOW ) CheckpointHook
before_test: (VERY_HIGH ) RuntimeInfoHook
before_test_epoch: (NORMAL ) IterTimerHook
before_test_iter: (NORMAL ) IterTimerHook
after_test_iter:
(NORMAL ) IterTimerHook
(NORMAL ) SegVisualizationHook
(BELOW_NORMAL) LoggerHook
after_test_epoch:
(VERY_HIGH ) RuntimeInfoHook
(NORMAL ) IterTimerHook
(BELOW_NORMAL) LoggerHook
after_test: (VERY_HIGH ) RuntimeInfoHook
after_run: (BELOW_NORMAL) LoggerHook
/n/home02/shixuang/miniconda3/envs/vm/lib/python3.12/site-packages/mmseg/datasets/transforms/loading.py:83: UserWarning: reduce_zero_label will be deprecated, if you would like to ignore the zero label, please set reduce_zero_label=True when dataset initialized
warnings.warn('reduce_zero_label will be deprecated, '
/n/home02/shixuang/miniconda3/envs/vm/lib/python3.12/site-packages/mmseg/datasets/transforms/loading.py:83: UserWarning: reduce_zero_label will be deprecated, if you would like to ignore the zero label, please set reduce_zero_label=True when dataset initialized
warnings.warn('reduce_zero_label will be deprecated, '
/n/home02/shixuang/miniconda3/envs/vm/lib/python3.12/site-packages/mmseg/datasets/transforms/loading.py:83: UserWarning: reduce_zero_label will be deprecated, if you would like to ignore the zero label, please set reduce_zero_label=True when dataset initialized
warnings.warn('reduce_zero_label will be deprecated, '
/n/home02/shixuang/miniconda3/envs/vm/lib/python3.12/site-packages/mmseg/datasets/transforms/loading.py:83: UserWarning: reduce_zero_label will be deprecated, if you would like to ignore the zero label, please set reduce_zero_label=True when dataset initialized
warnings.warn('reduce_zero_label will be deprecated, '
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.0.norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.0.norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.0.norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.0.norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.0.norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.0.norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.0.op.out_norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.0.op.out_norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.0.op.out_norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.0.op.out_norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.0.op.out_norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.0.op.out_norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.0.norm2.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.0.norm2.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.0.norm2.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.0.norm2.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.0.norm2.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.0.norm2.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.1.norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.1.norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.1.norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.1.norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.1.norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.1.norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.1.op.out_norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.1.op.out_norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.1.op.out_norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.1.op.out_norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.1.op.out_norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.1.op.out_norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.1.norm2.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.1.norm2.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.1.norm2.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.1.norm2.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.1.norm2.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.0.blocks.1.norm2.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.0.norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.0.norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.0.norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.0.norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.0.norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.0.norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.0.op.out_norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.0.op.out_norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.0.op.out_norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.0.op.out_norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.0.op.out_norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.0.op.out_norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.0.norm2.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.0.norm2.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.0.norm2.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.0.norm2.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.0.norm2.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.0.norm2.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.1.norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.1.norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.1.norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.1.norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.1.norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.1.norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.1.op.out_norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.1.op.out_norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.1.op.out_norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.1.op.out_norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.1.op.out_norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.1.op.out_norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.1.norm2.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.1.norm2.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.1.norm2.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.1.norm2.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.1.norm2.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.1.blocks.1.norm2.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.0.norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.0.norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.0.norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.0.norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.0.norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.0.norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.0.op.out_norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.0.op.out_norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.0.op.out_norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.0.op.out_norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.0.op.out_norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.0.op.out_norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.0.norm2.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.0.norm2.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.0.norm2.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.0.norm2.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.0.norm2.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.0.norm2.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.1.norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.1.norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.1.norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.1.norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.1.norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.1.norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.1.op.out_norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.1.op.out_norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.1.op.out_norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.1.op.out_norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.1.op.out_norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.1.op.out_norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.1.norm2.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.1.norm2.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.1.norm2.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.1.norm2.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.1.norm2.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.1.norm2.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.2.norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.2.norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.2.norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.2.norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.2.norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.2.norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.2.op.out_norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.2.op.out_norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.2.op.out_norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.2.op.out_norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.2.op.out_norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.2.op.out_norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.2.norm2.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.2.norm2.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.2.norm2.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.2.norm2.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.2.norm2.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.2.norm2.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.3.norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.3.norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.3.norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.3.norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.3.norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.3.norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.3.op.out_norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.3.op.out_norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.3.op.out_norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.3.op.out_norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.3.op.out_norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.3.op.out_norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.3.norm2.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.3.norm2.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.3.norm2.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.3.norm2.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.3.norm2.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.3.norm2.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.4.norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.4.norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.4.norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.4.norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.4.norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.4.norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.4.op.out_norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.4.op.out_norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.4.op.out_norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.4.op.out_norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.4.op.out_norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.4.op.out_norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.4.norm2.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.4.norm2.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.4.norm2.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.4.norm2.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.4.norm2.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.4.norm2.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.5.norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.5.norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.5.norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.5.norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.5.norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.5.norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.5.op.out_norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.5.op.out_norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.5.op.out_norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.5.op.out_norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.5.op.out_norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.5.op.out_norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.5.norm2.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.5.norm2.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.5.norm2.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.5.norm2.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.5.norm2.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.5.norm2.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.6.norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.6.norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.6.norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.6.norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.6.norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.6.norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.6.op.out_norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.6.op.out_norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.6.op.out_norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.6.op.out_norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.6.op.out_norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.6.op.out_norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.6.norm2.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.6.norm2.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.6.norm2.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.6.norm2.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.6.norm2.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.6.norm2.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.7.norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.7.norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.7.norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.7.norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.7.norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.7.norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.7.op.out_norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.7.op.out_norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.7.op.out_norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.7.op.out_norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.7.op.out_norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.7.op.out_norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.7.norm2.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.7.norm2.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.7.norm2.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.7.norm2.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.7.norm2.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.2.blocks.7.norm2.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.0.norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.0.norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.0.norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.0.norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.0.norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.0.norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.0.op.out_norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.0.op.out_norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.0.op.out_norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.0.op.out_norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.0.op.out_norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.0.op.out_norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.0.norm2.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.0.norm2.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.0.norm2.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.0.norm2.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.0.norm2.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.0.norm2.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.1.norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.1.norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.1.norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.1.norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.1.norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.1.norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.1.op.out_norm.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.1.op.out_norm.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.1.op.out_norm.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.1.op.out_norm.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.1.op.out_norm.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.1.op.out_norm.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.1.norm2.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.1.norm2.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.1.norm2.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.1.norm2.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.1.norm2.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.layers.3.blocks.1.norm2.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.outnorm0.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.outnorm0.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.outnorm0.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.outnorm0.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.outnorm0.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.outnorm0.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.outnorm1.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.outnorm1.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.outnorm1.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.outnorm1.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.outnorm1.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.outnorm1.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.outnorm2.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.outnorm2.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.outnorm2.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.outnorm2.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.outnorm2.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.outnorm2.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.outnorm3.weight:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.outnorm3.weight:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.outnorm3.weight:decay_mult=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.outnorm3.bias:lr=6e-05
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.outnorm3.bias:weight_decay=0.0
06/03 13:29:23 - mmengine - INFO - paramwise_options -- backbone.outnorm3.bias:decay_mult=0.0
06/03 13:29:23 - mmengine - WARNING - The prefix is not set in metric class IoUMetric.
06/03 13:29:24 - mmengine - WARNING - "FileClient" will be deprecated in future. Please use io functions in https://mmengine.readthedocs.io/en/latest/api/fileio.html#file-io
06/03 13:29:24 - mmengine - WARNING - "HardDiskBackend" is the alias of "LocalBackend" and the former will be deprecated in future.
06/03 13:29:24 - mmengine - INFO - Checkpoints will be saved to /n/holylfs05/LABS/pfister_lab/Lab/coxfs01/pfister_lab2/Lab/shixuan/VMamba_block/VMamba/segmentation/exp_log/ft_cs.
/n/home02/shixuang/miniconda3/envs/vm/lib/python3.12/site-packages/torch/autograd/init.py:266: UserWarning: Grad strides do not match bucket view strides. This may indicate grad was not created according to the gradient layout contract, or that the param's strides changed since DDP was constructed. This is not an error, but may impair performance.
grad.sizes() = [150, 256, 1, 1], strides() = [256, 1, 256, 256]
bucket_view.sizes() = [150, 256, 1, 1], strides() = [256, 1, 1, 1] (Triggered internally at /opt/conda/conda-bld/pytorch_1711403463728/work/torch/csrc/distributed/c10d/reducer.cpp:322.)
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
/n/home02/shixuang/miniconda3/envs/vm/lib/python3.12/site-packages/torch/autograd/init.py:266: UserWarning: Grad strides do not match bucket view strides. This may indicate grad was not created according to the gradient layout contract, or that the param's strides changed since DDP was constructed. This is not an error, but may impair performance.
grad.sizes() = [150, 256, 1, 1], strides() = [256, 1, 256, 256]
bucket_view.sizes() = [150, 256, 1, 1], strides() = [256, 1, 1, 1] (Triggered internally at /opt/conda/conda-bld/pytorch_1711403463728/work/torch/csrc/distributed/c10d/reducer.cpp:322.)
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
/n/home02/shixuang/miniconda3/envs/vm/lib/python3.12/site-packages/torch/autograd/init.py:266: UserWarning: Grad strides do not match bucket view strides. This may indicate grad was not created according to the gradient layout contract, or that the param's strides changed since DDP was constructed. This is not an error, but may impair performance.
grad.sizes() = [150, 256, 1, 1], strides() = [256, 1, 256, 256]
bucket_view.sizes() = [150, 256, 1, 1], strides() = [256, 1, 1, 1] (Triggered internally at /opt/conda/conda-bld/pytorch_1711403463728/work/torch/csrc/distributed/c10d/reducer.cpp:322.)
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
/n/home02/shixuang/miniconda3/envs/vm/lib/python3.12/site-packages/torch/autograd/init.py:266: UserWarning: Grad strides do not match bucket view strides. This may indicate grad was not created according to the gradient layout contract, or that the param's strides changed since DDP was constructed. This is not an error, but may impair performance.
grad.sizes() = [150, 256, 1, 1], strides() = [256, 1, 256, 256]
bucket_view.sizes() = [150, 256, 1, 1], strides() = [256, 1, 1, 1] (Triggered internally at /opt/conda/conda-bld/pytorch_1711403463728/work/torch/csrc/distributed/c10d/reducer.cpp:322.)
Variable._execution_engine.run_backward( # Calls into the C++ engine to run the backward pass
06/03 13:30:03 - mmengine - INFO - Iter(train) [ 50/160000] base_lr: 1.9614e-06 lr: 1.9614e-06 eta: 1 day, 11:11:16 time: 0.1732 data_time: 0.0049 memory: 76621 loss: 6.2530 decode.loss_ce: 4.4636 decode.acc_seg: 0.6089 aux.loss_ce: 1.7894 aux.acc_seg: 0.1447
06/03 13:30:12 - mmengine - INFO - Iter(train) [ 100/160000] base_lr: 3.9627e-06 lr: 3.9627e-06 eta: 21:25:54 time: 0.1732 data_time: 0.0051 memory: 4748 loss: 6.0579 decode.loss_ce: 4.3081 decode.acc_seg: 0.4280 aux.loss_ce: 1.7497 aux.acc_seg: 3.0789
06/03 13:30:21 - mmengine - INFO - Iter(train) [ 150/160000] base_lr: 5.9640e-06 lr: 5.9640e-06 eta: 16:50:46 time: 0.1724 data_time: 0.0047 memory: 4748 loss: 5.7547 decode.loss_ce: 4.0519 decode.acc_seg: 13.3740 aux.loss_ce: 1.7028 aux.acc_seg: 2.2598
MzeroMiko
commented
3 weeks ago Maybe it is because the batchsize differs?
I noticed that in the log you provide, it says:
06/03 14:20:04 - mmengine - INFO - Iter(val) [500/500] eta: 0:00:00 time: 1.0236 data_time: 0.0012 memory: 75480while in mine, the total iter is 250, which may suggest that you used a smaller batch_size than mine.
2024/05/09 13:04:01 - mmengine - INFO - Iter(val) [250/250] eta: 0:00:00 time: 0.0413 data_time: 0.0017 memory: 2627 Changing batchsize solved the issue, many thanks!
Would you kindly report the variance of segmentation results for tinys1l8 on mIoU(SS)? It seems the fine tuning result might have a certain variance, with the same pth, I only got 47.1 mIoU(SS).
06/03 21:58:45 - mmengine - INFO - Saving checkpoint at 160000 iterations 06/03 21:58:48 - mmengine - INFO - Iter(val) [ 50/500] eta: 0:00:15 time: 0.0337 data_time: 0.0011 memory: 1789
06/03 21:58:50 - mmengine - INFO - Iter(val) [100/500] eta: 0:00:13 time: 0.0331 data_time: 0.0010 memory: 1868
06/03 21:58:52 - mmengine - INFO - Iter(val) [150/500] eta: 0:00:11 time: 0.0336 data_time: 0.0011 memory: 1594
06/03 21:58:53 - mmengine - INFO - Iter(val) [200/500] eta: 0:00:10 time: 0.0331 data_time: 0.0010 memory: 1668
06/03 21:58:55 - mmengine - INFO - Iter(val) [250/500] eta: 0:00:08 time: 0.0342 data_time: 0.0012 memory: 1636
06/03 21:58:57 - mmengine - INFO - Iter(val) [300/500] eta: 0:00:06 time: 0.0340 data_time: 0.0013 memory: 2610
06/03 21:58:59 - mmengine - INFO - Iter(val) [350/500] eta: 0:00:05 time: 0.0329 data_time: 0.0011 memory: 1594
06/03 21:59:00 - mmengine - INFO - Iter(val) [400/500] eta: 0:00:03 time: 0.0334 data_time: 0.0011 memory: 1627
06/03 21:59:02 - mmengine - INFO - Iter(val) [450/500] eta: 0:00:01 time: 0.0337 data_time: 0.0011 memory: 1643
06/03 21:59:04 - mmengine - INFO - Iter(val) [500/500] eta: 0:00:00 time: 0.0333 data_time: 0.0010 memory: 1857
06/03 21:59:06 - mmengine - INFO - per class results: 06/03 21:59:06 - mmengine - INFO - +---------------------+-------+-------+ | Class | IoU | Acc | +---------------------+-------+-------+ | wall | 76.56 | 88.21 | | building | 82.17 | 91.57 | | sky | 94.07 | 97.39 | | floor | 80.72 | 90.83 | | tree | 74.37 | 87.45 | | ceiling | 84.1 | 91.8 | | road | 83.44 | 89.96 | | bed | 87.58 | 95.44 | | windowpane | 60.53 | 76.88 | | grass | 66.84 | 81.72 | | cabinet | 59.24 | 70.35 | | sidewalk | 65.52 | 82.74 | | person | 79.28 | 90.91 | | earth | 35.16 | 50.57 | | door | 46.14 | 59.65 | | table | 57.42 | 73.76 | | mountain | 58.33 | 72.43 | | plant | 51.81 | 63.8 | | curtain | 73.27 | 84.31 | | chair | 53.73 | 66.52 | | car | 83.54 | 91.12 | | water | 51.89 | 68.62 | | painting | 71.89 | 86.33 | | sofa | 63.81 | 79.67 | | shelf | 37.43 | 55.24 | | house | 43.91 | 64.91 | | sea | 49.77 | 71.57 | | mirror | 67.69 | 77.68 | | rug | 60.14 | 68.75 | | field | 32.75 | 50.55 | | armchair | 38.95 | 56.56 | | seat | 62.39 | 80.7 | | fence | 40.75 | 56.26 | | desk | 45.26 | 68.17 | | rock | 41.67 | 66.83 | | wardrobe | 46.45 | 66.27 | | lamp | 59.79 | 71.94 | | bathtub | 73.95 | 81.16 | | railing | 35.65 | 47.87 | | cushion | 56.29 | 70.18 | | base | 26.91 | 40.25 | | box | 24.37 | 32.06 | | column | 46.33 | 57.02 | | signboard | 35.78 | 48.07 | | chest of drawers | 38.5 | 60.27 | | counter | 37.34 | 44.37 | | sand | 42.15 | 58.57 | | sink | 69.05 | 78.22 | | skyscraper | 60.29 | 76.34 | | fireplace | 75.61 | 90.32 | | refrigerator | 73.0 | 82.24 | | grandstand | 42.13 | 68.22 | | path | 29.02 | 41.26 | | stairs | 28.68 | 36.57 | | runway | 68.59 | 94.96 | | case | 42.58 | 62.0 | | pool table | 91.64 | 96.29 | | pillow | 54.62 | 62.69 | | screen door | 69.26 | 78.51 | | stairway | 27.27 | 34.16 | | river | 11.99 | 22.12 | | bridge | 28.49 | 34.33 | | bookcase | 40.23 | 64.0 | | blind | 41.81 | 49.08 | | coffee table | 52.27 | 77.9 | | toilet | 82.0 | 89.15 | | flower | 44.19 | 55.59 | | book | 45.39 | 67.3 | | hill | 11.65 | 21.12 | | bench | 43.48 | 53.27 | | countertop | 56.31 | 72.1 | | stove | 75.3 | 81.04 | | palm | 49.34 | 70.37 | | kitchen island | 35.54 | 68.26 | | computer | 60.59 | 73.25 | | swivel chair | 43.51 | 57.38 | | boat | 46.18 | 53.38 | | bar | 35.94 | 50.48 | | arcade machine | 47.77 | 52.44 | | hovel | 12.21 | 14.83 | | bus | 86.25 | 96.04 | | towel | 62.59 | 74.43 | | light | 52.68 | 60.26 | | truck | 23.62 | 28.02 | | tower | 34.57 | 48.77 | | chandelier | 65.68 | 79.97 | | awning | 23.85 | 28.48 | | streetlight | 24.89 | 32.14 | | booth | 46.41 | 48.75 | | television receiver | 69.37 | 80.44 | | airplane | 55.54 | 64.37 | | dirt track | 7.68 | 13.88 | | apparel | 38.39 | 52.77 | | pole | 22.21 | 30.71 | | land | 2.42 | 3.18 | | bannister | 9.67 | 12.92 | | escalator | 23.15 | 27.28 | | ottoman | 40.71 | 57.33 | | bottle | 34.82 | 58.07 | | buffet | 45.6 | 58.89 | | poster | 25.71 | 35.74 | | stage | 19.22 | 28.42 | | van | 44.19 | 63.16 | | ship | 49.3 | 72.98 | | fountain | 19.62 | 20.41 | | conveyer belt | 81.77 | 91.8 | | canopy | 15.42 | 18.41 | | washer | 67.93 | 70.48 | | plaything | 17.91 | 26.68 | | swimming pool | 51.12 | 63.27 | | stool | 37.42 | 56.01 | | barrel | 44.47 | 64.77 | | basket | 27.85 | 41.94 | | waterfall | 49.95 | 59.44 | | tent | 80.56 | 98.22 | | bag | 11.34 | 13.47 | | minibike | 67.39 | 82.21 | | cradle | 76.26 | 92.09 | | oven | 30.67 | 65.1 | | ball | 25.98 | 26.59 | | food | 33.19 | 40.36 | | step | 11.48 | 14.51 | | tank | 50.44 | 57.61 | | trade name | 28.77 | 33.4 | | microwave | 56.41 | 62.86 | | pot | 38.42 | 45.16 | | animal | 53.43 | 55.88 | | bicycle | 55.99 | 75.31 | | lake | 60.09 | 63.02 | | dishwasher | 70.64 | 81.01 | | screen | 69.63 | 79.2 | | blanket | 9.19 | 10.63 | | sculpture | 49.26 | 76.69 | | hood | 60.53 | 68.21 | | sconce | 43.28 | 52.17 | | vase | 33.14 | 45.48 | | traffic light | 25.63 | 44.32 | | tray | 3.16 | 7.58 | | ashcan | 43.88 | 52.82 | | fan | 56.37 | 69.33 | | pier | 63.08 | 77.07 | | crt screen | 4.71 | 11.84 | | plate | 52.65 | 67.62 | | monitor | 6.42 | 8.13 | | bulletin board | 46.63 | 58.2 | | shower | 0.05 | 0.18 | | radiator | 59.11 | 64.59 | | glass | 11.54 | 12.28 | | clock | 28.1 | 31.69 | | flag | 43.86 | 47.88 | +---------------------+-------+-------+ 06/03 21:59:06 - mmengine - INFO - Iter(val) [500/500] aAcc: 82.3300 mIoU: 47.1000 mAcc: 58.8000 data_time: 0.0011 time: 0.0335