The test performance is different from the paper

Hi，Thank you for your interesting work, I ran into a issue when trying to test the generalization performance, when using the trained model for testing, the performance is quite different from the one reported in your paper, what could be the problem? I used "python tools/test.py configs/dinov2/rein_dinov2_mask2former_512x512_bs1x4.py checkpoints/dinov2_rein_and_head.pth --backbone checkpoints/dinov2_converted.pth" , and the log is as follows:

2024/07/30 15:19:08 - mmengine - INFO -

System environment: sys.platform: linux Python: 3.11.9 (main, Apr 19 2024, 16:48:06) [GCC 11.2.0] CUDA available: True MUSA available: False numpy_random_seed: 42 GPU 0,1: NVIDIA GeForce RTX 3090 CUDA_HOME: /usr/local/cuda-10.2 NVCC: Cuda compilation tools, release 10.2, V10.2.8 GCC: gcc (Ubuntu 9.5.0-1ubuntu1~22.04) 9.5.0 PyTorch: 2.0.1 PyTorch compiling details: PyTorch built with:

GCC 9.3
C++ Version: 201703
Intel(R) oneAPI Math Kernel Library Version 2023.1-Product Build 20230303 for Intel(R) 64 architecture applications
Intel(R) MKL-DNN v2.7.3 (Git Hash 6dbeffbae1f23cbbeae17adb7b5b13f1f37c080e)
OpenMP 201511 (a.k.a. OpenMP 4.5)
LAPACK is enabled (usually provided by MKL)
NNPACK is enabled
CPU capability usage: AVX2
CUDA Runtime 11.7
NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37
CuDNN 8.5
Magma 2.6.1
Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.7, CUDNN_VERSION=8.5.0, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 -fabi-version=11 -Wno-deprecated -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wunused-local-typedefs -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-psabi -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Werror=cast-function-type -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_DISABLE_GPU_ASSERTS=ON, TORCH_VERSION=2.0.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=ON, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF,

TorchVision: 0.15.2 OpenCV: 4.10.0 MMEngine: 0.10.4

Runtime environment: cudnn_benchmark: True mp_cfg: {'mp_start_method': 'fork', 'opencv_num_threads': 0} dist_cfg: {'backend': 'nccl'} seed: 42 Distributed launcher: none Distributed training: False GPU number: 1

2024/07/30 15:19:09 - mmengine - INFO - Config: bdd_crop_size = ( 512, 512, ) bdd_root = '/dataset2/gjw/DG_SEG/bdd100k/' bdd_test_pipeline = [ dict(type='LoadImageFromFile'), dict(keep_ratio=True, scale=( 1280, 720, ), type='Resize'), dict(type='LoadAnnotations'), dict(type='PackSegInputs'), ] bdd_train_pipeline = [ dict(type='LoadImageFromFile'), dict(type='LoadAnnotations'), dict(scale=( 1280, 720, ), type='Resize'), dict(cat_max_ratio=0.75, crop_size=( 512, 512, ), type='RandomCrop'), dict(prob=0.5, type='RandomFlip'), dict(type='PhotoMetricDistortion'), dict(type='PackSegInputs'), ] bdd_type = 'CityscapesDataset' cityscapes_crop_size = ( 512, 512, ) cityscapes_root = '/dataset2/gjw/DG_SEG/cityscapes/' cityscapes_test_pipeline = [ dict(type='LoadImageFromFile'), dict(keep_ratio=True, scale=( 1024, 512, ), type='Resize'), dict(type='LoadAnnotations'), dict(type='PackSegInputs'), ] cityscapes_train_pipeline = [ dict(type='LoadImageFromFile'), dict(type='LoadAnnotations'), dict(scale=( 1024, 512, ), type='Resize'), dict(cat_max_ratio=0.75, crop_size=( 512, 512, ), type='RandomCrop'), dict(prob=0.5, type='RandomFlip'), dict(type='PhotoMetricDistortion'), dict(type='PackSegInputs'), ] cityscapes_type = 'CityscapesDataset' crop_size = ( 512, 512, ) custom_hooks = [ dict( checkpoint_path='checkpoints/dinov2_converted.pth', type='LoadBackboneHook'), ] default_hooks = dict( checkpoint=dict( by_epoch=False, interval=4000, max_keep_ckpts=5, rule='greater', save_best=[ 'mIoU', ], type='CheckpointHook'), logger=dict(interval=50, log_metric_by_epoch=False, type='LoggerHook'), param_scheduler=dict(type='ParamSchedulerHook'), sampler_seed=dict(type='DistSamplerSeedHook'), timer=dict(type='IterTimerHook'), visualization=dict(type='SegVisualizationHook')) default_scope = 'mmseg' embed_multi = dict(decay_mult=0.0, lr_mult=1.0) env_cfg = dict( cudnn_benchmark=True, dist_cfg=dict(backend='nccl'), mp_cfg=dict(mp_start_method='fork', opencv_num_threads=0)) gta_crop_size = ( 512, 512, ) gta_root = '/dataset2/gjw/DG_SEG/gta/' gta_test_pipeline = [ dict(type='LoadImageFromFile'), dict(keep_ratio=True, scale=( 1280, 720, ), type='Resize'), dict(type='LoadAnnotations'), dict(type='PackSegInputs'), ] gta_train_pipeline = [ dict(type='LoadImageFromFile'), dict(type='LoadAnnotations'), dict(scale=( 1280, 720, ), type='Resize'), dict(cat_max_ratio=0.75, crop_size=( 512, 512, ), type='RandomCrop'), dict(prob=0.5, type='RandomFlip'), dict(type='PhotoMetricDistortion'), dict(type='PackSegInputs'), ] gta_type = 'CityscapesDataset' launcher = 'none' load_from = 'checkpoints/dinov2_rein_and_head.pth' log_level = 'INFO' log_processor = dict(by_epoch=False) mapillary_crop_size = ( 512, 512, ) mapillary_root = '/dataset2/gjw/DG_SEG/mapillary/' mapillary_test_pipeline = [ dict(type='LoadImageFromFile'), dict(keep_ratio=True, scale=( 1024, 512, ), type='Resize'), dict(type='LoadAnnotations'), dict(type='PackSegInputs'), ] mapillary_train_pipeline = [ dict(type='LoadImageFromFile'), dict(type='LoadAnnotations'), dict(scale=( 1024, 512, ), type='Resize'), dict(cat_max_ratio=0.75, crop_size=( 512, 512, ), type='RandomCrop'), dict(prob=0.5, type='RandomFlip'), dict(type='PhotoMetricDistortion'), dict(type='PackSegInputs'), ] mapillary_type = 'CityscapesDataset' model = dict( backbone=dict( block_chunks=0, depth=24, embed_dim=1024, ffn_bias=True, ffn_layer='mlp', img_size=512, init_cfg=dict( checkpoint='checkpoints/dinov2_converted.pth', type='Pretrained'), init_values=1e-05, mlp_ratio=4, num_heads=16, patch_size=16, proj_bias=True, qkv_bias=True, reins_config=dict( embed_dims=1024, link_token_to_query=True, lora_dim=16, num_layers=24, patch_size=16, token_length=100, type='LoRAReins'), type='ReinsDinoVisionTransformer'), data_preprocessor=dict( bgr_to_rgb=True, mean=[ 123.675, 116.28, 103.53, ], pad_val=0, seg_pad_val=255, size=( 512, 512, ), std=[ 58.395, 57.12, 57.375, ], type='SegDataPreProcessor'), decode_head=dict( align_corners=False, enforce_decoder_input_project=False, feat_channels=256, in_channels=[ 1024, 1024, 1024, 1024, ], loss_cls=dict( class_weight=[ 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.1, ], loss_weight=2.0, reduction='mean', type='mmdet.CrossEntropyLoss', use_sigmoid=False), loss_dice=dict( activate=True, eps=1.0, loss_weight=5.0, naive_dice=True, reduction='mean', type='mmdet.DiceLoss', use_sigmoid=True), loss_mask=dict( loss_weight=5.0, reduction='mean', type='mmdet.CrossEntropyLoss', use_sigmoid=True), num_classes=19, num_queries=100, num_transformer_feat_level=3, out_channels=256, pixel_decoder=dict( act_cfg=dict(type='ReLU'), encoder=dict( init_cfg=None, layer_cfg=dict( ffn_cfg=dict( act_cfg=dict(inplace=True, type='ReLU'), embed_dims=256, feedforward_channels=1024, ffn_drop=0.0, num_fcs=2), self_attn_cfg=dict( batch_first=True, dropout=0.0, embed_dims=256, im2col_step=64, init_cfg=None, norm_cfg=None, num_heads=8, num_levels=3, num_points=4)), num_layers=6), init_cfg=None, norm_cfg=dict(num_groups=32, type='GN'), num_outs=3, positional_encoding=dict(normalize=True, num_feats=128), type='mmdet.MSDeformAttnPixelDecoder'), positional_encoding=dict(normalize=True, num_feats=128), replace_query_feat=False, strides=[ 4, 8, 16, 32, ], train_cfg=dict( assigner=dict( match_costs=[ dict(type='mmdet.ClassificationCost', weight=2.0), dict( type='mmdet.CrossEntropyLossCost', use_sigmoid=True, weight=5.0), dict( eps=1.0, pred_act=True, type='mmdet.DiceCost', weight=5.0), ], type='mmdet.HungarianAssigner'), importance_sample_ratio=0.75, num_points=12544, oversample_ratio=3.0, sampler=dict(type='mmdet.MaskPseudoSampler')), transformer_decoder=dict( init_cfg=None, layer_cfg=dict( cross_attn_cfg=dict( attn_drop=0.0, batch_first=True, dropout_layer=None, embed_dims=256, num_heads=8, proj_drop=0.0), ffn_cfg=dict( act_cfg=dict(inplace=True, type='ReLU'), add_identity=True, dropout_layer=None, embed_dims=256, feedforward_channels=2048, ffn_drop=0.0, num_fcs=2), self_attn_cfg=dict( attn_drop=0.0, batch_first=True, dropout_layer=None, embed_dims=256, num_heads=8, proj_drop=0.0)), num_layers=9, return_intermediate=True), type='ReinMask2FormerHead'), test_cfg=dict(crop_size=( 512, 512, ), mode='slide', stride=( 341, 341, )), train_cfg=dict(), type='EncoderDecoder') num_classes = 19 optim_wrapper = dict( constructor='PEFTOptimWrapperConstructor', optimizer=dict( betas=( 0.9, 0.999, ), eps=1e-08, lr=0.0001, type='AdamW', weight_decay=0.05), paramwise_cfg=dict( custom_keys=dict({ 'learnable_tokens': dict(decay_mult=0.0, lr_mult=1.0), 'level_embed': dict(decay_mult=0.0, lr_mult=1.0), 'norm': dict(decay_mult=0.0), 'query_embed': dict(decay_mult=0.0, lr_mult=1.0), 'reins.scale': dict(decay_mult=0.0, lr_mult=1.0) }), norm_decay_mult=0.0)) param_scheduler = [ dict( begin=0, by_epoch=False, end=40000, eta_min=0, power=0.9, type='PolyLR'), ] randomness = dict(seed=42) resume = False test_cfg = dict(type='TestLoop') test_dataloader = dict( batch_size=1, dataset=dict( datasets=[ dict( data_prefix=dict( img_path='leftImg8bit/val', seg_map_path='gtFine/val'), data_root='/dataset2/gjw/DG_SEG/cityscapes/', pipeline=[ dict(type='LoadImageFromFile'), dict(keep_ratio=True, scale=( 1024, 512, ), type='Resize'), dict(type='LoadAnnotations'), dict(type='PackSegInputs'), ], type='CityscapesDataset'), dict( data_prefix=dict( img_path='images/10k/val', seg_map_path='labels/sem_seg/masks/val'), data_root='/dataset2/gjw/DG_SEG/bdd100k/', img_suffix='.jpg', pipeline=[ dict(type='LoadImageFromFile'), dict(keep_ratio=True, scale=( 1280, 720, ), type='Resize'), dict(type='LoadAnnotations'), dict(type='PackSegInputs'), ], seg_map_suffix='.png', type='CityscapesDataset'), dict( data_prefix=dict( img_path='half/val_img', seg_map_path='half/val_label'), data_root='/dataset2/gjw/DG_SEG/mapillary/', img_suffix='.jpg', pipeline=[ dict(type='LoadImageFromFile'), dict(keep_ratio=True, scale=( 1024, 512, ), type='Resize'), dict(type='LoadAnnotations'), dict(type='PackSegInputs'), ], seg_map_suffix='.png', type='CityscapesDataset'), ], type='ConcatDataset'), num_workers=4, persistent_workers=True, sampler=dict(shuffle=False, type='DefaultSampler')) test_evaluator = dict( dataset_keys=[ 'citys', 'map', 'bdd', ], iou_metrics=[ 'mIoU', ], type='DGIoUMetric') train_bdd = dict( data_prefix=dict( img_path='images/10k/train', seg_map_path='labels/sem_seg/masks/train'), data_root='/dataset2/gjw/DG_SEG/bdd100k/', img_suffix='.jpg', pipeline=[ dict(type='LoadImageFromFile'), dict(type='LoadAnnotations'), dict(scale=( 1280, 720, ), type='Resize'), dict(cat_max_ratio=0.75, crop_size=( 512, 512, ), type='RandomCrop'), dict(prob=0.5, type='RandomFlip'), dict(type='PhotoMetricDistortion'), dict(type='PackSegInputs'), ], seg_map_suffix='.png', type='CityscapesDataset') train_cfg = dict( max_iters=40000, type='IterBasedTrainLoop', val_interval=10000) train_cityscapes = dict( data_prefix=dict( img_path='leftImg8bit/train', seg_map_path='gtFine/train'), data_root='/dataset2/gjw/DG_SEG/cityscapes/', pipeline=[ dict(type='LoadImageFromFile'), dict(type='LoadAnnotations'), dict(scale=( 1024, 512, ), type='Resize'), dict(cat_max_ratio=0.75, crop_size=( 512, 512, ), type='RandomCrop'), dict(prob=0.5, type='RandomFlip'), dict(type='PhotoMetricDistortion'), dict(type='PackSegInputs'), ], type='CityscapesDataset') train_dataloader = dict( batch_size=6, dataset=dict( data_prefix=dict(img_path='images', seg_map_path='labels'), data_root='/dataset2/gjw/DG_SEG/gta/', img_suffix='.png', pipeline=[ dict(type='LoadImageFromFile'), dict(type='LoadAnnotations'), dict( max_size=2048, resize_type='ResizeShortestEdge', scales=[ 256, 307, 358, 409, 460, 512, 563, 614, 665, 716, 768, 819, 870, 921, 972, 1024, ], type='RandomChoiceResize'), dict( cat_max_ratio=0.75, crop_size=( 512, 512, ), type='RandomCrop'), dict(prob=0.5, type='RandomFlip'), dict(type='PhotoMetricDistortion'), dict(type='PackSegInputs'), ], seg_map_suffix='_labelTrainIds.png', type='CityscapesDataset'), num_workers=2, persistent_workers=True, pin_memory=True, sampler=dict(shuffle=True, type='InfiniteSampler')) train_gta = dict( data_prefix=dict(img_path='images', seg_map_path='labels'), data_root='/dataset2/gjw/DG_SEG/gta/', img_suffix='.png', pipeline=[ dict(type='LoadImageFromFile'), dict(type='LoadAnnotations'), dict(scale=( 1280, 720, ), type='Resize'), dict(cat_max_ratio=0.75, crop_size=( 512, 512, ), type='RandomCrop'), dict(prob=0.5, type='RandomFlip'), dict(type='PhotoMetricDistortion'), dict(type='PackSegInputs'), ], seg_map_suffix='_labelTrainIds.png', type='CityscapesDataset') train_mapillary = dict( data_prefix=dict( img_path='training/images', seg_map_path='cityscapes_trainIdLabel/train/label'), data_root='/dataset2/gjw/DG_SEG/mapillary/', img_suffix='.jpg', pipeline=[ dict(type='LoadImageFromFile'), dict(type='LoadAnnotations'), dict(scale=( 1024, 512, ), type='Resize'), dict(cat_max_ratio=0.75, crop_size=( 512, 512, ), type='RandomCrop'), dict(prob=0.5, type='RandomFlip'), dict(type='PhotoMetricDistortion'), dict(type='PackSegInputs'), ], seg_map_suffix='.png', type='CityscapesDataset') train_pipeline = [ dict(type='LoadImageFromFile'), dict(type='LoadAnnotations'), dict( max_size=2048, resize_type='ResizeShortestEdge', scales=[ 256, 307, 358, 409, 460, 512, 563, 614, 665, 716, 768, 819, 870, 921, 972, 1024, ], type='RandomChoiceResize'), dict(cat_max_ratio=0.75, crop_size=( 512, 512, ), type='RandomCrop'), dict(prob=0.5, type='RandomFlip'), dict(type='PhotoMetricDistortion'), dict(type='PackSegInputs'), ] tta_model = dict(type='SegTTAModel') val_bdd = dict( data_prefix=dict( img_path='images/10k/val', seg_map_path='labels/sem_seg/masks/val'), data_root='/dataset2/gjw/DG_SEG/bdd100k/', img_suffix='.jpg', pipeline=[ dict(type='LoadImageFromFile'), dict(keep_ratio=True, scale=( 1280, 720, ), type='Resize'), dict(type='LoadAnnotations'), dict(type='PackSegInputs'), ], seg_map_suffix='.png', type='CityscapesDataset') val_cfg = dict(type='ValLoop') val_cityscapes = dict( data_prefix=dict(img_path='leftImg8bit/val', seg_map_path='gtFine/val'), data_root='/dataset2/gjw/DG_SEG/cityscapes/', pipeline=[ dict(type='LoadImageFromFile'), dict(keep_ratio=True, scale=( 1024, 512, ), type='Resize'), dict(type='LoadAnnotations'), dict(type='PackSegInputs'), ], type='CityscapesDataset') val_dataloader = dict( batch_size=1, dataset=dict( datasets=[ dict( data_prefix=dict( img_path='leftImg8bit/val', seg_map_path='gtFine/val'), data_root='/dataset2/gjw/DG_SEG/cityscapes/', pipeline=[ dict(type='LoadImageFromFile'), dict(keep_ratio=True, scale=( 1024, 512, ), type='Resize'), dict(type='LoadAnnotations'), dict(type='PackSegInputs'), ], type='CityscapesDataset'), dict( data_prefix=dict( img_path='images/10k/val', seg_map_path='labels/sem_seg/masks/val'), data_root='/dataset2/gjw/DG_SEG/bdd100k/', img_suffix='.jpg', pipeline=[ dict(type='LoadImageFromFile'), dict(keep_ratio=True, scale=( 1280, 720, ), type='Resize'), dict(type='LoadAnnotations'), dict(type='PackSegInputs'), ], seg_map_suffix='.png', type='CityscapesDataset'), dict( data_prefix=dict( img_path='half/val_img', seg_map_path='half/val_label'), data_root='/dataset2/gjw/DG_SEG/mapillary/', img_suffix='.jpg', pipeline=[ dict(type='LoadImageFromFile'), dict(keep_ratio=True, scale=( 1024, 512, ), type='Resize'), dict(type='LoadAnnotations'), dict(type='PackSegInputs'), ], seg_map_suffix='.png', type='CityscapesDataset'), ], type='ConcatDataset'), num_workers=4, persistent_workers=True, sampler=dict(shuffle=False, type='DefaultSampler')) val_evaluator = dict( dataset_keys=[ 'citys', 'map', 'bdd', ], iou_metrics=[ 'mIoU', ], type='DGIoUMetric') val_gta = dict( data_prefix=dict(img_path='images', seg_map_path='labels'), data_root='/dataset2/gjw/DG_SEG/gta/', img_suffix='.png', pipeline=[ dict(type='LoadImageFromFile'), dict(keep_ratio=True, scale=( 1280, 720, ), type='Resize'), dict(type='LoadAnnotations'), dict(type='PackSegInputs'), ], seg_map_suffix='_labelTrainIds.png', type='CityscapesDataset') val_mapillary = dict( data_prefix=dict(img_path='half/val_img', seg_map_path='half/val_label'), data_root='/dataset2/gjw/DG_SEG/mapillary/', img_suffix='.jpg', pipeline=[ dict(type='LoadImageFromFile'), dict(keep_ratio=True, scale=( 1024, 512, ), type='Resize'), dict(type='LoadAnnotations'), dict(type='PackSegInputs'), ], seg_map_suffix='.png', type='CityscapesDataset') vis_backends = [ dict(type='LocalVisBackend'), dict(type='TensorboardVisBackend'), ] visualizer = dict( name='visualizer', type='SegLocalVisualizer', vis_backends=[ dict(type='LocalVisBackend'), dict(type='TensorboardVisBackend'), ]) work_dir = './test_rein/rein_dinov2_mask2former_512x512_bs1x4_test'

2024/07/30 15:19:13 - mmengine - INFO - Distributed training is not used, all SyncBatchNorm (SyncBN) layers in the model will be automatically reverted to BatchNormXd layers if they are used. 2024/07/30 15:19:13 - mmengine - INFO - Hooks will be executed in the following order: before_run: (VERY_HIGH ) RuntimeInfoHook
(BELOW_NORMAL) LoggerHook

after_load_checkpoint: (NORMAL ) LoadBackboneHook

before_train: (VERY_HIGH ) RuntimeInfoHook
(NORMAL ) IterTimerHook
(VERY_LOW ) CheckpointHook

before_train_epoch: (VERY_HIGH ) RuntimeInfoHook
(NORMAL ) IterTimerHook
(NORMAL ) DistSamplerSeedHook

before_train_iter: (VERY_HIGH ) RuntimeInfoHook
(NORMAL ) IterTimerHook

after_train_iter: (VERY_HIGH ) RuntimeInfoHook
(NORMAL ) IterTimerHook
(NORMAL ) SegVisualizationHook
(BELOW_NORMAL) LoggerHook
(LOW ) ParamSchedulerHook
(VERY_LOW ) CheckpointHook

after_train_epoch: (NORMAL ) IterTimerHook
(LOW ) ParamSchedulerHook
(VERY_LOW ) CheckpointHook

before_val: (VERY_HIGH ) RuntimeInfoHook

before_val_epoch: (NORMAL ) IterTimerHook

before_val_iter: (NORMAL ) IterTimerHook

after_val_iter: (NORMAL ) IterTimerHook
(NORMAL ) SegVisualizationHook
(BELOW_NORMAL) LoggerHook

after_val_epoch: (VERY_HIGH ) RuntimeInfoHook
(NORMAL ) IterTimerHook
(BELOW_NORMAL) LoggerHook
(LOW ) ParamSchedulerHook
(VERY_LOW ) CheckpointHook

after_val: (VERY_HIGH ) RuntimeInfoHook

after_train: (VERY_HIGH ) RuntimeInfoHook
(VERY_LOW ) CheckpointHook

before_test: (VERY_HIGH ) RuntimeInfoHook

before_test_epoch: (NORMAL ) IterTimerHook

before_test_iter: (NORMAL ) IterTimerHook

after_test_iter: (NORMAL ) IterTimerHook
(NORMAL ) SegVisualizationHook
(BELOW_NORMAL) LoggerHook

after_test_epoch: (VERY_HIGH ) RuntimeInfoHook
(NORMAL ) IterTimerHook
(BELOW_NORMAL) LoggerHook

after_test: (VERY_HIGH ) RuntimeInfoHook

after_run: (BELOW_NORMAL) LoggerHook

2024/07/30 15:19:14 - mmengine - WARNING - The prefix 2024/07/30 15:19:15 - mmengine - INFO - Load checkpoint 2024/07/30 15:19:44 - mmengine - INFO - Iter(test) [ 50/3500] 2024/07/30 15:20:10 - mmengine - INFO - Iter(test) [ 100/3500] 2024/07/30 15:20:36 - mmengine - INFO - Iter(test) [ 150/3500] 2024/07/30 15:21:01 - mmengine - INFO - Iter(test) [ 200/3500] 2024/07/30 15:21:28 - mmengine - INFO - Iter(test) [ 250/3500] 2024/07/30 15:21:54 - mmengine - INFO - Iter(test) [ 300/3500] 2024/07/30 15:22:20 - mmengine - INFO - Iter(test) [ 350/3500] 2024/07/30 15:22:46 - mmengine - INFO - Iter(test) [ 400/3500] 2024/07/30 15:23:10 - mmengine - INFO - Iter(test) [ 450/3500] 2024/07/30 15:23:34 - mmengine - INFO - Iter(test) [ 500/3500] 2024/07/30 15:24:38 - mmengine - INFO - Iter(test) [ 550/3500] 2024/07/30 15:25:41 - mmengine - INFO - Iter(test) [ 600/3500] 2024/07/30 15:26:45 - mmengine - INFO - Iter(test) [ 650/3500] 2024/07/30 15:27:48 - mmengine - INFO - Iter(test) [ 700/3500] 2024/07/30 15:28:52 - mmengine - INFO - Iter(test) [ 750/3500] 2024/07/30 15:29:54 - mmengine - INFO - Iter(test) [ 800/3500] 2024/07/30 15:30:57 - mmengine - INFO - Iter(test) [ 850/3500] 2024/07/30 15:32:01 - mmengine - INFO - Iter(test) [ 900/3500] 2024/07/30 15:33:05 - mmengine - INFO - Iter(test) [ 950/3500] 2024/07/30 15:34:09 - mmengine - INFO - Iter(test) [1000/3500] 2024/07/30 15:35:13 - mmengine - INFO - Iter(test) [1050/3500] 2024/07/30 15:36:17 - mmengine - INFO - Iter(test) [1100/3500] 2024/07/30 15:37:20 - mmengine - INFO - Iter(test) [1150/3500] 2024/07/30 15:38:26 - mmengine - INFO - Iter(test) [1200/3500] 2024/07/30 15:39:36 - mmengine - INFO - Iter(test) [1250/3500] 2024/07/30 15:40:45 - mmengine - INFO - Iter(test) [1300/3500] 2024/07/30 15:41:49 - mmengine - INFO - Iter(test) [1350/3500] 2024/07/30 15:42:59 - mmengine - INFO - Iter(test) [1400/3500] 2024/07/30 15:44:08 - mmengine - INFO - Iter(test) [1450/3500] 2024/07/30 15:45:18 - mmengine - INFO - Iter(test) [1500/3500] 2024/07/30 15:45:36 - mmengine - INFO - Iter(test) [1550/3500] 2024/07/30 15:45:55 - mmengine - INFO - Iter(test) [1600/3500] 2024/07/30 15:46:14 - mmengine - INFO - Iter(test) [1650/3500] 2024/07/30 15:46:33 - mmengine - INFO - Iter(test) [1700/3500] 2024/07/30 15:46:50 - mmengine - INFO - Iter(test) [1750/3500] 2024/07/30 15:47:07 - mmengine - INFO - Iter(test) [1800/3500] 2024/07/30 15:47:23 - mmengine - INFO - Iter(test) [1850/3500] 2024/07/30 15:47:40 - mmengine - INFO - Iter(test) [1900/3500] 2024/07/30 15:47:57 - mmengine - INFO - Iter(test) [1950/3500] 2024/07/30 15:48:13 - mmengine - INFO - Iter(test) [2000/3500] 2024/07/30 15:48:30 - mmengine - INFO - Iter(test) [2050/3500] 2024/07/30 15:48:47 - mmengine - INFO - Iter(test) [2100/3500] 2024/07/30 15:49:04 - mmengine - INFO - Iter(test) [2150/3500] 2024/07/30 15:49:21 - mmengine - INFO - Iter(test) [2200/3500] 2024/07/30 15:49:37 - mmengine - INFO - Iter(test) [2250/3500] 2024/07/30 15:49:54 - mmengine - INFO - Iter(test) [2300/3500] 2024/07/30 15:50:11 - mmengine - INFO - Iter(test) [2350/3500] 2024/07/30 15:50:27 - mmengine - INFO - Iter(test) [2400/3500] 2024/07/30 15:50:44 - mmengine - INFO - Iter(test) [2450/3500] 2024/07/30 15:51:01 - mmengine - INFO - Iter(test) [2500/3500] 2024/07/30 15:51:18 - mmengine - INFO - Iter(test) [2550/3500] 2024/07/30 15:51:35 - mmengine - INFO - Iter(test) [2600/3500] 2024/07/30 15:51:52 - mmengine - INFO - Iter(test) [2650/3500] 2024/07/30 15:52:09 - mmengine - INFO - Iter(test) [2700/3500] 2024/07/30 15:52:26 - mmengine - INFO - Iter(test) [2750/3500] 2024/07/30 15:52:43 - mmengine - INFO - Iter(test) [2800/3500] 2024/07/30 15:52:59 - mmengine - INFO - Iter(test) [2850/3500] 2024/07/30 15:53:16 - mmengine - INFO - Iter(test) [2900/3500] 2024/07/30 15:53:33 - mmengine - INFO - Iter(test) [2950/3500] 2024/07/30 15:53:50 - mmengine - INFO - Iter(test) [3000/3500] 2024/07/30 15:54:06 - mmengine - INFO - Iter(test) [3050/3500] 2024/07/30 15:54:22 - mmengine - INFO - Iter(test) [3100/3500] 2024/07/30 15:54:38 - mmengine - INFO - Iter(test) [3150/3500] 2024/07/30 15:54:55 - mmengine - INFO - Iter(test) [3200/3500] 2024/07/30 15:55:11 - mmengine - INFO - Iter(test) [3250/3500] 2024/07/30 15:55:28 - mmengine - INFO - Iter(test) [3300/3500] 2024/07/30 15:55:44 - mmengine - INFO - Iter(test) [3350/3500] 2024/07/30 15:56:01 - mmengine - INFO - Iter(test) [3400/3500] 2024/07/30 15:56:17 - mmengine - INFO - Iter(test) [3450/3500] 2024/07/30 15:56:35 - mmengine - INFO - Iter(test) [3500/3500] 2024/07/30 15:56:35 - mmengine - INFO - ----------metrics 2024/07/30 15:56:35 - mmengine - INFO - per class results: 2024/07/30 15:56:35 - mmengine - INFO - +---------------+-------+-------+ | Class | IoU | Acc | +---------------+-------+-------+ | road | 93.03 | 96.91 | | sidewalk | 64.14 | 83.85 | | building | 84.66 | 93.8 | | wall | 47.13 | 79.35 | | fence | 38.94 | 68.35 | | pole | 43.94 | 56.14 | | traffic light | 51.36 | 73.13 | | traffic sign | 50.95 | 57.55 | | vegetation | 67.93 | 71.51 | | terrain | 33.8 | 69.9 | | sky | 76.77 | 94.14 | | person | 64.94 | 89.57 | | rider | 18.46 | 20.79 | | car | 89.48 | 93.74 | | truck | 39.34 | 90.08 | | bus | 70.27 | 96.14 | | train | 33.2 | 44.15 | | motorcycle | 28.04 | 79.13 | | bicycle | 34.3 | 36.89 | +---------------+-------+-------+ 2024/07/30 15:56:35 - mmengine - INFO - ----------metrics 2024/07/30 15:56:35 - mmengine - INFO - per class results: 2024/07/30 15:56:35 - mmengine - INFO - +---------------+-------+-------+ | Class | IoU | Acc | +---------------+-------+-------+ | road | 91.9 | 97.36 | | sidewalk | 60.26 | 67.74 | | building | 77.84 | 92.43 | | wall | 20.11 | 46.24 | | fence | 36.89 | 61.46 | | pole | 46.07 | 59.86 | | traffic light | 48.24 | 73.68 | | traffic sign | 47.46 | 59.85 | | vegetation | 55.25 | 57.92 | | terrain | 28.52 | 83.9 | | sky | 72.84 | 87.62 | | person | 68.58 | 81.83 | | rider | 14.01 | 19.32 | | car | 88.34 | 93.3 | | truck | 41.26 | 82.58 | | bus | 32.81 | 34.05 | | train | 4.63 | 6.41 | | motorcycle | 59.04 | 80.19 | | bicycle | 31.41 | 36.52 | +---------------+-------+-------+ 2024/07/30 15:56:35 - mmengine - INFO - ----------metrics 2024/07/30 15:56:35 - mmengine - INFO - per class results: 2024/07/30 15:56:35 - mmengine - INFO - +---------------+-------+-------+ | Class | IoU | Acc | +---------------+-------+-------+ | road | 84.63 | 95.98 | | sidewalk | 53.32 | 61.34 | | building | 74.76 | 90.29 | | wall | 41.04 | 67.49 | | fence | 39.3 | 68.32 | | pole | 39.75 | 56.79 | | traffic light | 42.54 | 66.08 | | traffic sign | 49.21 | 56.25 | | vegetation | 40.77 | 43.45 | | terrain | 27.98 | 93.44 | | sky | 75.17 | 86.89 | | person | 61.14 | 87.97 | | rider | 26.45 | 44.31 | | car | 82.2 | 88.96 | | truck | 29.25 | 94.95 | | bus | 50.59 | 66.81 | | train | 24.16 | 61.17 | | motorcycle | 54.52 | 76.79 | | bicycle | 40.56 | 44.38 | +---------------+-------+-------+ 2024/07/30 15:56:35 - mmengine - INFO - Iter(test) [3500/3500] is not set in metric class DGIoUMetric. from checkpoints/dinov2_rein_and_head.pth eta: 0:32:37 time: 0.5268 data_time: 0.0057 memory: 9894
eta: 0:30:58 time: 0.5249 data_time: 0.0063 memory: 1706
eta: 0:30:08 time: 0.5263 data_time: 0.0059 memory: 1706
eta: 0:29:10 time: 0.5135 data_time: 0.0061 memory: 1706
eta: 0:28:42 time: 0.5274 data_time: 0.0059 memory: 1706
eta: 0:28:11 time: 0.5205 data_time: 0.0060 memory: 1706
eta: 0:27:37 time: 0.5252 data_time: 0.0059 memory: 1706
eta: 0:27:09 time: 0.5076 data_time: 0.0060 memory: 1706
eta: 0:26:28 time: 0.4953 data_time: 0.0062 memory: 1706
eta: 0:25:54 time: 0.4889 data_time: 0.0060 memory: 1706
eta: 0:28:48 time: 1.2728 data_time: 0.0039 memory: 1731
eta: 0:31:04 time: 1.2885 data_time: 0.0039 memory: 1731
eta: 0:32:50 time: 1.2731 data_time: 0.0039 memory: 1731
eta: 0:34:12 time: 1.2744 data_time: 0.0039 memory: 1731
eta: 0:35:13 time: 1.2768 data_time: 0.0039 memory: 1731
eta: 0:35:56 time: 1.2533 data_time: 0.0038 memory: 1731
eta: 0:36:28 time: 1.2700 data_time: 0.0039 memory: 1731
eta: 0:36:53 time: 1.2915 data_time: 0.0038 memory: 1731
eta: 0:37:08 time: 1.2787 data_time: 0.0040 memory: 1731
eta: 0:37:14 time: 1.2749 data_time: 0.0040 memory: 1731
eta: 0:37:14 time: 1.2793 data_time: 0.0038 memory: 1731
eta: 0:37:08 time: 1.2761 data_time: 0.0039 memory: 1731
eta: 0:36:55 time: 1.2215 data_time: 0.0035 memory: 1731
eta: 0:36:45 time: 1.3884 data_time: 0.0040 memory: 1731
eta: 0:36:36 time: 1.3882 data_time: 0.0038 memory: 1731
eta: 0:36:22 time: 1.3875 data_time: 0.0038 memory: 1731
eta: 0:35:56 time: 1.3898 data_time: 0.0040 memory: 1731
eta: 0:35:34 time: 1.3966 data_time: 0.0041 memory: 1731
eta: 0:35:10 time: 1.3894 data_time: 0.0038 memory: 1731
eta: 0:34:42 time: 1.3907 data_time: 0.0040 memory: 1731
eta: 0:33:08 time: 0.3577 data_time: 0.0058 memory: 1793
eta: 0:31:39 time: 0.3610 data_time: 0.0061 memory: 1853
eta: 0:30:14 time: 0.3650 data_time: 0.0072 memory: 1731
eta: 0:28:53 time: 0.3572 data_time: 0.0057 memory: 1866
eta: 0:27:34 time: 0.3432 data_time: 0.0048 memory: 1871
eta: 0:26:18 time: 0.3119 data_time: 0.0042 memory: 1731
eta: 0:25:05 time: 0.3141 data_time: 0.0047 memory: 1726
eta: 0:23:54 time: 0.3266 data_time: 0.0042 memory: 1851
eta: 0:22:48 time: 0.3294 data_time: 0.0050 memory: 1730
eta: 0:21:43 time: 0.3303 data_time: 0.0053 memory: 1861
eta: 0:20:41 time: 0.3600 data_time: 0.0041 memory: 1843
eta: 0:19:41 time: 0.3446 data_time: 0.0042 memory: 1848
eta: 0:18:43 time: 0.3153 data_time: 0.0050 memory: 1864
eta: 0:17:46 time: 0.3308 data_time: 0.0045 memory: 1707
eta: 0:16:52 time: 0.3261 data_time: 0.0044 memory: 1846
eta: 0:15:59 time: 0.3263 data_time: 0.0040 memory: 1730
eta: 0:15:08 time: 0.3276 data_time: 0.0050 memory: 1858
eta: 0:14:17 time: 0.3130 data_time: 0.0048 memory: 1730
eta: 0:13:29 time: 0.3562 data_time: 0.0042 memory: 1858
eta: 0:12:42 time: 0.3129 data_time: 0.0045 memory: 1833
eta: 0:11:56 time: 0.3288 data_time: 0.0054 memory: 1858
eta: 0:11:11 time: 0.3410 data_time: 0.0039 memory: 1853
eta: 0:10:27 time: 0.3278 data_time: 0.0045 memory: 1731
eta: 0:09:44 time: 0.3572 data_time: 0.0047 memory: 1731
eta: 0:09:02 time: 0.3416 data_time: 0.0044 memory: 1731
eta: 0:08:21 time: 0.3273 data_time: 0.0047 memory: 1731
eta: 0:07:41 time: 0.3303 data_time: 0.0052 memory: 1866
eta: 0:07:02 time: 0.3280 data_time: 0.0045 memory: 1836
eta: 0:06:23 time: 0.3284 data_time: 0.0043 memory: 1872
eta: 0:05:45 time: 0.3404 data_time: 0.0039 memory: 1870
eta: 0:05:08 time: 0.3097 data_time: 0.0045 memory: 1868
eta: 0:04:31 time: 0.3401 data_time: 0.0048 memory: 1865
eta: 0:03:55 time: 0.3398 data_time: 0.0048 memory: 1873
eta: 0:03:20 time: 0.3395 data_time: 0.0048 memory: 1855
eta: 0:02:45 time: 0.3524 data_time: 0.0040 memory: 1804
eta: 0:02:11 time: 0.3560 data_time: 0.0043 memory: 1730
eta: 0:01:37 time: 0.3264 data_time: 0.0042 memory: 1707
eta: 0:01:04 time: 0.3263 data_time: 0.0043 memory: 1734
eta: 0:00:32 time: 0.3120 data_time: 0.0046 memory: 1723
eta: 0:00:00 time: 0.3112 data_time: 0.0045 memory: 1727
for citys------------ for bdd------------ for map------------ citys_aAcc: 88.5400 citys_mIoU: 54.2500 citys_mAcc: 73.4300 bdd_aAcc: 84.3200 bdd_mIoU: 48.7100 bdd_mAcc: 64.3300 map_aAcc: 80.2400 map_mIoU: 49.3300 map_mAcc: 71.1400 mean_aAcc: 84.3667 mean_mIoU: 50.7633 mean_mAcc: 69.6333 data_time: 0.0049 time: 0.6397

w1oves / Rein

The test performance is different from the paper #53

2024/07/30 15:19:08 - mmengine - INFO -

Runtime environment: cudnn_benchmark: True mp_cfg: {'mp_start_method': 'fork', 'opencv_num_threads': 0} dist_cfg: {'backend': 'nccl'} seed: 42 Distributed launcher: none Distributed training: False GPU number: 1