Reproduce Figure 13

[jinglin80]

Thank you for the great work!

May you share the code for reproducing the figure 13 (Visualization of feature correspondence) of the paper? We have a bit difficult with it.

Thanks, Jing

[pUmpKin-Co]

Hi~Sorry for my late reply. I have updated the implementation here. This implementation is highly borrowed from iBOT repo. Hope this will help you.

[jinglin80]

Thank you for updating! May you also share the tsne_visualization.py that was used to produce Figure 11?

[pUmpKin-Co]

Hi~Have updated. This implementation is borrowed from MMPretrain.

[jinglin80]

Thank you! However, when trying to replicate figure 11(e), I got the following error. May you help? python main_tsne.py --checkpoint CMID_ResNet50_bk_200ep --data-root /datasets/UCMerced_LandUse/ Traceback (most recent call last): File "official-CMID/Pretrain/main_tsne.py", line 8, in from models import build_model File "official-CMID/Pretrain/models/init.py", line 1, in from .build import build_model File "official-CMID/Pretrain/models/build.py", line 4, in from .clas_model import ClassifyModel, SwinClassifyModel File "official-CMID/Pretrain/models/clas_model.py", line 6, in from .swin_utils import resize_pos_embed File "official-CMID/Pretrain/models/swin_utils.py", line 11, in from mmcv.cnn.utils.weight_init import truncnormal ModuleNotFoundError: No module named 'mmcv.cnn.utils.weight_init'

[pUmpKin-Co]

Hi~It's may be related to version of MMCV. Please make sure the version in your environment is same as the project environment file.

[jinglin80]

Yes, it looks like an MMCV issue, as shown in the following warning.

import mmcv /opt/conda/lib/python3.7/site-packages/mmcv/init.py:21: UserWarning: On January 1, 2023, MMCV will release v2.0.0, in which it will remove components related to the training process and add a data transformation module. In addition, it will rename the package names mmcv to mmcv-lite and mmcv-full to mmcv. See https://github.com/open-mmlab/mmcv/blob/master/docs/en/compatibility.md for more details. 'On January 1, 2023, MMCV will release v2.0.0, in which it will remove '

[jinglin80]

Could you share the checkpoint file used to generate Figure 11?

[pUmpKin-Co]

plz refer to the CMID models pretrained on MillionAID in project main page.

[jinglin80]

I used the full UCM dataset and CMID_ResNet50_200ep.pth from the main page. However, the feat1.png I got does not resemble Figure 11(e). python main_tsne.py --checkpoint CMID_ResNet50_200ep.pth --data-root UCMerced_LandUse/Images

[jinglin80]

We also tried to reproduce Figure 11 (f) but got the following error message:

Traceback (most recent call last): File "mmrotate/official-CMID/Pretrain/main_tsne.py", line 145, in main() File "mmrotate/official-CMID/Pretrain/main_tsne.py", line 95, in main model = build_model(config, is_pretrain=False) File "/mmrotate/official-CMID/Pretrain/models/build.py", line 28, in build_model model = SwinClassifyModel(config) File "/mmrotate/official-CMID/Pretrain/models/clas_model.py", line 89, in init super(SwinClassifyModel, self).init(config) File "/mmrotate/official-CMID/Pretrain/models/clas_model.py", line 16, in init self.embed_dim = config.hidden_dim File "/opt/conda/lib/python3.7/site-packages/ml_collections/config_dict/config_dict.py", line 829, in getattr raise AttributeError(e) AttributeError: "'hidden_dim'"

To check the content of config, we add a print statement on line 15 of clas_model.py print(config) Output: config checkpoint: /mmrotate/models/backbones/pretrain/CMID_Swin-B_200ep.pth config: null data_root: UCMerced_LandUse/Images deterministic: false early_exaggeration: 12.0 init: random is_distribute: false layer_ind: 0,1,2,3,4 learning_rate: 200.0 local_rank: 0 max_num_class: 20 n_components: 2 n_iter: 1000 n_iter_without_progress: 300 output: /mnt/in-house/python/Q1/model/full2 perplexity: 30.0 pool_type: specified rank: 0 seed: 0 world_size: 1

What did we miss? Furthermore, we made the following changes:

official-CMID/Pretrain/models/swin_transformer.py

from mmcv.cnn.utils.weight_init import constant_init, trunc_normal_init, truncnormal

from mmengine.model.weight_init import constant_init, truncnormal, trunc_normal_init

from mmcv.runner.base_module import BaseModule, ModuleList

from mmengine.model import BaseModule, ModuleList

from mmcv.utils.parrots_wrapper import _BatchNorm

from mmengine.utils.dl_utils.parrots_wrapper import _BatchNorm

official-CMID/Pretrain/models/backbone_wrapper.py

from mmcv.cnn.utils.weight_init import constant_init, trunc_normal_init, truncnormal

from mmengine.model.weight_init import constant_init, truncnormal, trunc_normal_init

official-CMID/Pretrain/utils/tsne_utils.py, line 5

from mmcv.runner import BaseModule

from mmengine.model import BaseModule

official-CMID/Pretrain/models/mim_utils.py, line 12

from mmcv.utils import TORCH_VERSION, digit_version

from mmengine.utils import digit_version from mmengine.utils.dl_utils import TORCH_VERSION