Configs for AdvSeg and MinEnt

[Renp1ngs]

Dear Lukas: Thank you for your wonderful work and excellent code. Can you provide a configuration file that can be run directly using minent.py and advseg.py？ Thank you very much.

[lhoyer]

Thank you very much for your interest in our work. You can generate the config files yourself by running experiment 41 as explained in https://github.com/lhoyer/HRDA#training.

[Renp1ngs]

Yes, but I'm not quite sure how your interface is called. So do you have a configuration file that can be run directly and uses UDA type='minent' or type='advseg' can you give me a reference? thank you very much. Also about here https://github.com/lhoyer/HRDA/blob/master/mmseg/models/utils/dacs_transforms.py#L91

classes = torch.unique(labels)

Should it be so?

classes = torch.unique(label)

Thank you for your reply. Good luck.

[lhoyer]

Here are the automatically generated configs for reference:

{
    "_base_": [
        "../../_base_/default_runtime.py",
        "../../_base_/models/deeplabv2red_r50-d8.py",
        "../../_base_/datasets/uda_gtaCAugHR_to_cityscapesHR_1024x1024.py",
        "../../_base_/uda/advseg.py",
        "../../_base_/schedules/sgd.py",
        "../../_base_/schedules/poly10warm.py"
    ],
    "gpu_model": "NVIDIATITANRTX",
    "n_gpus": 1,
    "seed": 0,
    "model": {
        "pretrained": "open-mmlab://resnet101_v1c",
        "backbone": {
            "depth": 101
        },
        "decode_head": {
            "single_scale_head": "DLV2Head",
            "type": "HRDAHead",
            "attention_classwise": true,
            "hr_loss_weight": 0.1
        },
        "type": "HRDAEncoderDecoder",
        "scales": [
            1,
            0.5
        ],
        "hr_crop_size": [
            512,
            512
        ],
        "feature_scale": 0.5,
        "crop_coord_divisible": 8,
        "hr_slide_inference": true
    },
    "data": {
        "samples_per_gpu": 2,
        "workers_per_gpu": 1,
        "train": {}
    },
    "optimizer_config": null,
    "optimizer": {
        "lr": 0.0025,
        "paramwise_cfg": {
            "custom_keys": {}
        }
    },
    "runner": {
        "type": "IterBasedRunner",
        "max_iters": 40000
    },
    "checkpoint_config": {
        "by_epoch": false,
        "interval": 40000,
        "max_keep_ckpts": 1
    },
    "evaluation": {
        "interval": 4000,
        "metric": "mIoU"
    },
    "name": "220914_0758_gtaCAugHR2csHR_1024x1024_advseg_hrda1-512-01_dlv2red_r101v1c_poly10warm_sgd_00025_pmF_s0_2ce69",
    "exp": 41,
    "name_dataset": "gtaCAugHR2cityscapesHR_1024x1024",
    "name_architecture": "hrda1-512-0.1_dlv2red_r101v1c",
    "name_encoder": "r101v1c",
    "name_decoder": "hrda1-512-0.1_dlv2red",
    "name_uda": "advseg",
    "name_opt": "sgd_0.0025_pmFalse_poly10warm_1x2_40k",
    "work_dir": "work_dirs/local-exp41/220914_0758_gtaCAugHR2csHR_1024x1024_advseg_hrda1-512-01_dlv2red_r101v1c_poly10warm_sgd_00025_pmF_s0_2ce69",
    "git_rev": "301478d5bcaff72eae55c9b686197024bf181dd3"
}

and

{
    "_base_": [
        "../../_base_/default_runtime.py",
        "../../_base_/models/deeplabv2red_r50-d8.py",
        "../../_base_/datasets/uda_gtaCAugHR_to_cityscapesHR_1024x1024.py",
        "../../_base_/uda/minent.py",
        "../../_base_/schedules/sgd.py",
        "../../_base_/schedules/poly10warm.py"
    ],
    "gpu_model": "NVIDIATITANRTX",
    "n_gpus": 1,
    "seed": 0,
    "model": {
        "pretrained": "open-mmlab://resnet101_v1c",
        "backbone": {
            "depth": 101
        },
        "decode_head": {
            "single_scale_head": "DLV2Head",
            "type": "HRDAHead",
            "attention_classwise": true,
            "hr_loss_weight": 0.1
        },
        "type": "HRDAEncoderDecoder",
        "scales": [
            1,
            0.5
        ],
        "hr_crop_size": [
            512,
            512
        ],
        "feature_scale": 0.5,
        "crop_coord_divisible": 8,
        "hr_slide_inference": true
    },
    "data": {
        "samples_per_gpu": 2,
        "workers_per_gpu": 1,
        "train": {}
    },
    "optimizer_config": null,
    "optimizer": {
        "lr": 0.0025,
        "paramwise_cfg": {
            "custom_keys": {}
        }
    },
    "runner": {
        "type": "IterBasedRunner",
        "max_iters": 40000
    },
    "checkpoint_config": {
        "by_epoch": false,
        "interval": 40000,
        "max_keep_ckpts": 1
    },
    "evaluation": {
        "interval": 4000,
        "metric": "mIoU"
    },
    "name": "220914_0758_gtaCAugHR2csHR_1024x1024_minent_hrda1-512-01_dlv2red_r101v1c_poly10warm_sgd_00025_pmF_s0_2fd72",
    "exp": 41,
    "name_dataset": "gtaCAugHR2cityscapesHR_1024x1024",
    "name_architecture": "hrda1-512-0.1_dlv2red_r101v1c",
    "name_encoder": "r101v1c",
    "name_decoder": "hrda1-512-0.1_dlv2red",
    "name_uda": "minent",
    "name_opt": "sgd_0.0025_pmFalse_poly10warm_1x2_40k",
    "work_dir": "work_dirs/local-exp41/220914_0758_gtaCAugHR2csHR_1024x1024_minent_hrda1-512-01_dlv2red_r101v1c_poly10warm_sgd_00025_pmF_s0_2fd72",
    "git_rev": "301478d5bcaff72eae55c9b686197024bf181dd3"
}

[lhoyer]

For your second question, please have a look at https://github.com/lhoyer/DAFormer/issues/19.

[Renp1ngs]

Thank you very much. Wish you all the best.

[Renp1ngs]

I read through the code about auto-generating configuration files. This seems very stupid for my first question. I have to say that your code is really well written, and I admire it very much.

[lhoyer]

Thanks! I'm happy to hear that!

[Renp1ngs]

I'm very sorry to bother you again. When I checked the code of mmseg/uda/advseg.py, I found that model_D, which is FCDiscriminator, has two optimizers in your implementation. I tried the solution I gave, which is to modify the content in config. Is it correct?

optimizer = dict(
    lr=6e-05,
    paramwise_cfg=dict(
        custom_keys=dict(
            head=dict(lr_mult=10.0),
            pos_block=dict(decay_mult=0.0),
            norm=dict(decay_mult=0.0),
            model_D=dict(lr_mult=0))))

[lhoyer]

https://github.com/lhoyer/HRDA/blob/93af720322790357e6ea1b235468ab073ff6e439/mmseg/models/uda/advseg.py#L46

As you can see above the optimizer for model_D is hard coded. Therefore, you cannot change its behavior using the optimizer config. Some parameters (i.e. the learning rate) can be changed using the UDA config as shown below.

https://github.com/lhoyer/HRDA/blob/93af720322790357e6ea1b235468ab073ff6e439/configs/_base_/uda/advseg.py#L10

[Renp1ngs]

First of all have to thank you for your many replies to my question. But the problem I want to point out is that the discriminator parameter is registered by both optimizers, which I verified by stepping through. So I want to ask, the sgd optimizer set for the segmentation model will theoretically update the parameters of the discriminator, so is it necessary to control the lr_mult of optim_cfg equal to 0?

[lhoyer]

Thanks for the clarification. Now I see your point. Yes, the modified config should avoid an update of the discriminator.