How to generate auto-labeled images by myself?

[AndyChang666]

Hi, Thank you for sharing your codes! After I read the paper and code carefully, I still don't understand how to generate the auto-labeled images. In section 4 of the paper, it says "we select the top class prediction of the teacher network. We threshold the label based on teacher network output probability". However, I am confused about what a teacher network is. Moreover, I am curious that do you generate auto-labeled images during training or after training? And how to generate the auto-labeled images by myself? Thank you.

[karansapra]

Hi Andy

Sorry about the confusion.

The teacher network is trained with train + coarse + val. I can give you a training recipe if you want that. We train this model till completion and then label the coarse images after the training. Basically a summary of the recipe is a pre-trained mapillary model, cv3, train for 75 epochs, with weigh decay on.

The ability to dump auto labelling images is part of the code.

https://github.com/NVIDIA/semantic-segmentation/blob/055ee5f56e2fd735a7b9b8d1a2345f9c7df03880/train.py#L226

[AndyChang666]

So you label the coarse images after the training and then use this auto-labeled dataset to train again to get better results? And if I train with different networks will also impact the auto-labeled image result, right? Sure! That would be great if you could give me a training recipe. Thank you

[karansapra]

Yes to your first question, if your secondary network produces subpar output then it will hurt, it produces good output, then it wouldn't impact your IOU.

Yes, i ll get you that recipe this weekend.

[AndyChang666]

Got it. Thank you so much! Looking forward to your recipe.

[karansapra]

    "dataset": "cityscapes",
    "coarse_boost_classes": "3,4,12,14,15,16,17",
    "cv": 3,
    "syncbn": true,
    "apex": true,
    "fp16": true,
    "weight_decay": "1e-4",
    "arch": "ocrnet.HRNet_Mscale",
    "snapshot": "industrious-chicken",
    "crop_size": "1024,2048",
    "bs_trn": 1,
    "lr_schedule": "poly",
    "poly_exp": 1.5,
    "max_epoch": 75,
    "max_cu_epoch": 35,
    "optimizer": "radam",
    "lr": "5e-5",

[AndyChang666]

Thank you!

[WeiChihChern]

@karansapra Thank you for the auto-labeling explanation. I have another question is that, you trained a teacher network with with train + coarse + val, but wouldn't the coarse labels hinder teacher training? Because the coarse labels contain quite a lot of false information isn't it? Although according to the paper, it seems to be working pretty well. Would you please explain why would it still work? Thank you.

[hamzagorgulu]

In my understanding, the teacher network is nothing but the model itself. So they separate the coarse labels in the beginning and train the model with the fine labels. In the end, the auto-labeling is done by the trained model to get the annotations.