About the results on VOC2012 with 92 examples

[GritCS]

Thank you for your expressive and remarkable work and delicate code!! But when I reproducting the results on VOC2012 with 92 examples,I only get 59 with examples,while the reported result is 67.98.Can you offer some suggest? Thank you sincerely!

This is my log file! seg_20220414_150853.txt

This is my tb about contrastive_loss and unsupervised loss! they're kind of weired and on an upward trend.

[GritCS]

This is my config file. And I run my code in 4 gpu(4*V100),batch_size = 4

dataset: # Required. type: pascal_semi train: data_root: ../../../../../data/VOC2012 data_list: ../../../../../data/splits/pascal/92/labeled.txt flip: True GaussianBlur: False rand_resize: [0.5, 2.0]

rand_rotation: [-10.0, 10.0]

crop:
  type: rand
  size: [513, 513] # crop image with HxW size

val: data_root: ../../../../../data/VOC2012 data_list: ../../../../../data/splits/pascal/val.txt crop: type: center size: [513, 513] # crop image with HxW size batch_size: 4 n_sup: 92 noise_std: 0.1 workers: 2 mean: [123.675, 116.28, 103.53] std: [58.395, 57.12, 57.375] ignore_label: 255

trainer: # Required. epochs: 80 eval_on: True optimizer: type: SGD kwargs: lr: 0.001 # 4GPUs momentum: 0.9 weight_decay: 0.0001 lr_scheduler: mode: poly kwargs: power: 0.9 unsupervised: TTA: False drop_percent: 80 apply_aug: cutmix contrastive: loss_weight: 0.1 negative_high_entropy: True low_rank: 3 high_rank: 20 current_class_threshold: 0.3 current_class_negative_threshold: 1 unsupervised_entropy_ignore: 80 low_entropy_threshold: 20 num_negatives: 50 num_queries: 256 temperature: 0.5

saver: snapshot_dir: checkpoints/base_16 pretrain: ''

criterion: type: CELoss kwargs: use_weight: False

net: # Required. num_classes: 21 sync_bn: True ema_decay: 0.99 encoder: type: u2pl.models.resnet.resnet101 kwargs: multi_grid: True zero_init_residual: True fpn: True replace_stride_with_dilation: [False, True, True] #layer0...1 is fixed, layer2...4 decoder: type: u2pl.models.decoder.dec_deeplabv3_plus kwargs: inner_planes: 256 dilations: [12, 24, 36]

[Haochen-Wang409]

Hi, thanks for your approval!

From the loss curve, unsup_loss seems to be a little bit wired. The contra_loss can be regarded as a regularizer, which is expected to be around 0.2 during the whole training process. The provided training log seems to be incomplete since it only contains 27 epochs? Could you provide the whole log?

By the way, with only 92 labeled samples, the training procedure is a little bit noisy. Maybe you could try applying aux_loss, which could be found in: https://github.com/Haochen-Wang409/U2PL/blob/main/experiments/cityscapes/744/suponly/config.yaml#L57-L59

[Haochen-Wang409]

I guess it might beause unsup_loss is around 0.3, which is quite large than the sup_loss in your log file. Therefore, aux_loss or ohem might help.

[GritCS]

Yeap,there're only 27 epochs.Due to the poor performance,I interrupted the training. Thanks for your helpful suggestion!! I'll retraining the model with only 92 labeled samples with the config.yaml you provide. Btw, there's no need to change the value of relevant parameter about aux_loss (aux_plane: 1024 and loss_weight: 0.4 )?

Thanks sincerely again!

[Haochen-Wang409]

Yes, just set aux_plane: 1024 and loss_weight: 0.4.

Note that we did not adopt aux_loss or ohem in experiments on VOC.

[GritCS]

Hello! I have some question about the split of voc dataset .

When I train the model with classic voc 2012. Should I change 10582 to 1464 ( line 109 in the pascal_voc.py file. )?

What about you when you trains model on classic voc,Thanks!!

[Haochen-Wang409]

You do not need to change 10582. In fact, n_sup in here actually means number of unlabeled images, since it is used to build dataloader here.

For semi-supervised settings, usually an epoch is defined as the model have been trained on all unlabeled images.

[GritCS]

But I understand that the classic VOC 2012 dataset is divided into training,validationand test sets including 1,464, 1,449 and 1,456 images, respectively. So the number of unlabeled images should be 1464 - num_labeled. So this's 1464 not 10582. Meybe I mistake the composition of classic VOC 2012 dataset ? Thanks!

[Haochen-Wang409]

Augmented VOC contains 10582 images in total. For classical VOC, labeled images are selected from 1464 fine annotated samples and the remaining 9118 images are regarded as unlabeled images. Therefore, number of total images are 10582 rather than 1464.

[GritCS]

Augmented VOC contains 10582 images in total. For classical VOC, labeled images are selected from 1464 fine annotated samples and the remaining 9118 images are regarded as unlabeled images. Therefore, number of total images are 10582 rather than 1464.

Augmented VOC contains 10582 images in total. For classical VOC, labeled images are selected from 1464 fine annotated samples and the remaining 9118 images are regarded as unlabeled images. Therefore, number of total images are 10582 rather than 1464.

oo！！I see!! Thank you very much!!

[Haochen-Wang409]

Never mind~!