pre-training weight and 'auxiliary loss operations'

[hydxqing]

Recently,I'm learning about semantic segmentation, and I'm glad to learn your Repositories: pspnet-tf-fiction on github.This is a particularly good reproduction. And I have a few questions for you:

1. How do you import the pre-training weight of resnet_v1_101 into the model? I think the code is initialized with 'he'.
2. What is the role of 'auxiliary loss operations' in the model? When I use resnet_v1_50.ckpt, it will report an error. When I screen it out, the output loss will be larger or smaller. I have not learned enough about this part, and there are many things to learn from you. Your help is very important to me, and I am looking forward to your reply.

[holyseven]

Here is the code that imports the pre-trained weights. The 'he' initialization is a default intializer but will be overwritten by that code.

The code in this repository is only for resnet_v1_101 at this moment. I'm not sure what error you met. So if you would like to change to resnet_v1_50, the code you should modify is here, especially the function inference.

[hydxqing]

Thank you for your reply. Renet_v1_101 can't run because of the restriction of the graphics card. I can only use resnet_v1_50 at present. The network can output loss. And the value of "loss, precision, wd" after training 9000 times is "0.025793, 0.100266, 0.069967". Is this a bad result?

[holyseven]

Yes. Not sure what you did, loss is small but the precision is low.

An example log file I had:

step, loss, precision, wd
{'batch_size': 2, 'blur': 1, 'bn_frozen': 0, 'color_switch': 0, 'data_type': 32, 'database': 'CityScapes', 'ema_decay': 0.9, 'epsilon': 1e-05, 'eval_only': 0, 'fine_tune_filename': '../z_pretrained_weights/resnet_v1_101.ckpt', 'fisher_epsilon': 0, 'fisher_filename': './fisher_exp.npy', 'fix_blocks': 0, 'gpu_num': 4, 'has_aux_loss': 1, 'initializer': 'he', 'log_dir': 'pspmg-only-resnet', 'loss_type': 'normal', 'lr_step': None, 'lrn_rate': 0.01, 'mirror': 1, 'momentum': 0.9, 'network': 'pspnet', 'new_layer_names': None, 'norm_only': 0, 'optimizer': 'mom', 'poly_lr': 1, 'random_rotate': 0, 'random_scale': 1, 'resize_images_method': 'bilinear', 'resume_step': None, 'save_first_iteration': 0, 'scale_max': 1.75, 'scale_min': 0.5, 'server': 2, 'snapshot': 25000, 'step_size': 0.1, 'structure_in_paper': 0, 'subsets_for_training': 'train', 'test_batch_size': 5, 'test_image_size': 864, 'test_max_iter': None, 'train_image_size': 864, 'train_like_in_paper': 0, 'train_max_iter': 50000, 'weight_decay_mode': 1, 'weight_decay_rate': 1e-05, 'weight_decay_rate2': 1e-05}
20,1.828431,0.196262,0.233563
40,0.992438,0.246641,0.233615
60,0.778456,0.270597,0.233575
80,0.718698,0.293666,0.233507
100,0.664058,0.313219,0.233439

[hydxqing]

Oh,Thanks.I trained 30,000 times with resnet_v1_50, and the final accuracy is 62.2%. What is the best result of your training? You add the auxiliary loss to the final loss in the program directly .Do we not assign weights to auxiliary loss?

[holyseven]

62.2 on Cityscapes? I only tested resnet_v1_101 and had 78.2 without multi-scale test.

For auxiliary loss, its weight is 0.4.

[hydxqing]

I trained on my own dataset with multi-scale test. But when I increase the number of training, the accuracy has not changed. And do you have any good suggestions for optimization?

[holyseven]

It is a difficult question. Try Adam? But probably it won't change a lot. What is the mIoU score on the training set? If it's high, it is not a problem of optimization. Try different learning rates.

[hydxqing]

Thanks.But is mIOU not the accuracy of output?

[holyseven]

It is.

[hydxqing]

But I seem to have problems in training. 1.In the training process, as the number of iterations increased, the loss almost remained unchanged, but the accuracy increased slowly. 2.And the accuracy of the test set is always higher than that of the training set. Do you know why？

[holyseven]

It is a good sign. The mIoU is computed by accumulating the confusion matrix. For training, the confusion matrix will rarely be reinitialized, so the mIoU is affected by the beginning examples but will increase slowly even when the training has already converged.

the accuracy of the test set is always higher than that of the training set.

this is because there is no overfitting. The mIoU on the test set is almost equal to the "true" mIoU on the training set.