Problems in training Camvid dataset

[Mayy1994]

Thanks for sharing so wonderful video semantic segmentation work.

I did the experiments on Cityscapes dataset, and I can reproduce the data in your paper. However, I have encountered some problems when training the Camvid dataset. (The pretrained models are downloaded from the links you provided, and the requirements are satisfied.)

The PDA curves on testing are very weird, but the cfnet loss and the semantic loss are decreasing. Attached are the IoU accuracy across different testing epochs when the propagation distance is 1, 5, and 9, respectively. The last two images are the training losses.

I am confused about the above performances. The model seems over-fitting. Have you seen this kind of problem in your training? Do you have any idea how to fix this training problem? In the "train.sh" file, both experiments adopt the same training epochs, learning rates, adjustment schedule of learning rates. Is there anything wrong with this setting?

Besides, I can obtain the exact same segmentation performances on Camvid just like the images in the repository folder "result --> spatial_correction_camvid" when I loaded the trained "spatial_correction_camvid" model provided in this repository. But unfortunately, I cannot achieve the same PDA in your paper. I can't figure it out.

      Fig. 12 in your paper

      The PDA curve using the trained "spatial_correction_camvid" model provided in this repository.

Looking forward to your response. Many thanks.

[jfzhuang]

Many thanks for your interest in our work. I have to apologize that we have uploaded a previous version of codes and trained model weights for the camvid experiment by mistake. This version of codes lack of common data augmentation operations (such as random rotation and cropping), which are essential to alleviate the problem of overfitting. We will update the codes and corresponding model weights in the next few days. After that, You can conduct the camvid experiment again.

[jfzhuang]

We have updated the codes and trained model weight. Welcome to conduct the camvid experiment again.

[Mayy1994]

Thank you very much for your kind and quick response. I will conduct the experiment again.

[Mayy1994]

Hi, the updated "spatial_correction_camvid" model is the same as the previous one. I have printed the parameters of this model, it is the same as the previous model. The PDA curve in the testing dataset also remains unchanged. Could you please check whether you have uploaded the correct one?

Besides, I conducted the experiment on Camvid dataset (with the latest repository) again. I noticed that you added data augmentation of random cropping and random flipping, but the training becomes very unstable. The IoU accuracy across different testing epochs when the propagation distance is 1, 5, and 9 are shown in the images below.

I saved the best.pth and loaded it for testing. The PDA curve in the testing dataset using my trained model is shown in the following image. Still, I can't obtain the performance as your paper. It is similar to the previous performance.

[jfzhuang]

1. I'm so sorry to only update the model weight on the BaiduYun and forget about the Dropbox. Now, you can check the new model on the Dropbox.
2. Your training curve seems weird and you can check the curve in my camvid experiment in the following. Please check if you have updated the corresponding codes (e.g., dataset/camvid.py and python/train.py)

[Mayy1994]

Thank you so much! I have reproduced the experimental results on Camvid dataset. Bravo!