search

hellochick / ICNet-tensorflow

TensorFlow-based implementation of "ICNet for Real-Time Semantic Segmentation on High-Resolution Images".
405 stars 153 forks source link

Training on cityscapes dataset #51

Open junweiy opened 6 years ago

junweiy commented 6 years ago

Hi,

I tried to run train.py to train the model from stretch on cityscapes dataset, and only commented the line that restores from pre-trained model and modified the path of dataset. My program managed to reach loss of 1 in the first 100 steps, however the loss continued to fluctuate around 1 until 5000 steps. I am wondering if there's anything that I should further modify, any ideas?

Thanks!

jinkos commented 6 years ago

I am also struggling to train from scratch on cityscapes

I am using the model with filter_scale=2 and I am using a flat learning rate of 0.01 using MomentumOptimizer. I am using proper data and have done plenty of testing to make sure that my evaluation is A OK.

I am wondering of the L2 loss is too heavily weighted. I know the paper mentions weight decay of 0.0001. After 10 hours of training, my L2 has come down from 1.0 to 0.5 and is still falling.

But has anyone actually successfully trained the cityscapes dataset from scratch using filter_scale=2? I would like to know how they have done it.

harora commented 5 years ago

Hi

Did you get success in training on cityscapes dataset? I'm not able to reproduce the result. The loss goes to around 0.5 but predictions are coming wrong(2-3% mIoU)

tzhong518 commented 4 years ago

I'm having the same problem... Loss went down but the mIoU is only 2%. Have you succeeded in training from scratch? Thanks!

Hi

Did you get success in training on cityscapes dataset? I'm not able to reproduce the result. The loss goes to around 0.5 but predictions are coming wrong(2-3% mIoU)