Semantic Flow for Fast and Accurate Scene Parsing

[lucasjinreal]

Semantic Flow for Fast and Accurate Scene Parsing

this model can be supported

[lucasjinreal]

It's the only realtime model achieves mIOU 80 on cityscapes.

[xvjiarui]

Hi @jinfagang Thanks for your suggestion. We will consider it in the next release.

[lxtGH]

@xvjiarui @jinfagang Hi thanks for your attention. I can work it for mmseg. I will make a pull request after i get the results. The official implementation is here: https://github.com/lxtGH/SFSegNets

[xvjiarui]

Hi @lxtGH How is it going?

[lxtGH]

@xvjiarui Hi ! I am training now. I will gvie the pull request if I get serveral results including Mapillary pretraining.

[Junjun2016]

@xvjiarui Hi ! I am training now. I will gvie the pull request if I get serveral results including Mapillary pretraining.

Looking forward to it. Could you please also have a pull request of DecoupleSegNet.

[xvjiarui]

Hi @lxtGH Thx! Looking forward to this PR.

[lxtGH]

I have some problems to re-produce the performance with mmseg. I try a lots of times using r18 backbone. but the results are inferior(73.x). It's very strange.

[lucasjinreal]

@lxtGH what's the should results of r18 and how many gap between mmseg and original one?

[lxtGH]

r18 is around 78.2 % in original repo. However, for mmseg, there are 5 % gap. I am very confused.

[baibaidj]

r18 is around 78.2 % in original repo. However, for mmseg, there are 5 % gap. I am very confused.

Hi. do you have any updates?

[zhangrui-wolf]

Under the default setting of mmseg, the highest accuracy of training is 75.05%. I didn't train the model after that.

The reasons for the huge differences in results may be: 1.The author uses some skills in the training. This can be found after reading the code again. 2.batch size is different. 3.The backbone network resnet18 used by the author is different from that in mmseg, stem_channel is 128, 4.The pre-training parameters are better.

Especially the last reason. Although I have not trained this model since then, I have trained other models in mmseg using the pre-training parameters published by the author. Compared with the results of the same model trained with the pre-training parameters of mmseg, it is found that the miou can be increased by about 1.5%.

[lucasjinreal]

@zhangrui-wolf I have tried original author's sf_segnets, his training result is not reproduceable. I highly doubt they correctly opensource their code or not. Even there exist some logic bug make some model can not be trained.

They claimed it's pytorch issue, but I trained the result is not mIOU less than 78 but totally divergency, not possible to reproduce with their training strategy and loss function.

[zhangrui-wolf]

@jinfagang Instead of using his source code for training directly, I copied his core code on mmseg. A member of the same group was trained using the source code. The result will not diverge, but the training process is unstable and the accuracy can not be achieved in the end. If divergent, the learning rate can be adjusted according to the batch size.

[lucasjinreal]

@zhangrui-wolf The first epoch loss become 1.0, then after 23 epoch loss become 6.7, and then stable at 6.5....

I don't know what's model can have such weired loss curve. At least even they converge, the mIOU can not be reproduced.

However, what surprise me is the checkpoint provided by author actually get a very nice mIOU on cityscapes eval dataset... And result seems very nice.

I am so confused.

[zhangrui-wolf]

@jinfagang Have you verified his pre-training model on the test set? Is it possible that he used some trick to make the model over-fit on the verification set?

[lucasjinreal]

@zhangrui-wolf No, I don't eval by my hand, but the result roughly looks better than my other SOTA models. I don't know if they using some tricks, but the author himself seems not even fammilar with his code (I asked him his code has a logic bug can not be train on some model such as dsn, he claimed it's my pytorch version problem but actually same error even I using his pytorch version).

That model is unable to reproduce original author's result. And I think original author doesn't want to fix it....

[xvjiarui]

@lxtGH Do you have any idea?

[lxtGH]

@xvjiarui Now, I got results about 77.3. However, it still 0.8 gap with original paper. I will use original backbone with stem channel 128.

[lucasjinreal]

@lxtGH Sounds like good. However original repo seems used very complicated augmentation and centriods clustering preprocess. These steps not sure can be migrated to mmseg or not.

[lxtGH]

Now, it tunes to 77.9. still 0.2-0.3 gap. I will try more settings.

[jyang68sh]

@lxtGH Hi I'm also getting 73.x result using resnet18. I am wondering what was the change you made to get a 4 percent improve? Thanks!