Different eval metrics for model with/without bn

[alasin]

Hi,

When I evaluate on cityscapes using train_30k and train_30k_bnnomerge models, I am getting different mIOU of 65.6% and 59.3% respectively. As per my understanding, they should ideally give the same results. Am I missing something?

Thanks

[hellochick]

Hey, this is also my question, and I still have no idea about why it happened. Maybe there is a bug in my bn_model, but I can't find it. Another possibility is that the author did not upload the final version of bnnomerge model. If you find any thing helpful to this issue, please tell me, thanks.

[alasin]

The weights provided by them are fine as I can actually convert the train_30k_bnnomerge weights to train_30k weights by merging the batchnorm layer weights with conv layer weights. I also tried to compare the activations after every bn layer(for bn model) and conv layer(for no bn model) and noticed that the error between them keeps on increasing as we go deeper. My guess is that bn weights are somehow not being loaded properly or something else that is buggy with the bn model.

[hellochick]

Hey, can you tell me how you convert weights by merging the batchnorm layer weights with conv weights? Did you do this using .npy or .caffemodel ? I think maybe the problem is that bn weights not converted properly, I'll check again.

[alasin]

I used the npy weights. Here's the script I wrote:

https://github.com/alasin/python-scripts/blob/master/convert_bn.py

[hellochick]

Thank you for the script. I'll check the method I load bn weight, also the model.

[hellochick]

Hey @alasin, issue #41 have solved this problem.