Segmentation Models Inference speed Improvement

[djaym7]

All the semantic models in the model zoo contain aux output which is only necessary during training and is just extra overhead during prediction.

The documentation and guide for "Test with {segmantation} model" should be updated to use aux less inference.

Solution : "add ignore_extra=True while loading parameters as shown below. Without it, when "aux" =False, the model doesnot have symbol for aux paramters and it throws error.

[bryanyzhu]

Yes, we are aware of this problem. This requires code refactor and re-save all the models without aux weights.

Setting ignore_extra is not a good practice unless you know what you are doing. Otherwise, you may build a wrong model or load wrong weights without knowing it.

We provide an alternative for segmentation inference, a function called predict. There is no aux computation inside, so the speed is fine. You can just use it as

pred_segmask = net.predict(x)

[djaym7]

Yes i am aware of predict method but it's only good if the model is rebuilt through gluon and can't if the model is loaded through mx.gluon.nn.SymbolBlock.imports and so added this.

[bryanyzhu]

@zhanghang1989 Do you think we should set ignore_extra =True or we refactor the code to save the model aux-lessly?

[bryanyzhu]

@djaym7 I think for now, we will stick to what we have, because

1. Setting ignore_extra to True is dangerous, which may not be user expected.
2. Refactor the code, re-train and re-save all the models take significant effort.

If you use symbol to load the model, you can manually set ignore_extra=True in your use case. But thank you very much for raising this issue, we will keep it in mind.