Compiling the densenet model from onnx zoo is time consuming

[SharonYuIBM]

Hello, With the fix from commit: https://github.com/onnx/onnx-mlir/commit/4ffadba05a810729fb16982917c119117d5dd5e1

I'm now able to successfully compile the denesnet model from onnx zoo: https://github.com/onnx/models/blob/master/vision/classification/densenet-121/model/densenet-9.onnx

However, the compilation takes more than 15 minutes on my environment:

# yushuang @ yushuangs-MacBook-Pro in ~/Downloads [16:36:57] C:127
$ /Users/yushuang/Work/WMLz/SourceCode/onnx-mlir/build/bin/onnx-mlir --EmitLib ~/Downloads/dlc/zoo/densenet121/model.onnx
Shared library /Users/yushuang/Downloads/dlc/zoo/densenet121/model.so has been compiled.
(onnx)
# yushuang @ yushuangs-MacBook-Pro in ~/Downloads [16:54:48]

I wonder why it takes that long to process this model? Could you help review and see if this can be improved?

Thank you!

[tjingrant]

@doru1004 had some thoughts on this topic.

[doru1004]

Some of the overhead are due to replaceOp being called multiple times in a single matchAndRewrite pattern hence it happening sequentially. If we could enable more parallelism to safely happen at that level it would speed up the process.

[tungld]

My observation on ResNet50 was that if we turned off memory pool, compiling time was reduced from 4 minutes to 29 seconds. Not sure it is applicable with DenseNet or not.

[tjingrant]

If anyone is interested in trying turning off memory pooling to assess compilation speed without it, you can comment out passes starting from here: https://github.com/onnx/onnx-mlir/blob/9e2bc007816b6f504d999b41eff81321859a700d/src/MainUtils.cpp#L449

Though obviously, you don't get the memory saving benefits in this case.

[SharonYuIBM]

Thank you all for the review and suggestions.

I tried by turning off memory pooling which helps reduce the compilation time of the DenseNet model from 18 minutes to 1.5 minutes. I have some additional questions:

1. Does it mean memory pooling is a trade-off between compilation efficiency and memory saving?
2. The code comment seems to indicate we have the plan to make memory pooling optional? Is this in plan?

Thanks.

[tjingrant]

Does it mean memory pooling is a trade-off between compilation efficiency and memory saving?

No, it should not slow down compilation. This issue obviously needs to be fixed for memory pooling optimization to be practical.

The code comment seems to indicate we have the plan to make memory pooling optional? Is this in plan?

As the comment indicates, yes, it is the plan.