Open PapaMadeleine2022 opened 5 years ago
Hey, this is the MXNet Label Bot. Thank you for submitting the issue! I will try and suggest some labels so that the appropriate MXNet community members can help resolve it. Here are my recommended labels: Performance
Could you try the CPU inference which is significantly improved since 0.7?
https://mxnet.incubator.apache.org/versions/master/tutorials/mkldnn/MKLDNN_README.html
The issue may be similar to https://github.com/apache/incubator-mxnet/issues/13928
However, I do not have any machine with NVIDIA GPU to test it : (
@wkcn I dont know which op in my model is slower in newer mxnet. no dilated convolutional layers in my model which is a ocr recognition model with simple cnn and rnn. Can give some detailed test advises?
@IvyGongoogle Could you please use the profiler to measure the execution time? http://mxnet.incubator.apache.org/versions/master/tutorials/python/profiler.html?highlight=profiler
How much time in your model among different MXNet? Since MXNet supports large tensors now, it may drop a little performance.
@wkcn sorry, I can not find some C++ codes about how to use profiler.
@IvyGongoogle Could you please provide the forward time between different versions of MXNet? Is it obvious?
@wkcn thanks for your reply,
mxnet v0.8:
initialization time(that is `MXPredCreat` function time): 69906ms(the first run)/1416ms/1419ms/1471ms, my model inference time:
218ms/209.3ms/207.6ms/207.4ms
mxnet v1.0.0:
initialization time(that is `MXPredCreat` function time): 170572ms(the first run)/2779ms/2794ms/2756ms, my model inference time:
218.5ms/218.9ms/215.5ms/224.8ms
as we can see that the initialization time of higher mxnet is nearly double in contrast to mxnet v0.8, and my model inference time is also slower.
@leleamol - Can you please take a look at this issue and suggest why it may be high initialization time with latest MXNet?
@IvyGongoogle - Is it possible to share a minimum reproducible example?
Also, it may be related to this issue - https://github.com/apache/incubator-mxnet/issues/14569 where we have changed few things, like moving to Int64, to support large tensors.
Hello, I have a problem: I use
libmxnet.so
compiled with mxnetv0.8 comparison to mxnetv1.0(or v1.3 and v1.4) to run my code to infer a batch images using C++ API, I find the inference speed with mxnet of higher version is slower than mxnetv0.8. What causes this problem? How to fix it? anyone can give some advises?envs: cuda8/cudnn5.1.10/nvidia-driver384.81