search

dotnet / machinelearning

ML.NET is an open source and cross-platform machine learning framework for .NET.
https://dot.net/ml
MIT License
9.02k stars 1.88k forks source link

`ImageClassification` prediction appears slow #5179

Closed leblancdavid closed 4 years ago

leblancdavid commented 4 years ago

I don't have much experience with DNN (more with traditional machine learning algorithms) so I went through the automated visual inspection tutorial. The training of the 400 subset images is relatively quick. However, the prediction time is incredibly slow. Running the prediction through 400 images took about 90sec using ResnetV2101 and 10sec using MobilenetV2. For my application, I would need it to be about at least 5x to 10x faster (hopefully close to real-time).

My question is, am I doing something wrong or is there something I can do to improve prediction speed?

Thank you!

michaelgsharp commented 4 years ago

Would you be able to share your code so we can see what you are doing?

leblancdavid commented 4 years ago

Right now I'm just running the tutorial code (https://docs.microsoft.com/en-us/dotnet/machine-learning/tutorials/image-classification-api-transfer-learning). When I run Evaluate I notice it takes a long time. How long should it take?

wangyems commented 4 years ago

Would you also share your cpu/gpu configuration? It might be normal to me with that inference speed using ResNet or MobileNet on my laptop.

leblancdavid commented 4 years ago

I'm just running on a regular desktop PC. I just wanted to make sure that I had it implemented correctly. If you are getting similar times than that makes sense. Is there an option or parameter selection that improves inference speed?

michaelgsharp commented 4 years ago

What versions of the Nugets are you using? Specifically ML.Net?

leblancdavid commented 4 years ago

1.5.0 ML.Net, 1.15.1 TensorFlow

michaelgsharp commented 4 years ago

I am not positive whether it should be faster or not, but a couple of thoughts:

1 - The first pass of the image recognition is slower then the subsequent passes. This is due to the tensorflow graph warming up with the first run. Test and see what happens when you run it a second time in the same pipeline.

2 - I have not been able to find any issues causing slowdowns in our code, but I will be finishing up a comparison running the model directly with tensorflow's python api. I'll let you know the results. If our code is slower, then we may have an issue we need to look into. If not, then thats probably the way it is, at least for the models chosen.

leblancdavid commented 4 years ago

I'm guessing it's just the nature of the architecture then. I may just use decision trees or some of the other algorithms to improve the prediction speed. Thank you for your help!