search

hollance / Forge

A neural network toolkit for Metal
MIT License
1.27k stars 172 forks source link

[question] TensorFlow Lite #45

Closed y-ich closed 3 years ago

y-ich commented 3 years ago

Hi, @hollance san,

Have you ever evaluate TensorFlow Lite on iOS/iPadOS for GPU(I mean not for Neural Engine)?

As you had pointed out at some article, Core ML is slower than MPSCNN. So I expected Metal delegate of TensorFlow Lite and tried but am disappointed it’s performance. If you have some insights, could you tell me some?

Thanks.

hollance commented 3 years ago

I haven't really used TFLite very much, so I couldn't say. I also think Core ML is the go-to choice these days, I would only try MPSCNN if there was a really good reason for it.

y-ich commented 3 years ago

Thank you for your comment!

I know what you meant in some article, but also it is true even today. Core ML on GPU is twice or more slower than OpenCL implementation on macOS in another case I tried.

I want to get best performance on both ANE and GPU and expected it from TensorFlow Lite but…

Anyway thank you! Please take care of yourself in COVID-19 period.

hollance commented 3 years ago

Note that TF Lite's ANE support actually runs the model on Core ML. For Metal they use their own GPU kernels, plus maybe some MPSCNN (it's been a while since I checked).

y-ich commented 3 years ago

Sure, so I had expected GPU boost by TF Lite since Core ML on GPU is slow while it keeps speed in the case which ANE is available.

Thank you for your comment.