search

hollance / neural-engine

Everything we actually know about the Apple Neural Engine (ANE)
MIT License
1.97k stars 72 forks source link

[Question] Have you tried A17 Pro Neural Engine? #27

Closed y-ich closed 9 months ago

y-ich commented 9 months ago

Hi, Hollance san.

Have you tried A17 Pro Neural Engine?

Apple claim that it has 35TOPS, which is a big leap performance boost. But Geekbench ML shows that it is a just little. iPhone16,1(iPhone 15 Pro) = 3402 iPhone15,2(iPhone 14 Pro) = 3349 https://browser.geekbench.com/ml/v0/inference

I wonder that Apple may implement 4bit INT OPs for internal use, and legacy performance is almost same, just clock up. What do you think about it?

hollance commented 9 months ago

I have not tried it but it wouldn't surprise me if the performance we can get as third-party developers isn't the maximum, since we're limited to using the Neural Engine through Core ML.

(Also, those Geekbench scores may not actually use the Neural Engine but only the GPU.)

y-ich commented 9 months ago

Thank you for your comments! I hope that Apple will make all performance of neural engine open some day.

About Geekbench ML, they test CPU, GPU, and Core ML resources separately using TensorFlow Lite. Comparing those score, I think that Geekbench ML Core ML seems use Neural Engine normally though I am not sure whether it shows the best performance.

hollance commented 9 months ago

Does the TensorFlow Lite test use the CoreMLDelegate, though? (I don't know, never looked at it.) But even then there's no guarantee the model actually runs on the Neural Engine.

y-ich commented 9 months ago

Yes, they test the performances of CPU, GPU, and Core ML separately. The scores show Core ML uses Neural Engine.