search

hollance / neural-engine

Everything we actually know about the Apple Neural Engine (ANE)
MIT License
1.97k stars 72 forks source link

Issues discovered while running MobileNet V3 #2

Closed teijeong closed 3 years ago

teijeong commented 4 years ago

Hi, I'm working on integrating ANE to TFLite. While testing for MobileNet V3, I discovered following messages from os_log output.

Debug com.apple.espresso espresso "Kernel validation warning PoolingLayerBuilder (AVERAGE)_41 (pool) @ 33: Unsupported: (dilated)kernel width = 28 > 13"

Debug com.apple.espresso espresso "Kernel validation warning MulOpBuilder_49 (elementwise) @ 41: elementwise with channel broadcast supported only with constant vector or transplant input"

So the average pooling has hidden constraint that restricts size of kernel up to 13, and elementwise multiplication does not support broadcasted multiplication of [CxHxW] and [Cx1x1] tensors. (don't know what "transplant input" mean.)

hollance commented 4 years ago

These are useful messages, I was not aware that you could get this kind of debugging output out of Core ML!

teijeong commented 4 years ago

I ran mobilenet V3 with tweaks, to avoid those constraints. I lowered a global average pooling to multiple poolings, and tried upsampling [Cx1x1] input to [CxHxW] before multiplication. The attempt failed, but found these errors:

Kernel validation warning PoolingLayerBuilder (AVERAGE)_43 (pool) @ 35: Unsupported: stride_x = 4 > 2
Kernel validation warning UpsampleLayerBuilder_70 (upsample) @ 59: only scaling factor of 2 is supported, getting (28, 28)

So it seems pooling supports stride up to 2, and scaling factor of 2 in ANE. For the latter claim, not sure it does mean only 2, or power of 2.

hollance commented 4 years ago

These are very useful. I wish I'd known about this os_log instrument sooner, haha.

It's plausible that the ANE only supports the most common configurations such as stride 1 and 2, but not more. But it's good to have confirmation of this from these log mesages!

hollance commented 3 years ago

I integrated this info into the text. Thanks again!