search

tensorflow / tfjs

A WebGL accelerated JavaScript library for training and deploying ML models.
https://js.tensorflow.org
Apache License 2.0
18.51k stars 1.93k forks source link

[benchmarks][wasm] Test correctness of MobileNetV3 is false #6867

Closed qjia7 closed 2 years ago

qjia7 commented 2 years ago

Steps to reproduce:

  1. Open https://tensorflow.github.io/tfjs/e2e/benchmarks/local-benchmark/index.html
  2. Choose MobileNetV3 and wasm backend, click Test correctness.

Expect Prediction matches CPU is true. Actually got false. OS: windows browser: 105.0.5195.127

mattsoulanille commented 2 years ago

I can reproduce this on Chrome 106.0.5249.91 linux.

mattsoulanille commented 2 years ago

This is due to XNNPack not supporting a kernel size of 1x1 in AvgPool (which is pretty reasonable since that turns it into the identity function, not counting padding and strides). I'm working on a fix, but I need to account for strides and padding.

Why does MobileNetV3 have an AvgPool node with a 1x1 kernel that behaves as the identity function? Maybe there's something wrong in our conversion? Specifically, the node I'm talking about is StatefulPartitionedCall/StatefulPartitionedCall/predict/MobilenetV3/Logits/AvgPool.

mattsoulanille commented 2 years ago

Fixed by #6969. Confirmed with the local benchmark tool.

google-ml-butler[bot] commented 2 years ago

Are you satisfied with the resolution of your issue? Yes No