search

webmachinelearning / webnn-samples

🧠✨ Web Neural Network API samples
https://webmachinelearning.github.io/webnn-samples/
Apache License 2.0
109 stars 28 forks source link

Add SSD model into object detection sample #52

Closed huningxin closed 3 years ago

huningxin commented 3 years ago

Both YOLO and SSD models are referred in WebNN object detection use case. Now the object detection sample supports TinyYOLOV2. It would be good to support SSD as well.

Honry commented 3 years ago

@huningxin, which SSD models should we support?

Honry commented 3 years ago

Just synced with @huningxin offline, we planned to implement the SSD-MobileNetV1, as both ONNX and TFLite model zoo provide this model.

ONNX: https://github.com/onnx/models/tree/master/vision/object_detection_segmentation/ssd-mobilenetv1 TFLite: https://tfhub.dev/tensorflow/tfjs-model/ssd_mobilenet_v1/1/default/1

Honry commented 3 years ago

@huningxin, after some initial investigation, I found the model from ONNX is quite complex (see below screenshot) and without optimization, moreover, some ops e.g. tile, loop are missing in current spec. Here comes two proposals:

  1. Use onnxruntime-web to load and inference this mode.
  2. Convert and optimize the model from TFLite to ONNX, which is simple and optimized. But there's a customized op: "TFLite_Detection_PostProcess" in the tflite model and we have to converter which for removing the post process graph at first.

Besides above problems, we need to figure out what is done in the post process op. (The TFLite model is different from the one used in https://github.com/intel/webml-polyfill/tree/master/examples/object_detection/model).

image