On some Android devices, ssd model predict errors.

[yichuxue]

TensorFlow.js version: 1.2.6 tfjs-wechat plugin version: 0.0.5 WeChat version: 7.0.5 WeChat base API version: 2.7.7 WeChat IDE version: 1.02.1907242 Describe the problem or feature request

When I tested the SSD model, I found that some Android phones could not be predicted successfully. Here are some models and results I tested.

Iphone X(IOS 12.3) Result: Success Iphone 8P(IOS 12.3) Result: Success Iphone 7P(IOS 12.2) Result: Success Iphone 7P(IOS 12.3.1) Result: Success 华为荣耀 imagic(Android 9)：Success 小米 9(MIUI 10 && Android 9)：Result: Error 小米 k20(MIUI 10 && Android 9): Result: Error 锤子坚果 pro(Android 7.1.1): Result: Error vivi nex (Android ): Result: Error 小米 5X(Android 8.1): Result: Error

Error screenshot:

Code to reproduce the bug / link to feature request

[pyu10055]

@yichuxue which SSD model you are using? Are you using the coco-ssd model in tfjs-models repo?

[yichuxue]

@pyu10055 use coco-ssd. Using tensorflow object detection training, the configuration file is: ssd_mobilenet_v1_coco_2017_11_17.

[sanyuered]

https://github.com/tensorflow/tfjs-wechat/issues/24#issuecomment-520902070

I guess most of model prediction errors are caused by WeChat CameraContext.onCameraFrame API. Because the prediction result is based on the image size A(288x352) that is returned by onCameraFrame API, but the image size of camera component is B(375x375,200x200,etc.). The model prediction is correct for image size A, but is not correct for image size B.

https://github.com/tensorflow/tfjs-wechat/blob/master/demo/posenet/pages/index/index.wxml ` <camera device-position="back" flash="off" binderror="error" style="width: 288px; height: 352px;z-index: 5;" frame-size="small">

` The style size(width: 288px; height: 352px) of camera component is proper for iPhone WeChat, but is not proper for Android WeChat. Because the style size of camera component on Android WeChat should be 288x384 or another values.

[yichuxue]

@sanyuered #22 There may be such a reason as you suggest, but it has been ruled out. In the test, it was read directly, so the data was the same, but ios and android predicted different results.

[pyu10055]

@yichuxue I notice the issue is related the the data frame size, have you tested with an image on both iphone and android phones, do they have different results?

Ideally if wechat can provide a coordination mapping function, we should be able to correctly map the output to the camera canvas.

[pyu10055]

closing due to lack of activities.