how to detect in real time

shaqian / tflite-react-native

React Native library for TensorFlow Lite

https://www.npmjs.com/package/tflite-react-native

MIT License

289 stars 108 forks source link

how to detect in real time #3

Open Serenagirl opened 5 years ago

Serenagirl commented 5 years ago

this can detect images ,but if it can detect videos in real time?And can you give me some advice?

shaqian commented 5 years ago

A problem with react-native is streaming large data between JS and native module will lock up the bridge. That's why the react-native-camera module does not support streaming at the moment. You can refer to the discussion here: https://github.com/react-native-community/react-native-camera/issues/135

One workaround I can think of is to combine react-native-camera and this library in one native module and directly feed image from camera to tflite using native code.

Thanks, Qian

mdcroce commented 5 years ago

Check thist post: https://medium.com/@namar/high-performance-image-classification-with-react-native-336db0a96cd

ShaharyarMaroof commented 5 years ago

@mdcroce I used this library, but I am unable to get the image detection work with any other model other than the one provided in the article. I tried the starter PoseNet model provided on Tensorflow (link) and set the Output.json with the following array: ["nose", "leftEye", "rightEye", "leftEar", "rightEar", "leftShoulder", "rightShoulder", "leftElbow", "rightElbow", "leftWrist", "rightWrist", "leftHip", "rightHip", "leftKnee", "rightKnee", "leftAnkle", "rightAnkle"].

Any help in the matter will be appreciated. Thanks!