search

reneweb / react-native-tensorflow

A TensorFlow inference library for react native
Apache License 2.0
358 stars 59 forks source link

image input array #2

Closed mattryles closed 7 years ago

mattryles commented 7 years ago

Hi, Please go easy on me, i'm relatively new to tensorflow/ml I'm more of a web guy. I have a frozen model trained with the Tensorflow Object Detection API and hope to use react-native to deploy to android/ios.

When you say "Next you will need to feed an image in the form of a number array", what tools do you use in react-native to convert the image to a number array?

Thanks Matt

reneweb commented 7 years ago

Hey, This library is still very young and in it's current state it's more to play around / experiment with it, so you should be careful when using it for more sophisticated stuff. To answer your question you will need to load the image into react native - there is react-native-fs which allows to load it as base64. Next you will need to convert it from base64 into a number array (rgb most likely) that can be consumed by tensorflow, which will most likely be not that straight forward & then pass it to the feed function. Also note that this process is very inefficient as we need to pass the image between native & js side via base64 / number array. In the future I hope to make this process a lot more simple & efficient, but it is quite some work to achieve that. Also I should probably change the doc, as it doesn't require the data to be an image.

mattryles commented 7 years ago

Hi Rene, Thanks for the reply, that makes perfect sense, I was just confused because it looked like a simple 1d array of numbers, I'm used to the RGB arrays. I'll spend a bit of time prototyping at the weekend.

reneweb commented 7 years ago

No problem - have a look at this android example: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/examples/android/src/org/tensorflow/demo/TensorFlowImageClassifier.java. Especially at the recognizeImage function. It should give some guidance on how to use this library