onReady is not called in TensorCamera

keisan1231 commented 3 years ago

I'm from this

Thaks a lot for your repository.

I was abled to find onReady is called only for the first time.

now onReady is not called in TensorCamera.

In your environment , onReady is called everytime???

<TensorCamera
          style={styles.camera}
          type={Camera.Constants.Type.back}
          zoom={0}
          cameraTextureHeight={textureDims.height}
          cameraTextureWidth={textureDims.width}
          resizeHeight={tensorDims.height}
          resizeWidth={tensorDims.width}
          resizeDepth={3}
          onReady={(imageAsTensors) => {
            console.log("ONREADY");
            handleCameraStream(imageAsTensors);
          }}
          autorender={true}
        />

danielwind commented 3 years ago

Hi @keisan1231 thanks for your feedback.

Please excuse me if I do not follow entirely your question. The onReady callback in the TensorCamera component is supposed to be called every time the next frame is processed. It will register automatically once the high order component is mounted. You can refer to the TFJS source code here: https://github.com/tensorflow/tfjs/blob/master/tfjs-react-native/src/camera/camera_stream.tsx#L290

In this tutorial's repository, this callback gets executed every time so we can get the tensors feedback in real time. I believe you are stating that on your side it is only executing only for the first time and never gets called again? If so, it could be the TF framework is not being initialized properly or perhaps the component is not mounting correctly. Would you please share your code here so I can take a look and depict what the issue may be? Thanks!

keisan1231 commented 3 years ago

I tried this by same as your code😰

keisan1231 commented 3 years ago

useEffect() was called, but onReady was not called

keisan1231 commented 3 years ago

this log

backend.js:32 Initialization of backend rn-webgl failed

keisan1231 commented 3 years ago

danielwind commented 3 years ago

Hi @keisan1231

Thanks for the information. I believe you are hitting a bug with Expo and Tensorflow version incompatibility. The issue was previously reported here: https://github.com/tensorflow/tfjs/issues/3360

Can you please confirm what versions of Expo and TensorFlow you are using? Even if you are pulling it from this repository, if you previously installed expo then it may be an older version. Also, it may be worth checking all the GL versions:

"dependencies": { "@tensorflow-models/coco-ssd": "^2.1.0", "@tensorflow-models/mobilenet": "^2.0.4", "@tensorflow/tfjs": "^2.0.1", "@tensorflow/tfjs-react-native": "^0.3.0", "expo": "~38.0.8", "expo-camera": "~8.3.1", "expo-gl": "~8.3.1", "expo-gl-cpp": "~8.3.1",

Thanks

keisan1231 commented 3 years ago

yes, I pulled it from this repository.

expo then it may be an older version

Does this mean expo-cli???

What is your version??

Mine is % expo --version 3.26.2

danielwind commented 3 years ago

Hi @keisan1231

Sorry, not the expo-cli, but the expo package (you can review it from the node_modules directory of your project)

https://www.npmjs.com/package/expo

Thanks

keisan1231 commented 3 years ago

I update expo but onReady was not called.

"expo": "^38.0.10", ("expo": "~38.0.8", also not successfully)

I don't understand that 'onReady' was called only for the first time😓

I redo pulling from this repo, but it doesn't work successfully

danielwind commented 3 years ago

Hi @keisan1231

Can you please confirm the versions of all the @tensorflow dependencies match? (e.g "@tensorflow/tfjs": "^2.0.1")?

Thanks

keisan1231 commented 3 years ago

@danielwind

yes, dependecies are same as this repo.

"dependencies": {
    "@react-native-community/async-storage": "^1.11.0",
    "@tensorflow-models/coco-ssd": "^2.1.0",
    "@tensorflow-models/mobilenet": "^2.0.4",
    "@tensorflow/tfjs": "^2.0.1",
    "@tensorflow/tfjs-react-native": "^0.3.0",
    "expo": "~38.0.8",
    "expo-barcode-scanner": "~8.2.1",
    "expo-camera": "~8.3.1",
    "expo-gl": "~8.3.1",
    "expo-gl-cpp": "~8.3.1",
    "expo-image-picker": "~8.3.0",
    "expo-media-library": "~8.2.1",
    "expo-permissions": "~9.0.1",
    "expo-status-bar": "^1.0.2",
    "react": "~16.11.0",
    "react-dom": "~16.11.0",
    "react-native": "https://github.com/expo/react-native/archive/sdk-38.0.2.tar.gz",
    "react-native-fs": "^2.16.6",
    "react-native-paper": "^4.0.1",
    "react-native-picker-select": "^7.0.0",
    "react-native-shapes": "^0.1.0",
    "react-native-svg": "12.1.0",
    "react-native-web": "~0.11.7"
  },

danielwind commented 3 years ago

@keisan1231

OK. I am sure you are having some version incompatibility around Expo or Tensorflow. Please check if your TF and expo versions defined in package.json match exactly the ones that are downloaded in the node_modules directory of your project. Since Tensorflow has a hard requirement on the Expo framework, versions need to match exactly. This is obviously very inconvenient, but until the tfjs react-native port is more mature, this is what we get. These versions might get updated automatically based on the expo version you have installed. Mine is:

$ expo --version 3.22.3

It seems your version is higher based on your previous replies. What you need to do is to check if the versions of TFJS and Expo (within the node_modules directory) matches the ones defined in package.json. If they do NOT match, then you need to hardcode the version (this means, remove the ^ or ~ in the package.json dependencies version so you don't get a range of possible incompatible versions!)

For instance, for checking the versions in mode_modules please refer to the following image:

Screen Shot 2020-09-11 at 5 10 33 AM

Thanks

keisan1231 commented 3 years ago

@danielwind

Thank you for advise. I tried 3 patterns but all pattern did not give me success😰

one

 "dependencies": {
    "@react-native-community/async-storage": "^1.11.0",
+    "@tensorflow-models/coco-ssd": "2.1.0",
-    "@tensorflow-models/coco-ssd": "^2.1.0",
+    "@tensorflow-models/mobilenet": "2.0.4",
-    "@tensorflow-models/mobilenet": "^2.0.4",
+    "@tensorflow/tfjs": "2.0.1",
-    "@tensorflow/tfjs": "^2.0.1",
+    "@tensorflow/tfjs-react-native": "0.3.0",
-    "@tensorflow/tfjs-react-native": "^0.3.0",
    "expo": "~38.0.8",
    "expo-barcode-scanner": "~8.2.1",
    "expo-camera": "~8.3.1",
    "expo-gl": "~8.3.1",
    "expo-gl-cpp": "~8.3.1",
    "expo-image-picker": "~8.3.0",
    "expo-media-library": "~8.2.1",
    "expo-permissions": "~9.0.1",
    "expo-status-bar": "^1.0.2",
    "react": "~16.11.0",
    "react-dom": "~16.11.0",
    "react-native": "https://github.com/expo/react-native/archive/sdk-38.0.2.tar.gz",
    "react-native-fs": "^2.16.6",
    "react-native-paper": "^4.0.1",
    "react-native-picker-select": "^7.0.0",
    "react-native-shapes": "^0.1.0",
    "react-native-svg": "12.1.0",
    "react-native-web": "~0.11.7"
  },

two

 "dependencies": {
    "@react-native-community/async-storage": "^1.11.0",
    "@tensorflow-models/coco-ssd": "^2.1.0",
    "@tensorflow-models/mobilenet": "^2.0.4",
    "@tensorflow/tfjs": "^2.0.1",
    "@tensorflow/tfjs-react-native": "^0.3.0",
+    "expo": "38.0.8",
-    "expo": "~38.0.8",
+    "expo-barcode-scanner": "8.2.1",
-    "expo-barcode-scanner": "~8.2.1",
+    "expo-camera": "8.3.1",
-    "expo-camera": "~8.3.1",
+    "expo-gl": "8.3.1",
-    "expo-gl": "~8.3.1",
+    "expo-gl-cpp": "8.3.1",
-    "expo-gl-cpp": "~8.3.1",
+    "expo-image-picker": "8.3.0",
-    "expo-image-picker": "~8.3.0",
+    "expo-media-library": "8.2.1",
-    "expo-media-library": "~8.2.1",
+    "expo-permissions": "9.0.1",
-    "expo-permissions": "~9.0.1",
+    "expo-status-bar": "1.0.2",
-    "expo-status-bar": "^1.0.2",
    "react": "~16.11.0",
    "react-dom": "~16.11.0",
    "react-native": "https://github.com/expo/react-native/archive/sdk-38.0.2.tar.gz",
    "react-native-fs": "^2.16.6",
    "react-native-paper": "^4.0.1",
    "react-native-picker-select": "^7.0.0",
    "react-native-shapes": "^0.1.0",
    "react-native-svg": "12.1.0",
    "react-native-web": "~0.11.7"
  },

three

 "dependencies": {
    "@react-native-community/async-storage": "^1.11.0",
+    "@tensorflow-models/coco-ssd": "2.1.0",
-    "@tensorflow-models/coco-ssd": "^2.1.0",
+    "@tensorflow-models/mobilenet": "2.0.4",
-    "@tensorflow-models/mobilenet": "^2.0.4",
+    "@tensorflow/tfjs": "2.0.1",
-    "@tensorflow/tfjs": "^2.0.1",
+    "@tensorflow/tfjs-react-native": "0.3.0",
-    "@tensorflow/tfjs-react-native": "^0.3.0",
+    "expo": "38.0.8",
-    "expo": "~38.0.8",
+    "expo-barcode-scanner": "8.2.1",
-    "expo-barcode-scanner": "~8.2.1",
+    "expo-camera": "8.3.1",
-    "expo-camera": "~8.3.1",
+    "expo-gl": "8.3.1",
-    "expo-gl": "~8.3.1",
+    "expo-gl-cpp": "8.3.1",
-    "expo-gl-cpp": "~8.3.1",
+    "expo-image-picker": "8.3.0",
-    "expo-image-picker": "~8.3.0",
+    "expo-media-library": "8.2.1",
-    "expo-media-library": "~8.2.1",
+    "expo-permissions": "9.0.1",
-    "expo-permissions": "~9.0.1",
+    "expo-status-bar": "1.0.2",
-    "expo-status-bar": "^1.0.2",
    "react": "~16.11.0",
    "react-dom": "~16.11.0",
    "react-native": "https://github.com/expo/react-native/archive/sdk-38.0.2.tar.gz",
    "react-native-fs": "^2.16.6",
    "react-native-paper": "^4.0.1",
    "react-native-picker-select": "^7.0.0",
    "react-native-shapes": "^0.1.0",
    "react-native-svg": "12.1.0",
    "react-native-web": "~0.11.7"
  },

danielwind commented 3 years ago

Hi @keisan1231

I am really sorry to hear that. Just to make sure, can you please stick with the pattern #3 and do:

Delete the "node_modules" directory from your project.
Run "npm install" on your project to re-download all dependencies.

Thanks

keisan1231 commented 3 years ago

@danielwind

In all pattern, I did npm i after deleting directory ,doing git clone and revising package.json

danielwind commented 3 years ago

@keisan1231

Can you please try to downgrade tfjs to 2.0.0 ?

"@tensorflow/tfjs": "2.0.0"

Let me know if this works.

Thanks

keisan1231 commented 3 years ago

@danielwind

I tried "@tensorflow/tfjs": "2.0.0" and this pattern #3 by this way

But it did not work😰.

danielwind / pictionary-tutorial

onReady is not called in TensorCamera #1

one

two

three