Closed canibal2 closed 9 months ago
Did you install pods? Do you have any logs? Does it work outside a frame processor?
Yes I installed pods, but I test it in the android, and just I have this error "Cannot read property 'runSync' of undefined". But have not try in the outside of the frame processor.
The following code "const r = model.model.run([new Uint8Array([5])])" works, but it does not work when put into the frameprocessor (commented code).
import * as React from 'react'
import { useEffect } from 'react'
import { StyleSheet, View, Text, Platform } from 'react-native'
import {
useTensorflowModel,
} from 'react-native-fast-tflite'
import { Camera, useCameraDevice, useCameraPermission, useFrameProcessor } from 'react-native-vision-camera'
export default function App() {
const [result, setResult] = React.useState('')
const device = useCameraDevice("front")
const { hasPermission, requestPermission } = useCameraPermission();
useEffect(() => {
if (!hasPermission) {
requestPermission();
}
}, []);
const model = useTensorflowModel(
require('../assets/android.tflite'),
Platform.OS === 'ios' ? 'core-ml' : 'default'
)
const frameProcessor = useFrameProcessor((frame) => {
'worklet'
if (model.state !== "loaded") return
// const data = frame.toArrayBuffer()
// do RGB conversion if the Frame is not already in RGB Format
// const r = model.model.run([new Uint8Array([5])])
// console.log(r);
// const detection_boxes = outputs[0]
// const detection_classes = outputs[1]
// const detection_scores = outputs[2]
// const num_detections = outputs[3]
// console.log(`Detected ${num_detections![0]} objects!`)
// for (let i = 0; i < detection_boxes!.length; i += 4) {
// const confidence = detection_scores![i / 4]
// if (confidence! > 0.7) {
// // Draw a red box around the object!
// const left = detection_boxes![i]
// const top = detection_boxes![i + 1]
// const right = detection_boxes![i + 2]
// const bottom = detection_boxes![i + 3]
// console.log(left);
// console.log(top);
// console.log(right);
// }
// }
}, [model])
React.useEffect(() => {
if (model.model == null) return
console.log(`Running Model...`)
const r = model.model.run([new Uint8Array([5])])
r.then((output) => {
console.log(`Successfully ran Model!`, output)
setResult(`${output[0]}${output[1]}${output[2]}...`)
})
}, [model.model])
return (
<Camera
device={device!}
isActive
style={styles.camera}
frameProcessor={frameProcessor}
//ML Kit use YUV format
pixelFormat="yuv"
/>
)
}
const CAMERA_SIZE = 250;
const styles = StyleSheet.create({
container: {
flex: 1,
alignItems: 'center',
justifyContent: 'center',
},
cameraContainer: {
width: 450,
height: 500,
borderRadius: CAMERA_SIZE / 2,
marginVertical: 24,
},
camera: {
flex: 1,
},
});
and this is logcat log for the android. I filtered with com.tfliteexample
2023-12-26 10:51:43.691 7492-7492 ViewRootIm...w:16b4d4d] com.tfliteexample I setView = android.widget.PopupWindow$PopupDecorView@fac7149 TM=true
2023-12-26 10:51:43.721 7492-8238 ViewRootIm...w:16b4d4d] com.tfliteexample I Resizing android.view.ViewRootImpl@e66aa4e: frame=[0,83][1080,165] reportDraw=true forceLayout=false backDropFrame=Rect(0, 0 - 1080, 82)
2023-12-26 10:51:43.727 7492-7492 ViewRootIm...w:16b4d4d] com.tfliteexample I Relayout returned: old=(0,83,1080,2214) new=(0,83,1080,165) req=(1079,82)0 dur=19 res=0x7 s={true 481170579456} ch=true fn=-1
2023-12-26 10:51:43.731 7492-7604 OpenGLRenderer com.tfliteexample D eglCreateWindowSurface
2023-12-26 10:51:43.736 7492-7492 ViewRootIm...w:16b4d4d] com.tfliteexample I [DP] dp(1) 0 android.view.ViewRootImpl.reportNextDraw:11442 android.view.ViewRootImpl.performTraversals:4198 android.view.ViewRootImpl.doTraversal:2924
2023-12-26 10:51:43.736 7492-7492 ViewRootIm...w:16b4d4d] com.tfliteexample D Creating frameDrawingCallback nextDrawUseBlastSync=false reportNextDraw=true hasBlurUpdates=false
2023-12-26 10:51:43.736 7492-7492 ViewRootIm...w:16b4d4d] com.tfliteexample D Creating frameCompleteCallback
2023-12-26 10:51:43.743 7492-7634 ViewRootIm...w:16b4d4d] com.tfliteexample D Received frameDrawingCallback frameNum=1. Creating transactionCompleteCallback=false
2023-12-26 10:51:43.747 7492-7604 ViewRootIm...w:16b4d4d] com.tfliteexample D Received frameCompleteCallback lastAcquiredFrameNum=1 lastAttemptedDrawFrameNum=1
2023-12-26 10:51:43.750 7492-7492 ViewRootIm...w:16b4d4d] com.tfliteexample I [DP] pdf(0) 0 android.view.ViewRootImpl.lambda$addFrameCompleteCallbackIfNeeded$3$ViewRootImpl:5000 android.view.ViewRootImpl$$ExternalSyntheticLambda16.run:6 android.os.Handler.handleCallback:938
2023-12-26 10:51:43.751 7492-7492 ViewRootIm...w:16b4d4d] com.tfliteexample I [DP] rdf()
2023-12-26 10:51:43.751 7492-7492 ViewRootIm...w:16b4d4d] com.tfliteexample D reportDrawFinished (fn: -1)
2023-12-26 10:51:43.775 4906-4906 SurfaceFlinger surfaceflinger D Display 0 HWC layers:
DEVICE | 0x7abeb0d080 | 0100 | RGBA_8888 | 0.0 0.0 1080.0 2340.0 | 0 0 1080 2340 | com.tfliteexample/com.tfliteexample.MainActivity$_7492#0
DEVICE | 0x7ae4ac04c0 | 0100 | RGBA_8888 | 0.0 0.0 1080.0 82.0 | 0 83 1080 165 | PopupWindow:16b4d4d$_7492#0
DEVICE | 0x7abeb0ba00 | 0100 | RGBA_8888 | 0.0 0.0 1080.0 2340.0 | 0 0 1080 2340 | com.tfliteexample/com.tfliteexample.MainActivity$_7492#1
DEVICE | 0x7ae4ac0380 | 0100 | RGBA_8888 | 0.0 0.0 1080.0 2340.0 | 0 0 1080 2340 | com.tfliteexample/com.tfliteexample.MainActivity$_7492#2
DEVICE | 0x7ab5f07c40 | 0100 | RGBA_8888 | 0.0 0.0 1080.0 2340.0 | 0 0 1080 2340 | com.tfliteexample/com.tfliteexample.MainActivity$_7492#3
DEVICE | 0x7aab792580 | 0100 | RGBA_8888 | 0.0 0.0 1080.0 78.0 | 0 0 1080 78 | StatusBar$_5542#0
DEVICE | 0x7abeb0b8c0 | 0100 | RGBA_8888 | 0.0 0.0 1080.0 126.0 | 0 2214 1080 2340 | Naviga
2023-12-26 10:51:43.776 7492-7492 ViewRootIm...w:16b4d4d] com.tfliteexample I Relayout returned: old=(0,83,1080,165) new=(0,83,1080,165) req=(1079,82)0 dur=15 res=0x1 s={true 481170579456} ch=false fn=2
2023-12-26 10:51:43.786 7492-7492 ViewRootIm...w:16b4d4d] com.tfliteexample I handleResized, msg = 5 frames=ClientWindowFrames{frame=[0,83][1080,165] display=[0,83][1080,2214] backdrop=[0,0][1080,82]} forceNextWindowRelayout=false displayId=0 frameChanged=false backdropFrameChanged=false configChanged=false displayChanged=false mNextDrawUseBlastSync=false mergedConfiguration={mGlobalConfig={1.3 ?mcc?mnc [en_US,tr_TR] ldltr sw411dp w411dp h811dp 420dpi nrml long port finger -keyb/v/h -nav/h winConfig={ mBounds=Rect(0, 0 - 1080, 2340) mAppBounds=Rect(0, 83 - 1080, 2214) mMaxBounds=Rect(0, 0 - 1080, 2340) mWindowingMode=fullscreen mDisplayWindowingMode=fullscreen mActivityType=undefined mAlwaysOnTop=undefined mRotation=ROTATION_0 mPopOver=off mFreeformTaskPinningState=unpinned} s.1174 fontWeightAdjustment=0 ff=0 bf=0 bts=0 enb/d themeSeq=0} mOverrideConfig={1.3 ?mcc?mnc [en_US,tr_TR] ldltr sw411dp w411dp h811dp 420dpi nrml long port finger -keyb/v/h -nav/h winConfig={ mBounds=Rect(0, 0 - 1080, 2340) mAppBounds=Rect(0, 83 - 1080, 2214) mMaxBounds=Rect(0, 0 - 1080, 2340) mWindowingMode=fullscreen mDisplayWindowingMode=fullscreen mActivityType=standard mAlwaysOnTop=undefined mRotation=ROTATION_0 mPopOver=off mFreeformTaskPinningState=unpinned} s.1 fontWeightAdjustment=0 ff=0 bf=0 bts=0 enb/d themeSeq=0}}
2023-12-26 10:51:43.787 7492-7492 ViewRootIm...w:16b4d4d] com.tfliteexample I [DP] dp(1) 0 android.view.ViewRootImpl.reportNextDraw:11442 android.view.ViewRootImpl.handleResized:2235 android.view.ViewRootImpl.access$1200:303
2023-12-26 10:51:43.805 7492-7492 ViewRootIm...w:16b4d4d] com.tfliteexample I Relayout returned: old=(0,83,1080,165) new=(0,83,1080,165) req=(1079,82)0 dur=11 res=0x1 s={true 481170579456} ch=false fn=2
2023-12-26 10:51:43.806 7492-7492 ViewRootIm...w:16b4d4d] com.tfliteexample D Creating frameDrawingCallback nextDrawUseBlastSync=false reportNextDraw=true hasBlurUpdates=false
2023-12-26 10:51:43.806 7492-7492 ViewRootIm...w:16b4d4d] com.tfliteexample D Creating frameCompleteCallback
2023-12-26 10:51:43.807 7492-7635 ViewRootIm...w:16b4d4d] com.tfliteexample D Received frameDrawingCallback frameNum=2. Creating transactionCompleteCallback=false
2023-12-26 10:51:43.808 7492-7604 ViewRootIm...w:16b4d4d] com.tfliteexample D Received frameCompleteCallback lastAcquiredFrameNum=1 lastAttemptedDrawFrameNum=2
2023-12-26 10:51:43.809 7492-7492 ViewRootIm...w:16b4d4d] com.tfliteexample I [DP] pdf(0) 0 android.view.ViewRootImpl.lambda$addFrameCompleteCallbackIfNeeded$3$ViewRootImpl:5000 android.view.ViewRootImpl$$ExternalSyntheticLambda16.run:6 android.os.Handler.handleCallback:938
2023-12-26 10:51:43.809 7492-7492 ViewRootIm...w:16b4d4d] com.tfliteexample I [DP] rdf()
2023-12-26 10:51:43.809 7492-7492 ViewRootIm...w:16b4d4d] com.tfliteexample D reportDrawFinished (fn: 2)
2023-12-26 10:51:43.810 7492-7492 ViewRootIm...w:16b4d4d] com.tfliteexample D Scheduling a traversal=false due to a previous skipped traversal.
2023-12-26 10:51:43.933 7492-7604 OpenGLRenderer com.tfliteexample D setSurface called with nullptr
2023-12-26 10:51:43.933 7492-7604 OpenGLRenderer com.tfliteexample D setSurface() destroyed EGLSurface
2023-12-26 10:51:43.933 7492-7604 OpenGLRenderer com.tfliteexample D destroyEglSurface
2023-12-26 10:51:43.935 7492-7492 ViewRootIm...w:16b4d4d] com.tfliteexample I dispatchDetachedFromWindow
2023-12-26 10:51:43.950 7492-7492 InputTransport com.tfliteexample D Input channel destroyed: '2a19f9e', fd=231
2023-12-26 10:51:43.962 4906-4906 SurfaceFlinger surfaceflinger D Display 0 HWC layers:
DEVICE | 0x7abeb0d080 | 0100 | RGBA_8888 | 0.0 0.0 1080.0 2340.0 | 0 0 1080 2340 | com.tfliteexample/com.tfliteexample.MainActivity$_7492#0
DEVICE | 0x7abeb0ba00 | 0100 | RGBA_8888 | 0.0 0.0 1080.0 2340.0 | 0 0 1080 2340 | com.tfliteexample/com.tfliteexample.MainActivity$_7492#1
DEVICE | 0x7ae4ac0380 | 0100 | RGBA_8888 | 0.0 0.0 1080.0 2340.0 | 0 0 1080 2340 | com.tfliteexample/com.tfliteexample.MainActivity$_7492#2
DEVICE | 0x7ab5f07c40 | 0100 | RGBA_8888 | 0.0 0.0 1080.0 2340.0 | 0 0 1080 2340 | com.tfliteexample/com.tfliteexample.MainActivity$_7492#3
DEVICE | 0x7aab792580 | 0100 | RGBA_8888 | 0.0 0.0 1080.0 78.0 | 0 0 1080 78 | StatusBar$_5542#0
DEVICE | 0x7abeb0b8c0 | 0100 | RGBA_8888 | 0.0 0.0 1080.0 126.0 | 0 2214 1080 2340 | NavigationBar0$_5542#0
DEVICE | 0x7ab5f05a80 | 0140 | RGBA_8888 | 0.0 0.0 1080.0 151.0 | 0 0 1080 151 | ScreenDecor
2023-12-26 10:51:44.007 7492-7492 unknown:ReactNative com.tfliteexample W Packager connection already open, nooping.
2023-12-26 10:51:44.024 7492-7492 unknown:Re...nceManager com.tfliteexample E destroyRootView called
2023-12-26 10:51:44.024 7492-7492 unknown:Re...nceManager com.tfliteexample E destroyRootView called, unmountReactApplication
2023-12-26 10:51:44.061 7492-9362 unknown:ReactContext com.tfliteexample W initializeMessageQueueThreads() is called.
2023-12-26 10:51:44.069 7492-7492 unknown:ReactNative com.tfliteexample W Packager connection already open, nooping.
2023-12-26 10:51:44.075 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.views.traceupdateoverlay.TraceUpdateOverlayManager
2023-12-26 10:51:44.103 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.uimanager.LayoutShadowNode
2023-12-26 10:51:44.119 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.views.drawer.ReactDrawerLayoutManager
2023-12-26 10:51:44.127 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.views.scroll.ReactHorizontalScrollViewManager
2023-12-26 10:51:44.140 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.views.scroll.ReactHorizontalScrollContainerViewManager
2023-12-26 10:51:44.144 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.views.progressbar.ReactProgressBarViewManager
2023-12-26 10:51:44.149 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.views.progressbar.ProgressBarShadowNode
2023-12-26 10:51:44.152 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.views.scroll.ReactScrollViewManager
2023-12-26 10:51:44.161 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.views.switchview.ReactSwitchManager
2023-12-26 10:51:44.167 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.views.switchview.ReactSwitchManager$ReactSwitchShadowNode
2023-12-26 10:51:44.169 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.views.swiperefresh.SwipeRefreshLayoutManager
2023-12-26 10:51:44.174 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.views.text.frescosupport.FrescoBasedReactTextInlineImageViewManager
2023-12-26 10:51:44.176 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.views.text.frescosupport.FrescoBasedReactTextInlineImageShadowNode
2023-12-26 10:51:44.180 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.views.image.ReactImageManager
2023-12-26 10:51:44.186 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.views.modal.ReactModalHostManager
2023-12-26 10:51:44.191 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.views.modal.ModalHostShadowNode
2023-12-26 10:51:44.193 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.views.text.ReactRawTextManager
2023-12-26 10:51:44.194 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.views.text.ReactRawTextShadowNode
2023-12-26 10:51:44.196 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.views.textinput.ReactTextInputManager
2023-12-26 10:51:44.208 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.views.textinput.ReactTextInputShadowNode
2023-12-26 10:51:44.217 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.views.text.ReactTextViewManager
2023-12-26 10:51:44.223 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.views.text.ReactTextShadowNode
2023-12-26 10:51:44.226 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.views.view.ReactViewManager
2023-12-26 10:51:44.238 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.views.text.ReactVirtualTextViewManager
2023-12-26 10:51:44.241 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.views.text.ReactVirtualTextShadowNode
2023-12-26 10:51:44.245 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.facebook.react.views.unimplementedview.ReactUnimplementedViewManager
2023-12-26 10:51:44.249 7492-9365 unknown:Vi...rtyUpdater com.tfliteexample W Could not find generated setter for class com.mrousavy.camera.CameraViewManager
2023-12-26 10:51:44.258 7492-9365 unknown:ReactContext com.tfliteexample W initializeMessageQueueThreads() is called.
2023-12-26 10:51:44.263 7492-9365 CameraManager com.tfliteexample I registerAvailabilityCallback: Is device callback = false
2023-12-26 10:51:44.264 7492-9365 CameraManagerGlobal com.tfliteexample I postSingleUpdate device: camera id 0 status STATUS_PRESENT
2023-12-26 10:51:44.264 7492-9365 CameraManagerGlobal com.tfliteexample I postSingleUpdate device: camera id 1 status STATUS_NOT_AVAILABLE
2023-12-26 10:51:44.264 7492-9365 CameraManagerGlobal com.tfliteexample I postSingleUpdate device: camera id 2 status STATUS_PRESENT
2023-12-26 10:51:44.265 7492-9365 CameraManagerGlobal com.tfliteexample I postSingleUpdate device: camera id 3 status STATUS_PRESENT
2023-12-26 10:51:44.266 7492-9365 CameraDevices com.tfliteexample I Camera #0: Available!
2023-12-26 10:51:44.266 7492-9365 CameraDevices com.tfliteexample I Camera #1: Unavailable!
2023-12-26 10:51:44.270 7492-9365 CameraDevices com.tfliteexample I Camera #2: Available!
2023-12-26 10:51:44.270 7492-9365 CameraDevices com.tfliteexample I Camera #3: Available!
2023-12-26 10:51:44.945 7492-9364 ReactNativeJS com.tfliteexample I Running "TfliteExample" with {"rootTag":321}
2023-12-26 10:51:45.413 7492-9364 ReactNativeJS com.tfliteexample I Loading react-native-worklets-core...
2023-12-26 10:51:45.414 7492-9364 ReactNativeJS com.tfliteexample I Worklets loaded successfully
2023-12-26 10:51:45.425 7492-9364 VisionCameraProxy com.tfliteexample I Initializing VisionCameraProxy...
2023-12-26 10:51:45.425 7492-9364 VisionCameraProxy com.tfliteexample I Creating Worklet Context...
2023-12-26 10:51:45.429 7492-9364 VisionCameraProxy com.tfliteexample I Worklet Context created!
2023-12-26 10:51:45.441 7492-9364 ReactNativeJS com.tfliteexample I Installing bindings...
2023-12-26 10:51:45.444 7492-9364 Tflite com.tfliteexample I Loading C++ library...
2023-12-26 10:51:45.444 7492-9364 Tflite com.tfliteexample I Installing JSI Bindings for VisionCamera Tflite plugin...
2023-12-26 10:51:45.445 7492-9364 Tflite com.tfliteexample I Successfully installed JSI Bindings!
2023-12-26 10:51:45.445 7492-9364 ReactNativeJS com.tfliteexample I Successfully installed!
2023-12-26 10:51:47.735 7492-9364 ReactNativeJS com.tfliteexample E TypeError: Cannot read property 'runSync' of undefined
This error is located at:
in App
in RCTView (created by View)
in View (created by AppContainer)
in RCTView (created by View)
in View (created by AppContainer)
in AppContainer
in TfliteExample(RootComponent), js engine: hermes
2023-12-26 10:51:47.741 7492-9364 ReactNativeJS com.tfliteexample E TypeError: Cannot read property 'runSync' of undefined
This error is located at:
in App
in RCTView (created by View)
in View (created by AppContainer)
in RCTView (created by View)
in View (created by AppContainer)
in AppContainer
in TfliteExample(RootComponent), js engine: hermes
2023-12-26 10:51:47.925 7492-7492 Dialog com.tfliteexample I mIsDeviceDefault = false, mIsSamsungBasicInteraction = false, isMetaDataInActivity = false
2023-12-26 10:51:47.940 7492-7492 DecorView com.tfliteexample I [INFO] isPopOver=false, config=false
2023-12-26 10:51:47.941 7492-7492 DecorView com.tfliteexample I updateCaptionType >> DecorView@5d50bf1[], isFloating=false, isApplication=true, hasWindowControllerCallback=false, hasWindowDecorCaption=false
2023-12-26 10:51:47.942 7492-7492 DecorView com.tfliteexample D setCaptionType = 0, this = DecorView@5d50bf1[]
2023-12-26 10:51:47.951 7492-7492 DecorView com.tfliteexample I getCurrentDensityDpi: from app context. densityDpi=420 msg=resources_loaded
2023-12-26 10:51:47.970 7492-7492 DecorView com.tfliteexample I notifyKeepScreenOnChanged: keepScreenOn=false
2023-12-26 10:51:47.972 7492-7492 MSHandlerLifeCycle com.tfliteexample I isMultiSplitHandlerRequested: ignored. pkg=com.tfliteexample isActivity=false callers=com.android.internal.policy.DecorView.hasMultiSplitHandlerInNonFreeformMode:891 com.android.internal.policy.DecorView.notifyKeepScreenOnChanged:885 android.view.ViewRootImpl.applyKeepScreenOnFlag:2945 android.view.ViewRootImpl.collectViewAttributes:2969 android.view.ViewRootImpl.setView:1558
2023-12-26 10:51:47.976 4906-6214 SurfaceFlinger surfaceflinger I id=24172 createSurf (0x0),-1 flag=80004, 381bb49 com.tfliteexample/com.tfliteexample.MainActivity#0
2023-12-26 10:51:47.978 5200-7328 WindowManager system_server V Changing focus from Window{bd7f0f2 u0 com.tfliteexample/com.tfliteexample.MainActivity} to Window{381bb49 u0 com.tfliteexample/com.tfliteexample.MainActivity} displayId=0 Callers=com.android.server.wm.RootWindowContainer.updateFocusedWindowLocked:583 com.android.server.wm.WindowManagerService.updateFocusedWindowLocked:6497 com.android.server.wm.WindowManagerService.addWindow:2050 com.android.server.wm.Session.addToDisplayAsUser:211
2023-12-26 10:51:47.978 5200-7328 MARsPolicyManager system_server D onPackageResumedFG pkgName = com.tfliteexample, userId = 0
2023-12-26 10:51:47.992 7492-7492 ViewRootIm...nActivity] com.tfliteexample I setView = com.android.internal.policy.DecorView@5d50bf1 TM=true
2023-12-26 10:51:48.009 7492-7492 unknown:ReactContext com.tfliteexample W initializeMessageQueueThreads() is called.
2023-12-26 10:51:48.016 5200-7328 WindowManager system_server V Relayout Window{381bb49 u0 com.tfliteexample/com.tfliteexample.MainActivity}: viewVisibility=0 req=1080x2340 d0
2023-12-26 10:51:48.017 4906-4921 SurfaceFlinger surfaceflinger I id=24175 createSurf (1x1),-3 flag=40004, com.tfliteexample/com.tfliteexample.MainActivity$_7492#4
2023-12-26 10:51:48.018 5200-7328 WindowManager system_server D makeSurface duration=1 name=com.tfliteexample/com.tfliteexample.MainActivity$_7492
2023-12-26 10:51:48.030 5200-7328 InputDispatcher system_server D Focus request (0): 381bb49 com.tfliteexample/com.tfliteexample.MainActivity but waiting because NOT_VISIBLE
2023-12-26 10:51:48.031 5200-7328 InputDispatcher system_server D Focus left window (0): bd7f0f2 com.tfliteexample/com.tfliteexample.MainActivity
2023-12-26 10:51:48.034 7492-7492 InsetsSourceConsumer com.tfliteexample D ensureControlAlpha: for ITYPE_NAVIGATION_BAR on com.tfliteexample/com.tfliteexample.MainActivity
2023-12-26 10:51:48.036 7492-7492 InsetsSourceConsumer com.tfliteexample D ensureControlAlpha: for ITYPE_STATUS_BAR on com.tfliteexample/com.tfliteexample.MainActivity
2023-12-26 10:51:48.039 7492-7492 ViewRootIm...nActivity] com.tfliteexample I Relayout returned: old=(0,0,1080,2340) new=(0,0,1080,2340) req=(1080,2340)0 dur=15 res=0x7 s={true 481170579456} ch=true fn=-1
2023-12-26 10:51:48.041 7492-7604 OpenGLRenderer com.tfliteexample D eglCreateWindowSurface
2023-12-26 10:51:48.051 7492-7492 ViewRootIm...nActivity] com.tfliteexample I [DP] dp(1) 1 android.view.ViewRootImpl.reportNextDraw:11442 android.view.ViewRootImpl.performTraversals:4198 android.view.ViewRootImpl.doTraversal:2924
2023-12-26 10:51:48.051 7492-7492 ViewRootIm...nActivity] com.tfliteexample D Creating frameDrawingCallback nextDrawUseBlastSync=false reportNextDraw=true hasBlurUpdates=false
2023-12-26 10:51:48.051 7492-7492 ViewRootIm...nActivity] com.tfliteexample D Creating frameCompleteCallback
2023-12-26 10:51:48.056 7492-7634 ViewRootIm...nActivity] com.tfliteexample D Received frameDrawingCallback frameNum=1. Creating transactionCompleteCallback=false
2023-12-26 10:51:48.064 7492-7604 ViewRootIm...nActivity] com.tfliteexample D Received frameCompleteCallback lastAcquiredFrameNum=1 lastAttemptedDrawFrameNum=1
2023-12-26 10:51:48.066 7492-7492 ViewRootIm...nActivity] com.tfliteexample I [DP] pdf(0) 1 android.view.ViewRootImpl.lambda$addFrameCompleteCallbackIfNeeded$3$ViewRootImpl:5000 android.view.ViewRootImpl$$ExternalSyntheticLambda16.run:6 android.os.Handler.handleCallback:938
2023-12-26 10:51:48.066 7492-7492 ViewRootIm...nActivity] com.tfliteexample I [DP] rdf()
2023-12-26 10:51:48.066 7492-7492 ViewRootIm...nActivity] com.tfliteexample D reportDrawFinished (fn: -1)
2023-12-26 10:51:48.067 5200-8650 WindowManager system_server D finishDrawingWindow: Window{381bb49 u0 com.tfliteexample/com.tfliteexample.MainActivity} mDrawState=DRAW_PENDING
2023-12-26 10:51:48.070 7492-7492 ViewRootIm...nActivity] com.tfliteexample I MSG_WINDOW_FOCUS_CHANGED 0 1
2023-12-26 10:51:48.072 5200-5241 WindowManager system_server I Reparenting to leash, surface=Surface(name=381bb49 com.tfliteexample/com.tfliteexample.MainActivity)/@0xfb0158b, leashParent=Surface(name=ActivityRecord{74ff1f4 u0 com.tfliteexample/.MainActivity t451})/@0xa2887fb
2023-12-26 10:51:48.073 4906-4922 SurfaceFlinger surfaceflinger I id=24176 createSurf (0x0),-1 flag=24004, Surface(name=381bb49 com.tfliteexample/com.tfliteexample.MainActivity)/@0xfb0158b - animation-leash of window_animation#0
2023-12-26 10:51:48.074 5200-5241 WindowManager system_server D makeSurface duration=1 leash=Surface(name=Surface(name=381bb49 com.tfliteexample/com.tfliteexample.MainActivity)/@0xfb0158b - animation-leash of window_animation)/@0x90bb668
2023-12-26 10:51:48.081 5200-8650 InputDispatcher system_server D Focus entered window (0): 381bb49 com.tfliteexample/com.tfliteexample.MainActivity
2023-12-26 10:51:48.083 7492-7492 ViewRootIm...nActivity] com.tfliteexample I MSG_WINDOW_FOCUS_CHANGED 1 1
2023-12-26 10:51:48.085 7492-7492 InputMethodManager com.tfliteexample D startInputInner - Id : 0
2023-12-26 10:51:48.085 7492-7492 InputMethodManager com.tfliteexample I startInputInner - mService.startInputOrWindowGainedFocus
2023-12-26 10:51:48.120 4906-4906 SurfaceFlinger surfaceflinger D Display 0 HWC layers:
DEVICE | 0x7abeb0e0c0 | 0100 | RGBA_8888 | 0.0 0.0 1080.0 2340.0 | 0 0 1080 2340 | com.tfliteexample/com.tfliteexample.MainActivity$_7492#0
DEVICE | 0x7abeb0ba00 | 0100 | RGBA_8888 | 0.0 0.0 1080.0 2340.0 | 0 0 1080 2340 | com.tfliteexample/com.tfliteexample.MainActivity$_7492#1
DEVICE | 0x7ae4ac0380 | 0100 | RGBA_8888 | 0.0 0.0 1080.0 2340.0 | 0 0 1080 2340 | com.tfliteexample/com.tfliteexample.MainActivity$_7492#2
DEVICE | 0x7ab5f07ec0 | 0100 | RGBA_8888 | 0.0 0.0 1080.0 2340.0 | 0 0 1080 2340 | com.tfliteexample/com.tfliteexample.MainActivity$_7492#3
DEVICE | 0x7ae4abf840 | 0100 | RGBA_8888 | 0.0 0.0 1080.0 242.0 | 0 2098 1080 2340 | com.tfliteexample/com.tfliteexample.MainActivity$_7492#4
DEVICE | 0x7ab5f07b00 | 0100 | RGBA_8888 | 0.0 0.0 1080.0 78.0 | 0 0 1080 78 | StatusBar$_5542#0
DEVICE | 0x7abeb0b8c0 | 0100 | RGBA_8888 | 0.0 0.0 1080.0 126.0 |
2023-12-26 10:51:48.284 5200-5241 WindowManager system_server I Reparenting to original parent: Surface(name=ActivityRecord{74ff1f4 u0 com.tfliteexample/.MainActivity t451})/@0xa2887fb, destroy=true, surface=Surface(name=381bb49 com.tfliteexample/com.tfliteexample.MainActivity)/@0xfb0158b
2023-12-26 10:51:48.295 4906-4906 Layer surfaceflinger I id=24176 removeFromCurrentState Surface(name=381bb49 com.tfliteexample/com.tfliteexample.MainActivity)/@0xfb0158b - animation-leash of window_animation#0 (112)
2023-12-26 10:51:48.295 4906-4906 SurfaceFlinger surfaceflinger I id=24176 Removed Surface(name=381bb49 com.tfliteexample/com.tfliteexample.MainActivity)/@0xfb0158b - animation-leash of window_animation#0 (112)
2023-12-26 10:51:48.296 4906-4906 Layer surfaceflinger I id=24176 Destroyed Surface(name=381bb49 com.tfliteexample/com.tfliteexample.MainActivity)/@0xfb0158b - animation-leash of window_animation#0
2023-12-26 10:51:48.591 5200-5240 SGM:GameManager system_server D identifyForegroundApp. com.tfliteexample, mCurrentUserId: 0, callerUserId: 0
2023-12-26 10:51:48.591 5200-5240 SGM:PkgDataHelper system_server D getGamePkgData(). com.tfliteexample
2023-12-26 10:51:49.069 7492-7492 ScrollView com.tfliteexample D initGoToTop
Hi ! I have the same here, everything was working fine and since this morning, it shows the error out of nowhere, nothing in my code has changed since
I started working on the example package
@chourabieure
Do you know when you first used it or which version you used? Maybe it's related to version 1.0.0.
@chourabieure Do you know when you first used it or which version you used? Maybe it's related to version 1.0.0.
I first used it couples of days ago, and did not change any version. Update on that though : It seems to work on my friend's laptop with his own xcode config .. it's pretty weird
Hm, so the error basically says model.model
is undefined
. However, model
itself is not undefined
, and it properly has a model.state
.
Please try logging all keys: console.log(model.state, Object.keys(model))
so we can figure out what's wrong.
just after loading the model i got : loaded {"0": "model", "1": "state"}
model.model :
{
"delegate": "default",
"inputs": [
{
"dataType": "int8",
"name": "normalized_input_image_tensor",
"shape": [
Array
]
}
],
"outputs": [
{
"dataType": "float32",
"name": "TFLite_Detection_PostProcess",
"shape": [
Array
]
},
{
"dataType": "float32",
"name": "TFLite_Detection_PostProcess:1",
"shape": [
Array
]
},
{
"dataType": "float32",
"name": "TFLite_Detection_PostProcess:2",
"shape": [
Array
]
},
{
"dataType": "float32",
"name": "TFLite_Detection_PostProcess:3",
"shape": [
Array
]
}
],
"run": [
FunctionrunModel
],
"runSync": [
FunctionrunModel
]
}
I manage to remove the error by writing :
const frameProcessor = useFrameProcessor(
(frame) => {
'worklet'
if (model.state !== 'loaded') return
const data = frame.toArrayBuffer()
if (!model.model) return // <<<<<<<<
const outputs = model.model.runSync([data])
}
)
But it randomly goes through the test and sometimes model.model isn't even defined
UPDATE :
Reading frame.toArrayBuffer()
is extremely slow and sometimes crashs my app. I can't even print the frame anymore
I don't think it comes from the model but from the frame itself thus the data sent to runSync
Aha, interesting!
frame.toArrayBuffer()
is extremely slow
How slow exactly? This should be able to run in realtime. E.g. max. 5-10ms per call on 720p-1080p frames. Make sure to choose a small video resolution, not 4k in your format
.
Also, if you resize the array buffer, make sure to not allocate new ones each time but instead cache it.
I decreased the resolution and the fps Now it won't even print once
it crashes as soon as i try to print the frame
import * as React from 'react'
import { StyleSheet, View, Platform } from 'react-native'
import {
Camera,
useCameraDevice,
useCameraFormat,
useFrameProcessor,
} from 'react-native-vision-camera'
import { useTensorflowModel } from 'react-native-fast-tflite'
export default function App() {
const device = useCameraDevice('back')
const model = useTensorflowModel(
require('../assets/2.tflite'),
Platform.OS === 'ios' ? 'core-ml' : 'default'
)
const format = useCameraFormat(device, [
{ videoResolution: { width: 1280, height: 720 } },
])
const frameProcessor = useFrameProcessor(
(frame) => {
'worklet'
// console.log(frame) // <<<< The app crashes avery time I uncomment this line
},
[model]
)
if (!device) return null
return (
<View style={StyleSheet.absoluteFill}>
<Camera
frameProcessor={frameProcessor}
device={device}
isActive={true}
style={StyleSheet.absoluteFill}
format={format}
fps={10}
/>
</View>
)
}
It's getting worse and Iam not even touching any project configuration, just starting new clients
Hope it helps
Printing the Frame is not something that should work, I can add a debug description here though to make that work.
Check out this PR: https://github.com/mrousavy/react-native-vision-camera/pull/2327
I setup everything, also I use face detection, its work perfectly. But this package not worked. I get this error TypeError: Cannot read property 'runSync' of undefined . I try to implement this code:
can you help me?