Closed rbayuokt closed 10 months ago
I think I've been able to fix this by using useSharedValue
from react-native-worklets-core
instead of reanimated
.
Per https://github.com/mrousavy/react-native-vision-camera/pull/1466,
Reanimated is no longer used as a Worklet Runtime. Instead, VisionCamera now uses react-native-worklets-core.
However, this presents a problem if I want to use the shared state in an animation. Reanimated doesn't recognize when changes to the state occur, so they don't trigger animation update.
Yea useSharedValue
is from RN Worklets Core, not from Reanimated.
Yea
useSharedValue
is from RN Worklets Core, not from Reanimated.
mmm, it still doesn't work when I pass it into useAnimatedStyle
. The line is not rendered. I was using react-native-svg
and converted the Line
into an AnimatedLine
with react-native-reanimated's 'createAnimatedComponent'. I'm sorry, but is there any way to draw an overlay like an animated line?
the native modules for Android and iOS is perfectly connected, it shows the values. I'm just missing the last step for drawing the overlay 😢 after it's finish I will publish this vision-camera frame processor library
I also have the same issue. I've successfully created my own version of vision-camera-ocr that does text recognition. Using latest react-native-vision-camera (v3).
The missing piece is how to get data from the worker in JS, out to my react component, and rendered.
When using "Worklets.createRunInJsFn" the app (iOS) crashes, when calling my backToReactThread function ^ It crashes in Native code, inside jsi::String JSCRuntime::createStringFromUtf8 calling: JSStringCreateWithUTF8CString
And using useShareValue (from Worklets) seems to have no affect. useAnimatedStyle ignores it..
How can we get some of the results/data from our useFrameProcessor javascript function, to the rest of react native so we can draw ui updates?
Hey! So VisionCamera V3 uses react-native-worklets-core, and react-native-reanimated uses it's own Worklets implementation, so the useSharedValue
is not compatible with the one from Reanimated (I think we could add interop support for that though, wdyt @tomekzaw?)
What you can do (and what I would suggest you to do anyways regardless of this issue) is to draw the entire overlay using @Shopify/react-native-skia. This is much faster and flexible/powerful for drawing such overlays. Use the draw callback to draw, that should also work with useSharedValue
. (afaik, right @chrfalch?)
Thank you @mrousavy ! Btw, is there a type-o in the offiical docs where it says:
"(Optional) If you want to use Frame Processors, you need to install react-native-worklets-core 1.0.0 or higher."
I think react-native-worklets-core only goes up to version 0.2.0.. I don't see 1.0.0 ?
Separately, I am still seeing the crash inside jsi::String JSCRuntime::createStringFromUtf8 calling: JSStringCreateWithUTF8CString . Do you have any tips or hints on where I can look for that? I wonder if it's because I have 0.2.0 and not 1.0.0 of react-native-worklets-core (which is no longer available on npm)?
@mrousavy I wonder if you could share a package.json from a working setup. Curious to see what version of react-native, react-native-worklets-core, and react-native-vision-camera are all working well together.
Package.json; the one here in example/
app.
And yea, you're right the docs should say 0.2.0 instead of 1.0.0.
I need a reproduceable example for that
@mrousavy thank you. that example was super helpful. I managed to get it building and running. In the end I had to:
After that, I was good to go. Thank you so much for your help.
Oh interesting! Could you maybe send a PR to react-native-worklets-core to fix the _closure issue? Thanks! cc @chrfalch
I'll close this for now, let me know if this issue is still active and we can re-open it
just fyi, in https://github.com/software-mansion/react-native-reanimated/pull/4803 we renamed _closure
to __closure
for the purpose of consistency so that all fields start with a double underscore.
Ah shoot haha okay yea we'll rename as well then 😅
Thanks for the heads up Tomasz!
Hi, if somebody else stumbles on thid issue my (ugly) solution is to copy the useSharedValue variable from the main thread:
import { useSharedValue, Worklets } from 'react-native-worklets-core'
import { useSharedValue as useSharedValueR } from 'react-native-reanimated'
const pose = useSharedValue(getDefaultPose())
const poseR = useSharedValueR(getDefaultPose())
const updateSkeleton = Worklets.createRunInJsFn((p) => {
poseR.value = { ...p }
})
const frameProcessor = useFrameProcessor(frame => {
'worklet'
...
updateSkeleton(pose.value)
Will add compat for that soon!
Hi, if somebody else stumbles on thid issue my (ugly) solution is to copy the useSharedValue variable from the main thread:
import { useSharedValue, Worklets } from 'react-native-worklets-core' import { useSharedValue as useSharedValueR } from 'react-native-reanimated' const pose = useSharedValue(getDefaultPose()) const poseR = useSharedValueR(getDefaultPose()) const updateSkeleton = Worklets.createRunInJsFn((p) => { poseR.value = { ...p } }) const frameProcessor = useFrameProcessor(frame => { 'worklet' ... updateSkeleton(pose.value)
I tried above thing but seems not working UI is not updating or showing
Verify first that updating your reanimated shared value from js works , then put a console log in the Worklets.createRunInJsF to verify it has been called
I tried above thing but seems not working UI is not updating or showing
Verify first that updating your reanimated shared value from js works, then put a console log in the Worklets.createRunInJsF to verify it has been called
I tried above thing but seems not working UI is not updating or showing
ys function has been calling but the UI is not updating unless a hard refresh means I have to update the state value to show the block. I'm using the latest version of vision-camera
Same, it is not working for me. @umbertoghio what version vc, wc, and reanimated are you using?
edit: nevermind, got it to work. Thanks for the fix!
@jslok may you send some snippets to make it workable? and please mention relevant versions for reanimated and vision camera and worklet-core
@jslok may you send some snippets to make it workable? and please mention relevant versions for reanimated and vision camera and worklet-core
vc 3.6.16 worklet-core 0.24 reaniated 3.6.1
Could not save an object in useSharedValue so I had to split it up into four different ones.
const rectWidth = useSharedValue(100) // rect width
const rectHeight = useSharedValue(100) // rect height
const rectX = useSharedValue(100) // rect x position
const rectY = useSharedValue(100) // rect y position
const rectWidthR = useSharedValueR(100) // rect width
const rectHeightR = useSharedValueR(100) // rect height
const rectXR = useSharedValueR(100) // rect x position
const rectYR = useSharedValueR(100) // rect y position
const updateRect = Worklets.createRunInJsFn((frame) => {
rectWidthR.value = frame.width
rectHeightR.value = frame.height
rectXR.value = frame.x
rectYR.value = frame.y
})
const frameProcessor = useFrameProcessor((frame) => {
'worklet'
const rectangle = cardDetect(frame)
if (rectangle.x) {
const { width: frameWidth, height: frameHeight } = frame
const xFactor = windowWidth / frameWidth
const yFactor = windowHeight / frameHeight
//return
// Assign animated values to rect shared values
rectWidth.value = rectangle.width * xFactor
rectHeight.value = rectangle.height * yFactor
rectX.value = rectangle.x * xFactor
rectY.value = rectangle.y * yFactor
updateRect({
width: rectWidth.value,
height: rectHeight.value,
x: rectX.value,
y: rectY.value,
})
}
}, [])
// Animated styles for the box
const animatedStyle = useAnimatedStyle(() => {
return {
width: withSpring(rectWidthR.value),
height: withSpring(rectHeightR.value),
transform: [
{ translateX: withSpring(rectXR.value) },
{ translateY: withSpring(rectYR.value) },
],
}
})
What's happening?
hi Marc, I'm currently working on developing a
vision-camera-pose-detection
project with MLKit. I initially wrote the native code for iOS, and it connected perfectly. However, I encountered a problem when attempting to draw the overlay for the skeleton based on the keypoints.In general, I used
useSharedValue
from Reanimated V3 to pass a value. when I useuseSharedValue
I receive an error message like this:Reading from '_value' directly is only possible on the UI runtime.
I have already wrapped my JS function into a worklet, but I still get an error.
If I pass the value into
setState
it works, but it doesn't behave as expected. there is a delay, as shown in this video. this is my friend who tested ithttps://github.com/mrousavy/react-native-vision-camera/assets/49748249/1517267b-fcd5-4125-9715-97a561803644
"react-native-vision-camera": "^3.0.0", "react-native-worklets-core": "^0.2.0", "react-native-reanimated": "^3.4.2"
Reproduceable Code
PoseDetection.ts
Main Code:
Relevant log output
Camera Device
No response
Device
iPhone 11 ( 15.5 )
VisionCamera Version
3.0.0
Can you reproduce this issue in the VisionCamera Example app?
Additional information