mfkrause
commented
8 months ago This only seems to happen when using runAsync. If I use runAtTargetFps (tested with 15 FPS for now), it works seamlessly.
Closed mfkrause closed 6 months ago
mfkrause
commented
8 months ago This only seems to happen when using runAsync. If I use runAtTargetFps (tested with 15 FPS for now), it works seamlessly.
luicfrr
commented
8 months ago @mfkrause Hey, you're using my face detection package, glad it's helping you 😁
I had the same problem and as a workaround I added frame data inside detection result data, so instead of using frame.height try using result.frame.height.
For me this works.
The most strange thing is that on androi it works, you can use frame data directly without any issue but on IOS it freezes after some frames
bglgwyng
commented
8 months ago I made a minimal reproducer.
const frameProcessor = useFrameProcessor((frame) => {
'worklet'
runAsync(frame, () => {
'worklet'
foo(frame)
console.info('run frame processor')
})
}, [])And the definition of foo should be
const foo = (frame: Frame) => {
'worklet'
frame.height
}When foo doesn't include any access to frame's property(height here), the frame processor doesn't stop.
frodriguez-hu
commented
6 months ago Check this, get the result and operate on the current processor and not on the callback
https://github.com/nonam4/react-native-vision-camera-face-detector/issues/28
mrousavy
commented
6 months ago Hey - I think this issue has been fixed in VisionCamera 4.0.0. 🥳
Please try V4 and let me know if you still experience this issue;
mrousavy
commented
6 months ago more accurately; react-native-worklets-core 1.x.x fixed this issue I think.
bglgwyng
commented
5 months ago I think we need to reopen this issue. I still observe the same behavior with react-native-vision-camera@4.0.1 and react-native-worklets-core@1.2.0.
mrousavy
commented
5 months ago Ah shit. Well can we create a new issue with logs from latest react-native-vision-camera and rnwc then?
luicfrr
commented
5 months ago @mrousavy I created a new issue on worklets-core repo with all logs from latest versions
mrousavy
commented
5 months ago Great, thanks
luicfrr
commented
5 months ago @mrousavy I messed up with issues I'm subscribed 😅 this don't have anything to do with worklets-core. Sorry.
But anyway, what solved this "freeze" issue for me was incrementing frame ref count before doing anything else with frame. Ex:
const faces = detectFaces( frame )
const internal = frame as FrameInternal
// incremend frame ref before doing any task with frame
internal.incrementRefCount()
runOnJs(
faces,
frame
).finally( () => {
'worklet'
// decrement frame ref after all tasks
internal.decrementRefCount()
} )@bglgwyng Hope this helps
bglgwyng
commented
5 months ago Thanks! But isn't it a workaround? runAsync internally increases/decreases the ref count. Should we perform this redundant operation?
mrousavy
commented
5 months ago I fixed this issue in my latest PR to main. Ref counting now works properly on iOS as well
What's happening?
I'm trying to update some Shared Values from Reanimated (using
createRunInJsFn) based on the results of a frame processor plugin and the size of the frame itself. Specifically, I'm detecting faces using a plugin (which uses MLKit) and then want to normalize the detected bounding box based on the frame and screen size.However, the frame processor suddenly stops running after only a few frames. Console logs stop appearing and, if the result is drawn onto a Skia Canvas, that stops updating, too. The weird part: When I remove the normalization / multiplication from the code below and simply use the raw result bounding box from the plugin, the frame processor happily runs for seemingly forever. Only once I begin to multiply the bounding box with the frame size (and window size), the frame processor only runs for a few frames.
The problem is not specific to the frame processor plugin used in the reproducable code below. I could also reproduce it when using a custom model and react-native-fast-tflite.
Reproduceable Code
Relevant log output
Probably not relevant:
Camera Device
Device
iPhone 15 Pro (iOS 17.2)
VisionCamera Version
3.9.0
Can you reproduce this issue in the VisionCamera Example app?
I didn't try (⚠️ your issue might get ignored & closed if you don't try this)
Additional information