🐛 Function inside `runAsync` in `useSkiaFrameProcessor` runs only once

[lukaszkurantdev]

What's happening?

When I want to use runAsync inside useSkiaFrameProcessor, the worklet function inside runAsync runs only once – inside useFrameProcessor it works correctly. I'm using react-native-worklets-core v1.3.0 and I've enabled option processNestedWorklets in reanimated babel configuration.

For example, logs from first example looks like below:

But when I'm using useFrameProcessor (second example) it looks like this:

Reproduceable Code

// This doesn't work correctly ('test 2' fired only once)
const frameProcessor = useSkiaFrameProcessor(frame => {
  'worklet';

  console.log('test');

  runAsync(frame, () => {
    'worklet';
    console.log('test2');
  });

  frame.render();
}, []);

// This works
const frameProcessor = useFrameProcessor(frame => {
  'worklet';

  console.log('test');

  runAsync(frame, () => {
    'worklet';
    console.log('test2');
  });
}, []);

Relevant log output

Successfully registered Frame Processor Plugin "detectPose"!
Successfully registered Frame Processor Plugin "resize"!
flipper: FlipperClient::addPlugin Inspector
flipper: FlipperClient::addPlugin Preferences
flipper: FlipperClient::addPlugin React
flipper: FlipperClient::addPlugin Network
Running application posedetection2 ({
    initialProps =     {
    };
    rootTag = 1;
})
nw_path_necp_check_for_updates Failed to copy updated result (22)
SocketStream read error [0x302a74140]: 1 54
nw_protocol_socket_reset_linger [C1.1.1:1] setsockopt SO_LINGER failed [22: Invalid argument]
nw_socket_handle_socket_event [C6:1] Socket SO_ERROR [61: Connection refused]
nw_connection_get_connected_socket_block_invoke [C6] Client called nw_connection_get_connected_socket on unconnected nw_connection
TCP Conn 0x302a7d720 Failed : error 0:61 [61]
nw_connection_copy_connected_local_endpoint_block_invoke [C6] Client called nw_connection_copy_connected_local_endpoint on unconnected nw_connection
nw_connection_copy_connected_remote_endpoint_block_invoke [C6] Client called nw_connection_copy_connected_remote_endpoint on unconnected nw_connection
Unbalanced calls start/end for tag 19
Running "posedetection2" with {"rootTag":1,"initialProps":{}}
Loading react-native-worklets-core...
Worklets loaded successfully
VisionCameraProxy: Creating Worklet Context...
VisionCameraProxy: Worklet Context Created!
11:14:00.431: [info] 📸 VisionCamera.didSetProps(_:): Updating 17 props: [isActive, preview, onViewReady, right, cameraId, position, left, onError, onShutter, onStarted, onInitialized, enableBufferCompression, onStopped, bottom, enableFrameProcessor, onCodeScanned, top]
11:14:00.433: [info] 📸 VisionCamera.configure(_:): configure { ... }: Waiting for lock...
11:14:00.438: [info] 📸 VisionCamera.configure(_:): configure { ... }: Updating CameraSession Configuration... Difference(inputChanged: true, outputsChanged: true, videoStabilizationChanged: true, orientationChanged: true, formatChanged: true, sidePropsChanged: true, torchChanged: true, zoomChanged: true, exposureChanged: true, audioSessionChanged: true, locationChanged: true)
11:14:00.438: [info] 📸 VisionCamera.configureDevice(configuration:): Configuring Input Device...
11:14:00.438: [info] 📸 VisionCamera.configureDevice(configuration:): Configuring Camera com.apple.avfoundation.avcapturedevice.built-in_video:0...
11:14:00.450: [info] 📸 VisionCamera.configureDevice(configuration:): Successfully configured Input Device!
11:14:00.450: [info] 📸 VisionCamera.configureOutputs(configuration:): Configuring Outputs...
11:14:00.450: [info] 📸 VisionCamera.configureOutputs(configuration:): Adding Video Data output...
11:14:00.451: [info] 📸 VisionCamera.configureOutputs(configuration:): Successfully configured all outputs!
11:14:00.452: [info] 📸 VisionCamera.getPixelFormat(for:): Available Pixel Formats: ["420v", "420f", "BGRA"], finding best match... (pixelFormat="yuv", enableHdr={false}, enableBufferCompression={false})
11:14:00.452: [info] 📸 VisionCamera.getPixelFormat(for:): Using PixelFormat: 420f...
11:14:00.678: [info] 📸 VisionCamera.onCameraStarted(): Camera started!
11:14:00.678: [info] 📸 VisionCamera.onSessionInitialized(): Camera initialized!
11:14:00.679: [info] 📸 VisionCamera.configure(_:): Beginning AudioSession configuration...
11:14:00.679: [info] 📸 VisionCamera.configureAudioSession(configuration:): Configuring Audio Session...
11:14:00.681: [info] 📸 VisionCamera.configure(_:): Committed AudioSession configuration!
11:14:00.684: [info] 📸 VisionCamera.configure(_:): Beginning Location Output configuration...
11:14:00.689: [info] 📸 VisionCamera.configure(_:): Finished Location Output configuration!
test
test2
test
test
test
test
test
test

Camera Device

{
  "minZoom": 1,
  "minFocusDistance": 12,
  "id": "com.apple.avfoundation.avcapturedevice.built-in_video:0",
  "hardwareLevel": "full",
  "isMultiCam": false,
  "name": "Back Camera",
  "minExposure": -8,
  "hasFlash": true,
  "maxZoom": 16,
  "supportsFocus": true,
  "physicalDevices": [
    "wide-angle-camera"
  ],
  "neutralZoom": 1,
  "supportsRawCapture": false,
  "formats": [],
  "hasTorch": true,
  "position": "back",
  "maxExposure": 8,
  "supportsLowLightBoost": false,
  "sensorOrientation": "landscape-right"
}

Device

iPhone 11 (iOS 17.4.1)

VisionCamera Version

4.0.1

Can you reproduce this issue in the VisionCamera Example app?

I didn't try (⚠️ your issue might get ignored & closed if you don't try this)

Additional information

• [ ] I am using Expo
• [X] I have enabled Frame Processors (react-native-worklets-core)
• [X] I have read the Troubleshooting Guide
• [X] I agree to follow this project's Code of Conduct
• [X] I searched for similar issues in this repository and found none.

[mrousavy]

Thanks for the bug report! I fixed this bug in VisionCamera 4.0.2 🥳

[lukaszkurantdev]

@mrousavy Unfortunately, this bug still exists in 4.0.2 :(

[mrousavy]

lemme take another look.

[mrousavy]

This might fix your issue; https://github.com/mrousavy/react-native-vision-camera/pull/2849 bit more complex than I thought it would be

[mrousavy]

Yup I just tried runAsync on my iPhone in a Skia Frame Processor and it works flawlessly!