🐛 libc++abi: terminating with uncaught exception of type facebook::jsi::JSError: Exception in HostFunction: Javascript worklet error

[QSuraj]

What were you trying to do?

Using custom frame processor.

Created this new issue since the old one was closed.

Package details -

"react-native": "0.69.0",
"react-native-reanimated": "2.10.0",
"react-native-vision-camera": "2.14.1",

Reproduceable Code

QFrameProcessor.js

// eslint-disable-next-line import/no-duplicates
import 'react-native-reanimated';
import { Camera, useCameraDevices, useFrameProcessor } from 'react-native-vision-camera';
// eslint-disable-next-line import/no-duplicates
import React from 'react';

const processFrame = ( frame ) => {
  'worklet';

  // eslint-disable-next-line no-undef
  return __processFrame( frame );
};

const QFrameProcessor = ( props ) => {
  const devices = useCameraDevices( 'wide-angle-camera' );
  const device = devices.back;

  const frameProcessor = useFrameProcessor( ( frame ) => {
    'worklet';

    const data = processFrame( frame );
    console.log( 'data', data );
  }, [] );

  if ( device === undefined || !frameProcessor ) {
    return null;
  }

  return (
    <Camera
      isActive
      style={props.style}
      device={device}
      frameProcessor={frameProcessor}
      frameProcessorFps={1} />
  );
};

export default React.memo( QFrameProcessor );

QFrameProcessorPlugin.m

#import <VisionCamera/FrameProcessorPlugin.h>
#import <VisionCamera/Frame.h>

@interface QFrameProcessorPlugin : NSObject
@end

@implementation QFrameProcessorPlugin

static inline id processFrame(Frame* frame, NSArray* args) {
  CMSampleBufferRef buffer = frame.buffer;
  UIImageOrientation orientation = frame.orientation;
  // code goes here
  return @[];
}

VISION_EXPORT_FRAME_PROCESSOR(processFrame)

@end

Ideally the frame processor should have started sending me an empty array to the console.log

What happened instead?

The following error was thrown in Xcode console and the app crashed.

Relevant log output

2022-10-27 09:36:48.545198+0530 quizizz_game_v3_app[11942:1108041] [native] VisionCamera.didSetProps(_:): Updating 10 prop(s)...
2022-10-27 09:36:48.545253+0530 quizizz_game_v3_app[11942:1108363] [native] VisionCamera.configureCaptureSession(): Configuring Session...
2022-10-27 09:36:48.545262+0530 quizizz_game_v3_app[11942:1108363] [native] VisionCamera.configureCaptureSession(): Initializing Camera with device com.apple.avfoundation.avcapturedevice.built-in_video:6...
2022-10-27 09:36:48.545273+0530 quizizz_game_v3_app[11942:1108363] [native] VisionCamera.configureCaptureSession(): Adding Video input...
2022-10-27 09:36:48.545556+0530 quizizz_game_v3_app[11942:1108312] FrameProcessorBindings: Setting new frame processor...
2022-10-27 09:36:48.545578+0530 quizizz_game_v3_app[11942:1108312] FrameProcessorBindings: Adapting Shareable value from function (conversion to worklet)...
2022-10-27 09:36:48.545708+0530 quizizz_game_v3_app[11942:1108312] FrameProcessorBindings: Successfully created worklet!
2022-10-27 09:36:48.545754+0530 quizizz_game_v3_app[11942:1108390] FrameProcessorBindings: Converting worklet to Objective-C callback...
2022-10-27 09:36:48.546358+0530 quizizz_game_v3_app[11942:1108390] FrameProcessorBindings: Frame processor set!
2022-10-27 09:36:48.589822+0530 quizizz_game_v3_app[11942:1108363] [native] VisionCamera.configureCaptureSession(): Adding Video Data output...
2022-10-27 09:36:48.590355+0530 quizizz_game_v3_app[11942:1108363] [native] VisionCamera.invokeOnInitialized(): Camera initialized!
2022-10-27 09:36:48.590399+0530 quizizz_game_v3_app[11942:1108363] [native] VisionCamera.configureCaptureSession(): Session successfully configured!
2022-10-27 09:36:48.590872+0530 quizizz_game_v3_app[11942:1108363] [native] VisionCamera.configureFormat(): Configuring Format...
2022-10-27 09:36:48.590877+0530 quizizz_game_v3_app[11942:1108363] [native] VisionCamera.configureDevice(): Configuring Device...
2022-10-27 09:36:48.590905+0530 quizizz_game_v3_app[11942:1108363] [native] VisionCamera.configureDevice(): Device successfully configured!
2022-10-27 09:36:48.591010+0530 quizizz_game_v3_app[11942:1108363] [native] VisionCamera.didSetProps(_:): Starting Session...
2022-10-27 09:36:48.997963+0530 quizizz_game_v3_app[11942:1108363] [native] VisionCamera.didSetProps(_:): Started Session!
libc++abi: terminating with uncaught exception of type facebook::jsi::JSError: Exception in HostFunction: Javascript worklet error

Error: Javascript worklet error
    at _f (native)

Error: Exception in HostFunction: Javascript worklet error

Error: Javascript worklet error
    at _f (native)
    at _f (native)
terminating with uncaught exception of type facebook::jsi::JSError: Exception in HostFunction: Javascript worklet error

Error: Javascript worklet error
    at _f (native)

Error: Exception in HostFunction: Javascript worklet error

Error: Javascript worklet error
    at _f (native)
    at _f (native)

Device

iPhone 14 ( iOS 16.1 )

VisionCamera Version

2.14.1

Additional information

• [ ] I am using Expo
• [X] I have read the Troubleshooting Guide
• [X] I agree to follow this project's Code of Conduct
• [X] I searched for similar issues in this repository and found none.

The patch to fix Xcode 14 was already applied, this occurs even after adding that patch. Reference to the patch - https://github.com/mrousavy/react-native-vision-camera/issues/1244#issuecomment-1261364351

[QSuraj]

Finally figured out the issue.

I was using Vision Camera Plugin Builder CLI as suggested in the docs to build the custom frame processor which created the frame processor in the separate directory called QFrameProcessor and with in that QFrameProcessor.m file was added.

This should not happen.

QFrameProcessor.m file should be directly added to the root directory of the project.

All thanks vision-camera-plugin-builder I was stuck with this.

@mrousavy

[mrousavy]

Hey! That's a bug in the CLI then, would be awesome if you could send a PR to them and fix that :) cc @mateusz1913

[mateusz1913]

@QSuraj could you submit a bug issue in vision-camera-plugin-builder repo? Please include a minimal reproduction repo with 2 vision camera plugins - one generated with CLI & one created manually in XCode fixed

[mateusz1913]

@QSuraj This is fixed with https://github.com/mateusz1913/vision-camera-plugin-builder/pull/24 and available with vision-camera-plugin-builder v0.2.0 🚀 - if you want to check, make sure that npx didn't cache older version of CLI on your machine

[QSuraj]

Receiving following crash when I run npx vision-camera-plugin-builder ios

npm ERR! cb.apply is not a function

npm ERR! A complete log of this run can be found in:
npm ERR!     <directory>/.npm/_logs/2022-10-28T03_11_05_574Z-debug.log
Install for [ 'vision-camera-plugin-builder@latest' ] failed with code 1

Here is log.

0 info it worked if it ends with ok
1 verbose cli [
1 verbose cli   '<directory>/.nvm/versions/node/v14.17.5/bin/node',
1 verbose cli   '<directory>/.nvm/versions/node/v14.17.5/lib/node_modules/npx/node_modules/npm/bin/npm-cli.js',
1 verbose cli   'install',
1 verbose cli   'vision-camera-plugin-builder@latest',
1 verbose cli   '--global',
1 verbose cli   '--prefix',
1 verbose cli   '<directory>/.npm/_npx/73253',
1 verbose cli   '--loglevel',
1 verbose cli   'error',
1 verbose cli   '--json'
1 verbose cli ]
2 info using npm@5.1.0
3 info using node@v14.17.5
4 verbose npm-session e0aa96245e8dd287
5 silly install loadCurrentTree
6 silly install readGlobalPackageData
7 http fetch GET 200 https://registry.npmjs.org/vision-camera-plugin-builder 29ms (from cache)
8 silly pacote tag manifest for vision-camera-plugin-builder@latest fetched in 46ms
9 verbose stack TypeError: cb.apply is not a function
9 verbose stack     at <directory>.nvm/versions/node/v14.17.5/lib/node_modules/npx/node_modules/npm/node_modules/graceful-fs/polyfills.js:287:18
9 verbose stack     at FSReqCallback.oncomplete (fs.js:193:5)
10 verbose cwd <path-to-project>
11 verbose Darwin 21.6.0
12 verbose argv "<directory>/.nvm/versions/node/v14.17.5/bin/node" "<directory>/.nvm/versions/node/v14.17.5/lib/node_modules/npx/node_modules/npm/bin/npm-cli.js" "install" "vision-camera-plugin-builder@latest" "--global" "--prefix" "<directory>/.npm/_npx/73253" "--loglevel" "error" "--json"
13 verbose node v14.17.5
14 verbose npm  v5.1.0
15 error cb.apply is not a function
16 verbose exit [ 1, true ]

[nzmsyhrl]

For me, it doesn't give any error but it didn't detect the QR code

[mateusz1913]

@QSuraj let's move discussion to the vision-camera-plugin-builder repo - issue 23

@mrousavy could you close and lock the issue?

[lexer]

@mateusz1913 @QSuraj I have the same issue.

I'm using:

   "react-native-vision-camera": "2.15.1",
   "vision-camera-code-scanner": "0.2.0"

Node version: v18.8.0

[lexer]

My issue got fixed with this: https://github.com/mrousavy/react-native-vision-camera/pull/1308

[QSuraj]

Ya, that did fix the issue for me but after Xcode 15 we can't use +(void)load for swift files. Which was the fix in https://github.com/mrousavy/react-native-vision-camera/pull/1308

Now it throws the error reported here because of the above change - https://github.com/mrousavy/react-native-vision-camera/issues/1802

@lexer

[mrousavy]

So Xcode 13 is static initialize, Xcode 14 is +load, and Xcode 15 is static initialize again? 🤯

[mrousavy]

@QSuraj does it work if you revert the change in #1308 for you on Xcode 15?

[QSuraj]

Yes, it does help with the build being successful but vision-camera-code-scanner crashes the moment the frameProcessor starts which is using VISION_EXPORT_SWIFT_FRAME_PROCESSOR.

libc++abi: terminating due to uncaught exception of type facebook::jsi::JSError: Exception in HostFunction: <unknown>

Error: Exception in HostFunction: <unknown>
    at _f (native)

@mrousavy

[vincicat]

@QSuraj does it work if you revert the change in #1308 for you on Xcode 15?

reverted, the initial app crash avoided, but later the app caught an error when the camera calling the frame processor (with https://github.com/rodgomesc/vision-camera-code-scanner , @0.2.0):

Property '__scanCodes' doesn't exist

ReferenceError: Property '__scanCodes' doesn't exist
at scanBarcodes ([PROJECT PATH]/node_modules/vision-camera-code-scanner/src/index.ts (300:7):1:52)
at scanBarcodes (native)
at _f ([PROJECT PATH]/src/components/QRCodeScanner.js [REDACTED])
at _f (native)
at _f ([PROJECT PATH]/node_modules/react-native-vision-camera/src/hooks/useFrameProcessor.ts (28:21):1:425)
at _f (native)

note: camera still running but the error appears whenever it calls the frame processor

[yedi97]

@QSuraj does it work if you revert the change in #1308 for you on Xcode 15?

reverted, the initial app crash avoided, but later the app caught an error when the camera calling the frame processor (with https://github.com/rodgomesc/vision-camera-code-scanner , @0.2.0):

Property '__scanCodes' doesn't exist

ReferenceError: Property '__scanCodes' doesn't exist
at scanBarcodes ([PROJECT PATH]/node_modules/vision-camera-code-scanner/src/index.ts (300:7):1:52)
at scanBarcodes (native)
at _f ([PROJECT PATH]/src/components/QRCodeScanner.js [REDACTED])
at _f (native)
at _f ([PROJECT PATH]/node_modules/react-native-vision-camera/src/hooks/useFrameProcessor.ts (28:21):1:425)
at _f (native)

note: camera still running but the error appears whenever it calls the frame processor

Same issue

[QSuraj]

Finally figured out the issue. After reverting back from https://github.com/mrousavy/react-native-vision-camera/pull/1308 we need to add @interface objc_name (FrameProcessorPlugin) <FrameProcessorPluginBase> previously it was just @interface objc_name (FrameProcessorPlugin).

#define VISION_EXPORT_SWIFT_FRAME_PROCESSOR(name, objc_name)                        \
objc_name : NSObject<FrameProcessorPluginBase>                                      \
@end                                                                                \
                                                                                    \
@interface objc_name (FrameProcessorPlugin) <FrameProcessorPluginBase>              \
@end                                                                                \
@implementation objc_name (FrameProcessorPlugin)                                    \
                                                                                    \
__attribute__((constructor)) static void VISION_CONCAT(initialize_, objc_name)()    \
{                                                                                   \
  [FrameProcessorPluginRegistry addFrameProcessorPlugin:@"__" @ #name callback:^id(Frame* frame, NSArray<id>* args) {    \
    return [objc_name callback:frame withArgs:args];                               \
  }];                                                                               \
}

@mrousavy we need to make the above change for all the frame processors based on swift to work.

[joren-vos-aca]

Finally figured out the issue. After reverting back from #1308 we need to add @interface objc_name (FrameProcessorPlugin) <FrameProcessorPluginBase> previously it was just @interface objc_name (FrameProcessorPlugin).

@mrousavy we need to make the above change for all the frame processors based on swift to work.

I can confirm that this fixes the problem!

[degeloper34]

Finally figured out the issue. After reverting back from #1308 we need to add @interface objc_name (FrameProcessorPlugin) <FrameProcessorPluginBase> previously it was just @interface objc_name (FrameProcessorPlugin).

#define VISION_EXPORT_SWIFT_FRAME_PROCESSOR(name, objc_name)                        \
objc_name : NSObject<FrameProcessorPluginBase>                                      \
@end                                                                                \
                                                                                    \
@interface objc_name (FrameProcessorPlugin) <FrameProcessorPluginBase>              \
@end                                                                                \
@implementation objc_name (FrameProcessorPlugin)                                    \
                                                                                    \
__attribute__((constructor)) static void VISION_CONCAT(initialize_, objc_name)()    \
{                                                                                   \
  [FrameProcessorPluginRegistry addFrameProcessorPlugin:@"__" @ #name callback:^id(Frame* frame, NSArray<id>* args) {    \
    return [objc_name callback:frame withArgs:args];                               \
  }];                                                                               \
}

@mrousavy we need to make the above change for all the frame processors based on swift to work.

This fix solves the crash during app launch, thanks!

[medonja]

I hope this fix will soon be released :)

[vogdev]

same

[shilpa12233]

My app runs fine on simulator, RN 0.68.2 but crashes in react core pod RCTCxxBridge file - at _reactInstance.reset(new Instance); when I try running on device. iOS - 17, iphone 15, xcode 15

[carlgrob5171989]

@vincicat Did you resolve it?

[DarkMatter513]

when is this being released in the react-native-vision camera? this is not added to the 2.16.2 please add a commit with this patch.