expo assets path

[smreo3839]

Hi Can this code also work in an Expo environment? I put the tflite model in the assets/model folder, but it seems like the return value of const detectedObjects = detectObjects(frame, frameProcessorConfig); is undefined, so it looks like it cannot load the model.

I added "assets": ["./assets/model/"] in app.json since I am using an expo environment. I have confirmed that react-native-vision-camera works correctly by prebuilding with expo.

thanks

[smreo3839]

import { Camera, useCameraDevices, useFrameProcessor } from 'react-native-vision-camera'; import { useSharedValue, runOnJS } from 'react-native-reanimated'; import { DetectedObject, detectObjects, FrameProcessorConfig } from 'vision-camera-realtime-object-detection'; import { StyleSheet, Text, View } from 'react-native'; import React, { useEffect, useState } from 'react'; import { useWindowDimensions } from 'react-native'; export default function App() { const [hasPermission, setHasPermission] = useState(false); const devices = useCameraDevices('wide-angle-camera'); const device = devices.back;

useEffect(() => { (async () => { const status = await Camera.requestCameraPermission(); setHasPermission(status === 'authorized'); })(); }, []);

const windowDimensions = useWindowDimensions(); const [objects, setObjects] = useState<DetectedObject[]>([]);

const frameProcessorConfig: FrameProcessorConfig = { modelFile: 'best-fp16.tflite', scoreThreshold: 0.5, };

const frameProcessor = useFrameProcessor((frame) => { 'worklet'; const detectedObjects = detectObjects(frame, frameProcessorConfig); runOnJS(setObjects)( detectedObjects.map((obj) => ({ ...obj, top: obj.top windowDimensions.height, left: obj.left windowDimensions.width, width: obj.width windowDimensions.width, height: obj.height windowDimensions.height, })) ); }, []);

[jaroslawkrol]

@smreo3839 Thanks for reporting this. I need to confess that I have never tested it with Expo. I will check it and try to fix it ASAP (2 days or so)

[smreo3839]

@smreo3839 Thanks for reporting this. I need to confess that I have never tested it with Expo. I will check it and try to fix it ASAP (2 days or so)

Thank you. Your library has a really useful feature that we need for our university project, so we are really happy to have it.

[devatina11yb]

We are facing the same issue.

According to Expo documentation, you would need to customize Metro configuration for it to include .tflite files in the bundle. Then you would be able to require or import those assets (see Local assets).

@jaroslawkrol Have you made any progress on this issue?

[jaroslawkrol]

@smreo3839 @devatina11yb I apologize to you, but my regular work in recent months has completely consumed me. Yes, I tried to add Expo support, but without much results. I tried to implement this using the expo-asset library (react-native-asset doesn't support Expo) and the first thing I came to is that files with the tflite extension are not added to the application assets by default. To get around this problem change the metro.config.js file as below:

const { getDefaultConfig } = require('expo/metro-config');

// module.exports = getDefaultConfig(__dirname); <!-- comment this line

module.exports = (async () => {
    const {
        resolver: {sourceExts, assetExts},
    } = await getDefaultConfig(__dirname);
    return {
        resolver: {
            assetExts: [assetExts, 'txt', 'xml', 'png', 'jpg', 'pb', 'tflite'],
            sourceExts: [...sourceExts, 'txt', 'xml', 'png', 'jpg', 'pb', 'tflite'],
        },
    };
})();

But using imported file in Native Module is my nightmare now, but I'm not going surrender. If you can help, happily will merge any PR of it. Unfortunately, until it is resolved, I must consider Expo unsupported

[catherinejchen]

I was hoping to use this with my expo project as well :(( bummer that it doesn't integrate with it

[lucksp]

Try the Expo docs on importing assets: https://docs.expo.dev/guides/customizing-metro/#assets

You may need a custom EAS build also.