Open ImCitizen13 opened 2 months ago
It's hard to diagnose this without any code. Are you able to share the code, or a minimal repo that shows the error?
Make sure that the hook is being called from within the context provider.
function MyComponent() {
// will not work because it's outside the context
const detector = useObjectDetector()
return (<ObjectDetectionModelContextProvider> {...} </ObjectDetectionModelContextProvider>
}
function MyComponent() {
return (<ObjectDetectionModelContextProvider> <Child/> </ObjectDetectionModelContextProvider>
}
function Child () {
// will work because it is inside the context
const detector = useObjectDetector()
return <View/>
Also make sure you are using a development build, as the library has native dependencies that need to be bundled and won't work in Expo GO.
Hi there @trevor-coleman , Thank you for your response.
I got a bit busy last week, Today I got to try the solution you proposed and it eliminated the issue; however, I remembered that when I added the <ObjectDetectionModelContextProvider>
as a parent it kept infinitely re-rendering the children.
Below is a code example that showcases the issue, I followed the example in thehttps://docs.infinite.red/react-native-mlkit/object-detection/using-a-custom-model/#3-fetch-the-model-using-the-useobjectdetectionmodel-hook-and-use-it-to-detect-objects-in-an-image and I had to rectify some mismatching types.
result[]
import React, { useEffect, useState } from "react";
import {
Canvas,
useImage,
Skia,
Image,
SkImage,
} from "@shopify/react-native-skia";
import { Stack, useLocalSearchParams } from "expo-router";
// ML-Kit
import {
useObjectDetector,
RNMLKitObjectDetectionObject,
RNMLKitObjectDetectionContext,
AssetRecord,
useObjectDetectionModels,
} from "@infinitered/react-native-mlkit-object-detection";
import { RNMLKitDetectedObject } from "@infinitered/react-native-mlkit-object-detection/build/RNMLKitObjectDetectionModule";
export default function ParentWithContextProvider() {
// Get Path Params
const { imagePath, width, height } = useLocalSearchParams<{
imagePath: string;
width: string;
height: string;
}>();
console.log(imagePath);
const { ObjectDetectionModelContextProvider } = useObjectDetectionModels({
loadDefaultModel: true,
defaultModelOptions: {
shouldEnableMultipleObjects: true,
shouldEnableClassification: true,
detectorMode: "singleImage",
},
});
return (
<ObjectDetectionModelContextProvider>
<ChildView />
</ObjectDetectionModelContextProvider>
);
}
export function ChildView() {
const model = useObjectDetector();
const [modelIsLoaded, setModelLoaded] = useState(model?.isLoaded() ?? false);
// the output of the model is an array of `RNMLKitDetectedObject` objects
const [result, setResult] = useState<RNMLKitObjectDetectionObject[]>([]);
useEffect(() => {
// Loading models is done asynchronously, so in a useEffect we need to wrap it in an async function
async function loadModel() {
if (!model || modelIsLoaded) return;
// load the model
await model.load();
// set the model loaded state to true
setModelLoaded(true);
}
loadModel();
}, [model, modelIsLoaded]);
useEffect(() => {
if (!modelIsLoaded || !model) return;
// model.detectObjects is async, so when we use it in a useEffect, we need to wrap it in an async function
async function detectObjects(image: string) {
if (!model) return;
const result = await model.detectObjects(image);
setResult(result);
}
detectObjects(
"file:///data/user/0/com.meltohamy.health4u/cache/mrousavy-8371983902135514488.jpg"
);
}, [model, modelIsLoaded]);
return (
<View style={styles.container}>
<Text style={{ color: "white" }}>{Date().toString()}</Text>
<Text style={{ color: "white" }}>Result:{JSON.stringify(result)}</Text>
</View>
);
}
const styles = ...
I will try the custom model and will let you know what happens, as it could be the issue here. Thank you again for your response.
Thanks for sharing this -- I'm currently investigating an issue with Object detection on Expo v51, so this may be related.
I have the same issue with constant rerendering ... @ImCitizen13 did you solve this?
I'm facing a similar issue with Expo v51 -- @trevor-coleman were you able to resolve this?
We have a patch coming shortly -- we were away at Chain React all last week, so it's been a but held up.
We just release a new patch which should fix errors with Expo 51. Check the PR for the latest version numbers
If that doesn't fix the infinite looping error let me know, and I'll dig into this further.
The useEffect
is not strictly necessary. Maybe try running the detection in the component body:
let result
if(model && modelIsLoaded) {
result = await model.detectObjects()
}
I've got to get on some client work, but I will try and repro when I get a few mins.
@trevor-coleman I got a custom model working 🥳, but the default model still rerenders infinitely. For the custom model, I had to double-check the types, as the demo code is a bit off. This v. works. It uses ModelInfo instead of AssetRecord.:
const MODELS: Record<string, ModelInfo> = {
myCustomModel: {
model: require("../assets/models/efficientnet_lite0-unit8.tflite"),
options: {
shouldEnableMultipleObjects: false,
shouldEnableClassification: true,
detectorMode: "singleImage",
maxPerObjectLabelCount: 1,
},
},
};
And, instead of destructuring with const { model } = useObjectDetector("myCustomModel"), use direct assignment: const model = useObjectDetector("myCustomModel");.
Ah thanks for the heads up -- I'll take a look and update the docs.
Hi there, I've been trying to get the __MLKIT Object Detection_ to work in my Expo project, however I'm facing an issue.
Issue:
I get the following error:
Error: useRNMLKitObjectDetectionContext must be used within a <RNMLKitObjectDetectionContext.Provider>
Steps:
1- I followed the installation instructions on https://docs.infinite.red/react-native-mlkit/object-detection/ 2- Added the
ObjectDetectionModelContextProvider
to my app 3- Attempted to load the model usinguseObjectDetector()
, however, I get the the error mentioned above, which means that the provider is not working correctly.Setup:
Let me know If I can provide further info