Error: useRNMLKitObjectDetectionContext must be used within a <RNMLKitObjectDetectionContext.Provider>

[ImCitizen13]

Hi there, I've been trying to get the __MLKIT Object Detection_ to work in my Expo project, however I'm facing an issue.

Issue:

I get the following error: Error: useRNMLKitObjectDetectionContext must be used within a <RNMLKitObjectDetectionContext.Provider>

Steps:

1- I followed the installation instructions on https://docs.infinite.red/react-native-mlkit/object-detection/ 2- Added the ObjectDetectionModelContextProvider to my app 3- Attempted to load the model using useObjectDetector(), however, I get the the error mentioned above, which means that the provider is not working correctly.

Setup:

• [x] Expo 51
• [x] Bun
• [x] react-native-mlkit-object-detection 2.0.0
• [ ] Typescript

Let me know If I can provide further info

[trevor-coleman]

It's hard to diagnose this without any code. Are you able to share the code, or a minimal repo that shows the error?

Make sure that the hook is being called from within the context provider.

function MyComponent() {
  // will not work because it's outside the context
 const detector = useObjectDetector()
  return (<ObjectDetectionModelContextProvider> {...} </ObjectDetectionModelContextProvider>
}  

function MyComponent()  {
  return (<ObjectDetectionModelContextProvider> <Child/> </ObjectDetectionModelContextProvider>
}  

function Child () {
   // will work because it is inside the context
   const detector = useObjectDetector()
   return <View/>

Also make sure you are using a development build, as the library has native dependencies that need to be bundled and won't work in Expo GO.

[ImCitizen13]

Hi there @trevor-coleman , Thank you for your response. I got a bit busy last week, Today I got to try the solution you proposed and it eliminated the issue; however, I remembered that when I added the <ObjectDetectionModelContextProvider> as a parent it kept infinitely re-rendering the children.

Experiment

Below is a code example that showcases the issue, I followed the example in thehttps://docs.infinite.red/react-native-mlkit/object-detection/using-a-custom-model/#3-fetch-the-model-using-the-useobjectdetectionmodel-hook-and-use-it-to-detect-objects-in-an-image and I had to rectify some mismatching types.

Issues

1. The app still re-renders
2. I don't get any result result[]

Code

import React, { useEffect, useState } from "react";
import {
  Canvas,
  useImage,
  Skia,
  Image,
  SkImage,
} from "@shopify/react-native-skia";
import { Stack, useLocalSearchParams } from "expo-router";
// ML-Kit
import {
  useObjectDetector,
  RNMLKitObjectDetectionObject,
  RNMLKitObjectDetectionContext,
  AssetRecord,
  useObjectDetectionModels,
} from "@infinitered/react-native-mlkit-object-detection";
import { RNMLKitDetectedObject } from "@infinitered/react-native-mlkit-object-detection/build/RNMLKitObjectDetectionModule";

export default function ParentWithContextProvider() {
  // Get Path Params
  const { imagePath, width, height } = useLocalSearchParams<{
    imagePath: string;
    width: string;
    height: string;
  }>();

  console.log(imagePath);
  const { ObjectDetectionModelContextProvider } = useObjectDetectionModels({
    loadDefaultModel: true,
    defaultModelOptions: {
      shouldEnableMultipleObjects: true,
      shouldEnableClassification: true,
      detectorMode: "singleImage",
    },
  });

  return (
    <ObjectDetectionModelContextProvider>
      <ChildView />
    </ObjectDetectionModelContextProvider>
  );
}

export function ChildView() {
  const model = useObjectDetector();
  const [modelIsLoaded, setModelLoaded] = useState(model?.isLoaded() ?? false);
  // the output of the model is an array of `RNMLKitDetectedObject` objects
  const [result, setResult] = useState<RNMLKitObjectDetectionObject[]>([]);

  useEffect(() => {
    // Loading models is done asynchronously, so in a useEffect we need to wrap it in an async function
    async function loadModel() {
      if (!model || modelIsLoaded) return;
      // load the model
      await model.load();
      // set the model loaded state to true
      setModelLoaded(true);
    }

    loadModel();
  }, [model, modelIsLoaded]);

  useEffect(() => {
    if (!modelIsLoaded || !model) return;

    // model.detectObjects is async, so when we use it in a useEffect, we need to wrap it in an async function
    async function detectObjects(image: string) {
      if (!model) return;
      const result = await model.detectObjects(image);
      setResult(result);
    }
    detectObjects(
      "file:///data/user/0/com.meltohamy.health4u/cache/mrousavy-8371983902135514488.jpg"
    );
  }, [model, modelIsLoaded]);

  return (
    <View style={styles.container}>
      <Text style={{ color: "white" }}>{Date().toString()}</Text>
      <Text style={{ color: "white" }}>Result:{JSON.stringify(result)}</Text>
    </View>
  );
}

const styles = ...

I will try the custom model and will let you know what happens, as it could be the issue here. Thank you again for your response.

[trevor-coleman]

Thanks for sharing this -- I'm currently investigating an issue with Object detection on Expo v51, so this may be related.

[BoavistaLudwig]

I have the same issue with constant rerendering ... @ImCitizen13 did you solve this?

[DanielRHarris]

I'm facing a similar issue with Expo v51 -- @trevor-coleman were you able to resolve this?

[trevor-coleman]

We have a patch coming shortly -- we were away at Chain React all last week, so it's been a but held up.

[trevor-coleman]

We just release a new patch which should fix errors with Expo 51. Check the PR for the latest version numbers

If that doesn't fix the infinite looping error let me know, and I'll dig into this further.

The useEffect is not strictly necessary. Maybe try running the detection in the component body:

let result
if(model && modelIsLoaded) {
  result = await model.detectObjects()
}

I've got to get on some client work, but I will try and repro when I get a few mins.

[BoavistaLudwig]

@trevor-coleman I got a custom model working 🥳, but the default model still rerenders infinitely. For the custom model, I had to double-check the types, as the demo code is a bit off. This v. works. It uses ModelInfo instead of AssetRecord.:

const MODELS: Record<string, ModelInfo> = {
  myCustomModel: {
    model: require("../assets/models/efficientnet_lite0-unit8.tflite"),
    options: {
      shouldEnableMultipleObjects: false,
      shouldEnableClassification: true,
      detectorMode: "singleImage",
      maxPerObjectLabelCount: 1,
    },
  },
};

And, instead of destructuring with const { model } = useObjectDetector("myCustomModel"), use direct assignment: const model = useObjectDetector("myCustomModel");.

[trevor-coleman]

Ah thanks for the heads up -- I'll take a look and update the docs.