Big sizes of camera resolution can cause slow capture.

[nh9k]

Area Select

react-native-pytorch-core (core package)

Description

Hello, always thanks for contributions.

My test app needs an image with good resolution, but if the camera resolution size is larger than that of the playtorch tutorial, it will be slower than before to change the camera screen to the loading screen after pressing the capture button. I also tested it in the tutorial on android. i modified targetResolution={{ width: 1080, height: 1920 }} to targetResolution={{ width: 3000, height: 4000 }}. The image result is good in resolution, but the loading screen changes slowly on the camera screen. How can i improve the changing speed?

I saw the code in CamerView.tsx, but it doesn't seem to help me.

[raedle]

@nh9k, I can look at it, but please post a link to a Snack that reproduces the issue

[nh9k]

@raedle, thank you. my reproduced Snack link is here. This Snack only modified the camera resolution of the camera screen. The playtorch Snack link is here.

[raedle]

@nh9k, that's expected because it needs to copy more data from the camera pixel buffer into a tensor. Is there a model that uses a tensor shape this big?

[nh9k]

Alright @raedle , it's expected more time for copying more data, but it is different with package react-native-camera. This pacakge can capture image quickly despite the large image size. Is the camera pixel buffered with tensor during capture in the react-native-pytorch package? I can get an image that is not a tensor by capturing. I need a good image resolution for the text recognition model.

[raedle]

The demo in the object detection tutorial runs inference on the input image (not just taking a photo, the larger the image the longer the pre-processing or inference). Is that done as well in the comparison to react-native-camera?

I created a simple camera-only example with the target resolution set to targetResolution={{ width: 3000, height: 4000 }}. The response is relatively fast, and it is comparable to takePictureAsync from the react-native-camera except that react-native-camera will return the image base64 encoded and not the actual image object.

PlayTorch Snack: https://snack.playtorch.dev/@raedle/image-capture-example

import * as React from 'react';
import { Alert, SafeAreaView, StyleSheet } from 'react-native';
import { Camera } from 'react-native-pytorch-core';
import type { Image } from 'react-native-pytorch-core';

export default function App() {
  function handleCapture(image: Image) {
    Alert.alert(
      `Captured image with size w=${image.getWidth()}, h=${image.getHeight()}`
    );
  }

  return (
    <SafeAreaView style={StyleSheet.absoluteFill}>
      <Camera
        style={StyleSheet.absoluteFill}
        onCapture={handleCapture}
        targetResolution={{ width: 3000, height: 4000 }}
      />
    </SafeAreaView>
  );
}

https://user-images.githubusercontent.com/489051/215738324-48fb6751-6e46-4f93-a892-cd4e51cb0ec4.mov

[nh9k]

Thank you, @raedle! Just taking a photo is actually fast! Thank you for the experiment. But we have to apply with our models, and if we do that, the loading screen changes slowly on the camera screen. :cry:

[nh9k]

@raedle, How can i improve the changing speed? I have no idea..