am15h / tflite_flutter_helper

TensorFlow Lite Flutter Helper Library
https://pub.dev/packages/tflite_flutter_helper
Apache License 2.0
159 stars 285 forks source link

ImageProcessor doesn't run through tasks during process since 0.3.1 #50

Closed espbee closed 2 years ago

espbee commented 2 years ago

I'm running a custom Yolov4 model (also a Yolo3 for what it's worth) through here. Everything was lovely while I ran tflite_flutter 0.8.0 with 0.2.0 the helper. Since upgrading to 0.9.0 and 0.3.1, I don't get a suitable image input to run through the predictor, and it winds up breaking.

More specifically, I pre-process the camera frame after it's been wrapped in a TensorImage, and send it through the ImageProcessor--

and afterwards, in runForMultipleInputs, I getInputTensors and then try to run inputTensors.elementAt(i).setTo()inputs[i]). But there's an error from within setTo()-- one of the checks fails and throws a Bad state: failed precondition.

Back during image prep, I run the ImageProcessor from this helper lib like this:

imageProcessor ??= ImageProcessorBuilder() .add(ResizeWithCropOrPadOp(padSize!, padSize!)) .add(ResizeOp(INPUT_SIZE.toInt(), INPUT_SIZE.toInt(), ResizeMethod.NEAREST_NEIGHBOUR)) .add(NormalizeOp(1, 255)) .build();

inputImage = imageProcessor!.process(inputImage);

I think I've isolated the problem.

After running the TensorImage wrapped im.Image through the processor in the functional app using the older libs, I can no longer see the image as an im.image-- it's a float-value image instead and throws an error if I try to access image properties. However, in the non-functionla app with the newer libs, I can still see the image afterward, thogh it's byte length is slightly more than half of what it had been.

Any insight would be appreciated... Is the idea just to run those ops manually, do you think? Or has anyone run into anything like this?

espbee commented 2 years ago

Solved. If anyone happens upon this in a similar boat--

In the older lib, to make the inputImage mentioned above, I could do TensorImage inputImage = TensorImage.fromImage(image), image being an image lib image.

Now I had to specify the type, like: TensorImage inputImage = TensorImage(TfliteType.float32); inputImage.loadImage(image);

Thanks to @sDobrzanski in https://github.com/am15h/tflite_flutter_plugin/issues/167