am15h / tflite_flutter_plugin

TensorFlow Lite Flutter Plugin
https://pub.dev/packages/tflite_flutter
Apache License 2.0
503 stars 353 forks source link

Invalid input data #133

Open quochuynh67 opened 3 years ago

quochuynh67 commented 3 years ago

Hi, I am using posenet_mobilenet_v1_100_257x257_multi_kpt_stripped.tflit. but I don't know the input data size. The console log here

E/tflite (27994): tensorflow/lite/kernels/conv.cc:316 input->dims->size != 4 (1 != 4) E/tflite (27994): Node number 0 (CONV_2D) failed to prepare. E/flutter (27994): [ERROR:flutter/runtime/dart_isolate.cc(1137)] Unhandled exception: E/flutter (27994): Bad state: failed precondition E/flutter (27994): #0 checkState (package:quiver/check.dart:74:5) E/flutter (27994): #1 Interpreter.allocateTensors (package:tflite_flutter/src/interpreter.dart:142:5) E/flutter (27994): #2 Interpreter.runForMultipleInputs (package:tflite_flutter/src/interpreter.dart:180:7) E/flutter (27994): #3 Interpreter.run (package:tflite_flutter/src/interpreter.dart:157:5) E/flutter (27994): #4 Classifier.predict (package:flutter_pose_dance/classifier.dart:108:19) E/flutter (27994): #5 IsolateUtils.entryPoint (package:flutter_pose_dance/isolate_utils.dart:46:52) E/flutter (27994):

Could you recommend for me the document or the way to fix it? Thank you so much!

quochuynh67 commented 3 years ago

@am15h

quochuynh67 commented 3 years ago

When I check the code

inputTensors = getInputTensors(); for (int i = 0; i < inputs.length; i++) { inputTensors.elementAt(i).setTo(inputs[i]); } }

void setTo(Object src) { Uint8List bytes = _convertObjectToBytes(src); int size = bytes.length; final ptr = calloc(size); checkState(isNotNull(ptr), message: 'unallocated'); final externalTypedData = ptr.asTypedList(size); externalTypedData.setRange(0, bytes.length, bytes); checkState(tfLiteTensorCopyFromBuffer(_tensor, ptr.cast(), bytes.length) == // Error happen because this line TfLiteStatus.ok); calloc.free(ptr); }

bazinac commented 3 years ago

I am having the same issue with my Mobilenet classifier. @quochuynh67, have you managed to solve this somehow please?

hamidrezabstn commented 2 years ago

hi I am having the same issue

tafaust commented 2 years ago

I have the same issue. I found that the tfLiteTensorCopyFromBuffer(_tensor, ptr.cast(), bytes.length) produces the error type 'Pointer<NativeType>' is not a subtype of type 'Pointer<Void>'. If I modify the code as follows

tfLiteTensorCopyFromBuffer(_tensor, ptr.cast<Void>(), bytes.length)

it does not produce the error anymore but merely the return code 1. @am15h I am not sure if this is some error with dart:ffi because this issue is my first encounter with dart:ffi. Maybe you know more on this issue?

Let me know if I shall provide my code + image.


EDIT: I want to mention that I applied the changes from https://github.com/am15h/tflite_flutter_plugin/issues/101#issuecomment-821781506 in my flutter project.


EDIT2: I have just seen that the signature of the tfLiteTensorCopyFromBuffer in the bindings/tensor.dart is as follows

/*TfLiteStatus*/
int Function(
  Pointer<TfLiteTensor> tensor,
  Pointer<Void> inputData,
  int inputDataSize,
) tfLiteTensorCopyFromBuffer = tflitelib
    .lookup<NativeFunction<_TfLiteTensorCopyFromBuffer_native_t>>(
        'TfLiteTensorCopyFromBuffer')
    .asFunction();

typedef _TfLiteTensorCopyFromBuffer_native_t = /*TfLiteStatus*/ Int32 Function(
  Pointer<TfLiteTensor> tensor,
  Pointer<Void> inputData,
  Int32 inputDataSize,
);

The typedef defines the inputData as Pointer<Void> which can not be dereferenced nor written to AFAIU but serves as some sort of base pointer. This base pointer can be used after a cast and thus probably the ptr.cast() in the originally conflicting line. According to https://api.dart.dev/stable/2.16.2/dart-ffi/NativeType-class.html we should expect

/*TfLiteStatus*/
int Function(
  Pointer<TfLiteTensor> tensor,
  Pointer<NativeType> inputData,
  int inputDataSize,
) tfLiteTensorCopyFromBuffer = tflitelib
    .lookup<NativeFunction<_TfLiteTensorCopyFromBuffer_native_t>>(
        'TfLiteTensorCopyFromBuffer')
    .asFunction();

typedef _TfLiteTensorCopyFromBuffer_native_t = /*TfLiteStatus*/ Int32 Function(
  Pointer<TfLiteTensor> tensor,
  Pointer<NativeType> inputData,
  Int32 inputDataSize,
);

here. This change does for one resolve the runtime error but still returns 1 (= TfLiteStatus.error).

@am15h If you want me to submit a PR to patch all Pointer<Void> to Pointer<NativeType>, give me a sign ⭐. 😂


EDIT3: Obviously, the check at https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/c/c_api.cc#L215 fails and using tfLiteTensorByteSize(_tensor) to check on the size (-> bytes in the c++ implementation) yields 24800 which does not equal the buffer size 74400 in my case. Any ideas how I can debug this further? (I think that 24800*3=74400 must not be a coincidence...)


EDIT4: Apparently, I need to convert my image to grayscale first... 😅 The issue in my case however is that the grayscale transformed image does not reduce in size... the rgb format returns a UInt8List(width * height * 3) and I need to pick every third element or simply change the format to luminance which is fairly tough because it's wrapped by the ImageTensor. 😅

tafaust commented 2 years ago

For anyone struggling with this issue; see my previous post on how to debug the mismatch leading to the Bad state: failed precondition. Apparently, I forgot to convert the image to grayscale. I had to copy all the code from tflite_flutter_helper package into my code because it was not exported. PR pending: https://github.com/am15h/tflite_flutter_helper/pull/54/files

The grayscale conversion of the image package did not work in this case.