Closed mocleiri closed 2 years ago
I was able to code an example and run it with the following outputs:
~/git/tensorflow-micropython-examples/examples/person_detection$ ./micropython ./person_detection.py
interpreter_make_new: model size = 300568, tensor area = 139264
Classify No Person Image
setup 9612 bytes on the inputTensor.
'not a person' = -25, 'person' = 25
Classify Person Image
setup 9612 bytes on the inputTensor.
'not a person' = -113, 'person' = 113
I think something is wrong because the test code says we should expect the 'not a person' value to be greater than 'person' when looking at the no_person_image.
Refer to the test: https://github.com/tensorflow/tflite-micro/blob/main/tensorflow/lite/micro/examples/person_detection/person_detection_test.cc
Can also try this to get the test to run locally: make -f tensorflow/lite/micro/tools/make/Makefile person_detection_test_int8
git/tensorflow-micropython-examples/tensorflow$ make -f tensorflow/lite/micro/tools/make/Makefile test_person_detection_test_int8
find: ‘../google/’: No such file or directory
tensorflow/lite/micro/tools/make/downloads/flatbuffers already exists, skipping the download.
tensorflow/lite/micro/tools/make/downloads/pigweed already exists, skipping the download.
tensorflow/lite/micro/tools/make/downloads/person_model_int8 already exists, skipping the download.
tensorflow/lite/micro/tools/make/gen/linux_x86_64_default/bin/person_detection_test_int8 '~ALL TESTS PASSED~' linux
Testing TestInvoke
person data. person score: 113, no person score: -113
no person data. person score: -72, no person score: 72
Ran successfully
1/1 tests passed
Something is different on the no person test data. The person data results match
The upstream test is using a completely zero input for the not a person test: https://github.com/tensorflow/tflite-micro/blob/b6cbaea68bf6fd8d778ab82817c3089288b57b34/tensorflow/lite/micro/examples/person_detection/person_detection_test.cc#L105
I need to change the image used from the c++ test case to the actual not a person and then compare that result.
I filed an issue for this with tflite-micro#395
I merged the code associated into the main branch. I'm going to keep this open to track the response to the issue I filed about the model not working right on their no example.
It could be for a lot of different reasons. There could be a threshold like with micro speech or it could just be that the accuracy isn't the highest with image detection models that can fit onto micro controllers.
I decided to close this issue even with the tflite ticket open.
If they select a working no person image and update I will then update when I'm using on our end.
I don't have a camera but I think I should be able to fake it by getting the tensorflow lite model and then the pictures that are from the upstream repo for a person and a not person image and then run the model to classify those pictures.