mocleiri / tensorflow-micropython-examples

A custom micropython firmware integrating tensorflow lite for microcontrollers and ulab to implement the tensorflow micro examples.
MIT License
170 stars 79 forks source link

Implement person detection example #24

Closed mocleiri closed 2 years ago

mocleiri commented 2 years ago

I don't have a camera but I think I should be able to fake it by getting the tensorflow lite model and then the pictures that are from the upstream repo for a person and a not person image and then run the model to classify those pictures.

mocleiri commented 2 years ago

I was able to code an example and run it with the following outputs:

~/git/tensorflow-micropython-examples/examples/person_detection$ ./micropython ./person_detection.py
interpreter_make_new: model size = 300568, tensor area = 139264
Classify No Person Image
setup 9612 bytes on the inputTensor.
'not a person' = -25, 'person' = 25
Classify Person Image
setup 9612 bytes on the inputTensor.
'not a person' = -113, 'person' = 113
mocleiri commented 2 years ago

I think something is wrong because the test code says we should expect the 'not a person' value to be greater than 'person' when looking at the no_person_image.

Refer to the test: https://github.com/tensorflow/tflite-micro/blob/main/tensorflow/lite/micro/examples/person_detection/person_detection_test.cc

mocleiri commented 2 years ago

Can also try this to get the test to run locally: make -f tensorflow/lite/micro/tools/make/Makefile person_detection_test_int8

mocleiri commented 2 years ago

git/tensorflow-micropython-examples/tensorflow$ make -f tensorflow/lite/micro/tools/make/Makefile test_person_detection_test_int8 find: ‘../google/’: No such file or directory tensorflow/lite/micro/tools/make/downloads/flatbuffers already exists, skipping the download. tensorflow/lite/micro/tools/make/downloads/pigweed already exists, skipping the download. tensorflow/lite/micro/tools/make/downloads/person_model_int8 already exists, skipping the download. tensorflow/lite/micro/tools/make/gen/linux_x86_64_default/bin/person_detection_test_int8 '~ALL TESTS PASSED~' linux Testing TestInvoke person data. person score: 113, no person score: -113

no person data. person score: -72, no person score: 72

Ran successfully

1/1 tests passed


Something is different on the no person test data.  The person data results match
mocleiri commented 2 years ago

The upstream test is using a completely zero input for the not a person test: https://github.com/tensorflow/tflite-micro/blob/b6cbaea68bf6fd8d778ab82817c3089288b57b34/tensorflow/lite/micro/examples/person_detection/person_detection_test.cc#L105

I need to change the image used from the c++ test case to the actual not a person and then compare that result.

mocleiri commented 2 years ago

I filed an issue for this with tflite-micro#395

mocleiri commented 2 years ago

I merged the code associated into the main branch. I'm going to keep this open to track the response to the issue I filed about the model not working right on their no example.

It could be for a lot of different reasons. There could be a threshold like with micro speech or it could just be that the accuracy isn't the highest with image detection models that can fit onto micro controllers.

mocleiri commented 2 years ago

I decided to close this issue even with the tflite ticket open.

If they select a working no person image and update I will then update when I'm using on our end.