search

espressif / esp-tflite-micro

TensorFlow Lite Micro for Espressif Chipsets
Apache License 2.0
389 stars 83 forks source link

Cannot build the person detection model #27

Closed ASabovic closed 1 year ago

ASabovic commented 1 year ago

Hi guys,

I am trying to build the person detection model, but I always get the same error. I followed the guidelines for installation from the official esp website. Then, in the esp folder I cloned this git repository in order to get the person detection code. After that, I tried to build the example with the command: 

idf.py build

Here is the error that I get every time: 

-- Building ESP-IDF components for target esp32
CMake Error at /home/adnan/esp/esp-idf/tools/cmake/build.cmake:201 (message):
  Failed to resolve component 'tflite-lib'.
Call Stack (most recent call first):
  /home/adnan/esp/esp-idf/tools/cmake/build.cmake:232 (__build_resolve_and_add_req)
  /home/adnan/esp/esp-idf/tools/cmake/build.cmake:507 (__build_expand_requirements)
  /home/adnan/esp/esp-idf/tools/cmake/project.cmake:384 (idf_build_process)
  CMakeLists.txt:4 (project)

-- Configuring incomplete, errors occurred!
See also "/home/adnan/esp/esp-idf/tflite-micro-esp-examples/examples/person_detection/build/CMakeFiles/CMakeOutput.log".
cmake failed with exit code 1

P.S. I am able to run the simple examples from get-started folder and that part works without issues. I am using the ESP32-WROVER-IE board.

vikramdattu commented 1 year ago

Hi @ASabovic this should not happen. Did you make any changes to the example? The below line in particular is very important, and adds extra components to the build system: https://github.com/espressif/tflite-micro-esp-examples/blob/1987ce2e5ae7b518a0d0ca16fdecc1783460fd2b/examples/person_detection/CMakeLists.txt#L2

ASabovic commented 1 year ago

Hi @vikramdattu, no I did not. The CMakeLists file is the same as you sent. Maybe, I made a mistake somewhere during the installation. I followed the instruction for version v4.3.4 for Linux platform on this link https://docs.espressif.com/projects/esp-idf/en/v4.3.4/esp32/get-started/index.html#installation-step-by-step. After that, I just cloned your repo in my esp folder. Is there a possibility that I made a mistake with environment variable or some other prerequisites?

ASabovic commented 1 year ago

This is the full procedure I performed last time: 

adnan@adnan-Inspiron-7380:~/esp$ . $HOME/esp/esp-idf/export.sh
Detecting the Python interpreter
Checking "python" ...
Python 3.6.9
"python" has been detected
Adding ESP-IDF tools to PATH...
Using Python interpreter in /home/adnan/.espressif/python_env/idf4.4_py3.6_env/bin/python
Checking if Python packages are up to date...
Python requirements from /home/adnan/esp/esp-idf/requirements.txt are satisfied.
Added the following directories to PATH:
  /home/adnan/esp/esp-idf/components/esptool_py/esptool
  /home/adnan/esp/esp-idf/components/espcoredump
  /home/adnan/esp/esp-idf/components/partition_table
  /home/adnan/esp/esp-idf/components/app_update
Done! You can now compile ESP-IDF projects.
Go to the project directory and run:

  idf.py build

adnan@adnan-Inspiron-7380:~/esp$ cp -r $IDF_PATH/examples/person_detection .
adnan@adnan-Inspiron-7380:~/esp$ cd person_detection/
adnan@adnan-Inspiron-7380:~/esp/person_detection$ idf.py build
Executing action: all (aliases: build)
Running cmake in directory /home/adnan/esp/person_detection/build
Executing "cmake -G Ninja -DPYTHON_DEPS_CHECKED=1 -DESP_PLATFORM=1 -DCCACHE_ENABLE=0 /home/adnan/esp/person_detection"...
-- IDF_TARGET not set, using default target: esp32
-- Found Git: /usr/bin/git (found version "2.38.1") 
-- Component directory /home/adnan/esp/esp-idf/components/esp-nn does not contain a CMakeLists.txt file. No component will be added
-- Component directory /home/adnan/esp/esp-idf/components/esp32-camera does not contain a CMakeLists.txt file. No component will be added
-- The C compiler identification is GNU 8.4.0
-- The CXX compiler identification is GNU 8.4.0
-- The ASM compiler identification is GNU
-- Found assembler: /home/adnan/.espressif/tools/xtensa-esp32-elf/esp-2021r2-patch5-8.4.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /home/adnan/.espressif/tools/xtensa-esp32-elf/esp-2021r2-patch5-8.4.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /home/adnan/.espressif/tools/xtensa-esp32-elf/esp-2021r2-patch5-8.4.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Project is not inside a git repository, or git repository has no commits; will not use 'git describe' to determine PROJECT_VER.
-- Building ESP-IDF components for target esp32
CMake Error at /home/adnan/esp/esp-idf/tools/cmake/build.cmake:201 (message):
  Failed to resolve component 'esp32-camera'.
Call Stack (most recent call first):
  /home/adnan/esp/esp-idf/tools/cmake/build.cmake:227 (__build_resolve_and_add_req)
  /home/adnan/esp/esp-idf/tools/cmake/build.cmake:507 (__build_expand_requirements)
  /home/adnan/esp/esp-idf/tools/cmake/project.cmake:384 (idf_build_process)
  CMakeLists.txt:4 (project)

-- Configuring incomplete, errors occurred!
See also "/home/adnan/esp/person_detection/build/CMakeFiles/CMakeOutput.log".
cmake failed with exit code 1
vikramdattu commented 1 year ago

Hi @ASabovic instead of copying the example to IDF's directory, just go to the tflite-micro-esp-examples/examples/person_detection of cloned repo and run idf.py build. This should work fine.

ASabovic commented 1 year ago

Hi @vikramdattu, I did it like that and this is the result. 

adnan@adnan-Inspiron-7380:~$ cd esp/
adnan@adnan-Inspiron-7380:~/esp$ git clone https://github.com/espressif/tflite-micro-esp-examples.git
Cloning into 'tflite-micro-esp-examples'...
remote: Enumerating objects: 2310, done.
remote: Counting objects: 100% (470/470), done.
remote: Compressing objects: 100% (242/242), done.
remote: Total 2310 (delta 309), reused 271 (delta 219), pack-reused 1840
Receiving objects: 100% (2310/2310), 2.04 MiB | 10.78 MiB/s, done.
Resolving deltas: 100% (1500/1500), done.
adnan@adnan-Inspiron-7380:~/esp$ cd tflite-micro-esp-examples/examples/person_detection/
adnan@adnan-Inspiron-7380:~/esp/tflite-micro-esp-examples/examples/person_detection$ idf.py build
idf.py: command not found
adnan@adnan-Inspiron-7380:~/esp/tflite-micro-esp-examples/examples/person_detection$

After this I set the environmental variables using this command: 

. $HOME/esp/esp-idf/export.sh

but still same error as from previous comments: 

adnan@adnan-Inspiron-7380:~/esp/tflite-micro-esp-examples/examples/person_detection$ . $HOME/esp/esp-idf/export.sh
Setting IDF_PATH to '/home/adnan/esp/esp-idf'
Detecting the Python interpreter
Checking "python" ...
Checking "python3" ...
Python 3.6.9
"python3" has been detected
Adding ESP-IDF tools to PATH...
Not using an unsupported version of tool cmake found in PATH: 3.21.3.
Not using an unsupported version of tool ninja found in PATH: 1.8.2.
Using Python interpreter in /home/adnan/.espressif/python_env/idf4.4_py3.6_env/bin/python
Checking if Python packages are up to date...
Python requirements from /home/adnan/esp/esp-idf/requirements.txt are satisfied.
Added the following directories to PATH:
  /home/adnan/esp/esp-idf/components/esptool_py/esptool
  /home/adnan/esp/esp-idf/components/espcoredump
  /home/adnan/esp/esp-idf/components/partition_table
  /home/adnan/esp/esp-idf/components/app_update
  /home/adnan/.espressif/tools/xtensa-esp32-elf/esp-2021r2-patch5-8.4.0/xtensa-esp32-elf/bin
  /home/adnan/.espressif/tools/esp32ulp-elf/2.35_20220830/esp32ulp-elf/bin
  /home/adnan/.espressif/tools/cmake/3.23.1/bin
  /home/adnan/.espressif/tools/openocd-esp32/v0.11.0-esp32-20220706/openocd-esp32/bin
  /home/adnan/.espressif/tools/ninja/1.10.2/
  /home/adnan/.espressif/python_env/idf4.4_py3.6_env/bin
  /home/adnan/esp/esp-idf/tools
Done! You can now compile ESP-IDF projects.
Go to the project directory and run:

  idf.py build

adnan@adnan-Inspiron-7380:~/esp/tflite-micro-esp-examples/examples/person_detection$ idf.py build
Executing action: all (aliases: build)
Running cmake in directory /home/adnan/esp/tflite-micro-esp-examples/examples/person_detection/build
Executing "cmake -G Ninja -DPYTHON_DEPS_CHECKED=1 -DESP_PLATFORM=1 -DCCACHE_ENABLE=0 /home/adnan/esp/tflite-micro-esp-examples/examples/person_detection"...
-- IDF_TARGET not set, using default target: esp32
-- Found Git: /usr/bin/git (found version "2.38.1") 
-- Component directory /home/adnan/esp/tflite-micro-esp-examples/components/esp-nn does not contain a CMakeLists.txt file. No component will be added
-- Component directory /home/adnan/esp/tflite-micro-esp-examples/components/esp32-camera does not contain a CMakeLists.txt file. No component will be added
-- The C compiler identification is GNU 8.4.0
-- The CXX compiler identification is GNU 8.4.0
-- The ASM compiler identification is GNU
-- Found assembler: /home/adnan/.espressif/tools/xtensa-esp32-elf/esp-2021r2-patch5-8.4.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /home/adnan/.espressif/tools/xtensa-esp32-elf/esp-2021r2-patch5-8.4.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /home/adnan/.espressif/tools/xtensa-esp32-elf/esp-2021r2-patch5-8.4.0/xtensa-esp32-elf/bin/xtensa-esp32-elf-g++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Building ESP-IDF components for target esp32
CMake Error at /home/adnan/esp/esp-idf/tools/cmake/build.cmake:201 (message):
  Failed to resolve component 'esp-nn'.
Call Stack (most recent call first):
  /home/adnan/esp/esp-idf/tools/cmake/build.cmake:232 (__build_resolve_and_add_req)
  /home/adnan/esp/esp-idf/tools/cmake/build.cmake:233 (__build_expand_requirements)
  /home/adnan/esp/esp-idf/tools/cmake/build.cmake:507 (__build_expand_requirements)
  /home/adnan/esp/esp-idf/tools/cmake/project.cmake:384 (idf_build_process)
  CMakeLists.txt:4 (project)

-- Configuring incomplete, errors occurred!
See also "/home/adnan/esp/tflite-micro-esp-examples/examples/person_detection/build/CMakeFiles/CMakeOutput.log".
cmake failed with exit code 1
adnan@adnan-Inspiron-7380:~/esp/tflite-micro-esp-examples/examples/person_detection$

It is constantly error related to the additional components that go with your repository. Any idea how to solve this? Thanks in advance!

vikramdattu commented 1 year ago

@ASabovic you also need submodule update for projects using subprojects. Inside your tflite-micro-esp-examples please run git submodule update --init --recursive and the build should go through.

ASabovic commented 1 year ago

@vikramdattu thanks a lot, it works now. As I am using the WROVER board, which camera pins are not defined in the configuration, I know there is a way to do that using command: 

idf.py menuconfig -> Application Configuration -> Camera configuration

As together with this board, I am using the Arducam OV2640 Mini 2MP camera module that has 8 pins, is it possible to define them in this configuration? Based on the current pinout there is 16 pins that should be considered, no?

vikramdattu commented 1 year ago

Hi @ASabovic the example above uses esp32-camera library and the standard 16-pin definition to directly connect the camera. Since you are using ArduCam which has its own chip for data transfer, you will need to find another way to interface these 8-pins and get camera output. I think this will help you in the right direction: https://github.com/zhaochenyuangit/ArduCAM-ESP32#arducam-ov2640-mini-2mp-plus-on-esp32

ASabovic commented 1 year ago

Hi @vikramdattu thanks for the explanation and useful link. Regarding the esp32-camera library that you used in your examples, can you just please confirm me if the example was tested on these two boards:

1) ESP32-CAM WIFI + Bluetooth with Camera Module OV2640 2MP 2) FREENOVE ESP32-WROVER CAM Board

vikramdattu commented 1 year ago

Hi, @ASabovic I have tested the example on below 2 camera boards:

  1. ESP-EYE
  2. ESP32-S3-EYE

For the purpose of an example, I thought this is good enough. Whenever I get my hands on the ESP32-Cam board, I will also test with that board.

ASabovic commented 1 year ago

Hi @vikramdattu, thanks for the answer. I have already ordered the esp-eye board to test the person detection model on it. This means that the current code can immediately be flashed on the board or first the camera configuration using idf.py menuconfing must be set to be compatible with the esp-eye? Thanks in advance!

vikramdattu commented 1 year ago

Hi @ASabovic, CONFIG_CAMERA_MODULE_ESP_EYE is selected by default. Hence, you should be good to go without any changes. https://github.com/espressif/tflite-micro-esp-examples/blob/e6e9001b45c0e1cf6a4455712a0b24bcab0b50bb/examples/person_detection/sdkconfig.defaults#L60

ASabovic commented 1 year ago

Thanks, @vikramdattu!

vikramdattu commented 1 year ago

@ASabovic did you solve your issue?

osugiw commented 1 year ago

Hi guys, I am using ESP32 Cam by AI thinker and it was built successfully. Moreover, the camera also init successfully. However, I can't find a way to do inference using the camera. Anyone could help me about this? I would appreciate so much for your help!

Here is the screenshot of my logs: image

vikramdattu commented 1 year ago

Hi @osugiw please follow this step to enable camera and display module if you have interfaced any: https://github.com/espressif/tflite-micro-esp-examples/tree/master/examples/person_detection#using-cli-for-inferencing By default it is set to CLI_ONLY_INFERENCE, i.e., inference using embedded images.

Thanks.

osugiw commented 1 year ago

Hi @osugiw please follow this step to enable camera and display module if you have interfaced any: https://github.com/espressif/tflite-micro-esp-examples/tree/master/examples/person_detection#using-cli-for-inferencing By default it is set to CLI_ONLY_INFERENCE, i.e., inference using embedded images.

Thanks.

Thank you for your respond! Well, I already followed those steps, but what I have to do next after build and flash successfully? Do I need to enter something to the CLI like inference for images ("detect_image 0")? By the way, I noticed something if I do not add app_camera_init() inside the main.cc file then my camera won't be initialized. Am I correct?

image

vikramdattu commented 1 year ago

What I have to do next after build and flash successfully? Do I need to enter something to the CLI like inference for images ("detect_image 0")?

When camera is enabled, you do not need to enter anything manually. There should be a printf for person_score and no person score. Please disable display support if you are not using it.

By the way, I noticed something if I do not add app_camera_init() inside the main.cc file then my camera won't be initialized.

Camera init is done intenally here, which gets called from setup. So, no need to call app_camera_init explicitly.

osugiw commented 1 year ago

What I have to do next after build and flash successfully? Do I need to enter something to the CLI like inference for images ("detect_image 0")?

When camera is enabled, you do not need to enter anything manually. There should be a printf for person_score and no person score. Please disable display support if you are not using it.

By the way, I noticed something if I do not add app_camera_init() inside the main.cc file then my camera won't be initialized.

Camera init is done intenally here, which gets called from setup. So, no need to call app_camera_init explicitly.

I am sorry for not reading more carefully for the initialization. Anyway, thanks for your explanation!

osugiw commented 1 year ago

Sorry to ask again @vikramdattu .. I tried your suggestion, and the results look like this: image

It means the camera not recognized yet right?

vikramdattu commented 1 year ago

Hi @osugiw from logs, I can see the commands getting registered. That means, you are still on CLI_ONLY_INFERENCE. Please check if that define is commented out. You may want to debug adding some printfs if it is the case. Please try a clean build to be sure.

osugiw commented 1 year ago

@vikramdattu Yap, from the beginning I already configured as follow, and chose CONFIG_CAMERA_MODULE_AI_THINKER from idf.py menuconfig. image

vikramdattu commented 1 year ago

Right, please comment out the define on line number 16. It restricts the operations to cli only.

osugiw commented 1 year ago

Right, please comment out the define on line number 16. It restricts the operations to cli only.

I have just realized the document was not correctly wrote. The steps tell us to uncomment #define CLI_ONLY_INFERENCE 1 if we want to use the camera. In contrast, we need to comment out if we want to use the camera. image

vikramdattu commented 1 year ago

Ohh, thanks for pointing it out! I will fix the doc.

osugiw commented 1 year ago

Ohh, thanks for pointing it out! I will fix the doc.

Well, thanks for your support too! Now it is running perfectly.