Static library of tflite-lib and esp-nn component and link it to project

[techlearnerr]

I want to create the static library of tflite-lib and esp-nn components of the project and link it to the project of micro_speech.

I find the static library of both components under the micro_speech\build\esp-idf\

esp-nn static library under 
micro_speech\build\esp-idf\esp-nn
libesp-nn.a

tflite-lib static library under 
micro_speech\build\esp-idf\tflite-lib
libtflite-lib.a

CMakeLists.txt as follows

cmake_minimum_required(VERSION 3.16)
#set(EXTRA_COMPONENT_DIRS ../../components/)  comment out this line to not call the library components 

include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(micro_speech)

I saved these 2 static libraries under the main/libs folder I tried to link these static libraries by modifying the main\CMakeLisst.txt file as follows

idf_component_register(
    SRCS main.c main_functions.cc audio_provider.cc feature_provider.cc
         model.cc recognize_commands.cc command_responder.cc
         micro_features_generator.cc micro_model_settings.cc ringbuf.c
    PRIV_REQUIRES spi_flash driver esp_timer esp_wifi nvs_flash
    INCLUDE_DIRS "." "include")

# Link against the static libraries
target_link_libraries(${COMPONENT_LIB} PRIVATE
    "${CMAKE_CURRENT_SOURCE_DIR}/libs/libtflite-lib.a"
    "${CMAKE_CURRENT_SOURCE_DIR}/libs/libesp-nn.a")
# Add include directories for the static libraries
target_include_directories(${COMPONENT_LIB} PRIVATE
    "${CMAKE_CURRENT_SOURCE_DIR}/libs"
    "${CMAKE_CURRENT_SOURCE_DIR}/libs/tensorflow/lite"
)

When i build flash and monitor the code, its build the code and code stuck at the main_function.cc

if ((model_input->dims->size != 2) || (model_input->dims->data[0] != 1) || (model_input->dims->data[1] != (kFeatureSliceCount * kFeatureSliceSize)) || (model_input->type != kTfLiteInt8)) {
    MicroPrintf("Bad input tensor parameters in model");
    return;
  }

when checking the components using idf.py menuconfig there was no extra component there

Extra info Target platform: ESP32-S3 IDF version release/v5.0 I will wait for your positive feedback

[vikramdattu]

@techlearnerr looks like you're linking the libraries correctly.

if ((model_input->dims->size != 2) || (model_input->dims->data[0] != 1) || 
       (model_input->dims->data[1] != (kFeatureSliceCount * kFeatureSliceSize)) ||
       (model_input->type != kTfLiteInt8)) {
    MicroPrintf("Bad input tensor parameters in model");
    return;
 }

Since, you mentioned about above check failing,

1. Did you make any changes to the model, settings from micro_model_settings.h etc?
2. Does the example run without above error when you not use static linkage?

[techlearnerr]

1- I updated the model and it's working fine without the static library.

Now esp32S3 is restarting at this point.

if ((model_input->dims->size != 2) || (model_input->dims->data[0] != 1) || 
       (model_input->dims->data[1] != (kFeatureSliceCount * kFeatureSliceSize)) ||
       (model_input->type != kTfLiteInt8)) {
    MicroPrintf("Bad input tensor parameters in model");
    return;
 }

Error after executing the above code is as follows

ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x8 (TG1WDT_SYS_RST),boot:0x8 (SPI_FAST_FLASH_BOOT)
Saved PC:0x420030ff
0x420030ff: panic_handler at D:/.espressif/frameworks/esp-idf-v5.0.1/components/esp_system/port/panic_handler.c:147 (discriminator 3)

SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3810,len:0x164c
load:0x403c9700,len:0xbe0
load:0x403cc700,len:0x2ef8
entry 0x403c9900

Am I missing something to create and link that library to the project?

[vikramdattu]

Hi @techlearnerr ,

I linked libs as you did and I am able to build, flash and run the example properly. I have to add additionally, following lines to main/CMakeList.txt:

target_include_directories(${COMPONENT_LIB} PRIVATE
    "${CMAKE_CURRENT_SOURCE_DIR}/libs"
    "${CMAKE_CURRENT_SOURCE_DIR}/../../../components/tflite-lib/"
    "${CMAKE_CURRENT_SOURCE_DIR}/../../../components/tflite-lib/third_party/gemmlowp"
    "${CMAKE_CURRENT_SOURCE_DIR}/../../../components/tflite-lib/third_party/kissfft"
    "${CMAKE_CURRENT_SOURCE_DIR}/../../../components/tflite-lib/third_party/flatbuffers/include"
    )

Since, you are getting error from below lines

if ((model_input->dims->size != 2) || (model_input->dims->data[0] != 1) || 
       (model_input->dims->data[1] != (kFeatureSliceCount * kFeatureSliceSize)) ||
       (model_input->type != kTfLiteInt8)) {
    MicroPrintf("Bad input tensor parameters in model");
    return;
 }

you can add prints over here and find a clue for the issue.

BTW, If your sole motivation to use libs for tflite-lib is to reduce the build time, you may consider using ccache. It will make subsequent builds blazing fast.

https://docs.espressif.com/projects/esp-idf/en/v4.2/esp32/api-guides/build-system.html#idf-py-options

[techlearnerr]

Hi @vikramdattu Thanks for your quick response.

Did you also comment/remove the following line from the CMakeLists.txt?

#set(EXTRA_COMPONENT_DIRS ../../components/) comment out this line to not call the library components

[techlearnerr]

Hi @vikramdattu I tried to debug the issue by adding the print statements

  model_input = interpreter->input(0);
  printf("Input tensor details:\n");
printf("Number of dimensions: %d\n", model_input->dims->size);
printf("Size of dimension 0: %d\n", model_input->dims->data[0]);
printf("Size of dimension 1: %d\n", model_input->dims->data[1]);
printf("Data type: %d\n", model_input->type);

  if ((model_input->dims->size != 2) || (model_input->dims->data[0] != 1) ||
    (model_input->dims->data[1] != (kFeatureSliceCount * kFeatureSliceSize)) ||
    (model_input->type != kTfLiteInt8)) {
    MicroPrintf("Bad input tensor parameters in model");
    return;
    }

output at the terminal is as follows

Input tensor details:

code is halted at this point.

unable to print the number of dimensions, Size of dimension 0, Size of dimension 1, and Data type

could you please help me with this issue?

[vikramdattu]

Hi @vikramdattu Thanks for your quick response.

Did you also comment/remove the following line from the CMakeLists.txt?

#set(EXTRA_COMPONENT_DIRS ../../components/) comment out this line to not call the library components

@techlearnerr yes, I commented out that line to remove those components out. BTW, I experimented on ESP32-S3 based dev board. Will try with ESP32-EYE and share observations.

Meanwhile can you check this?

If your sole motivation to use libs for tflite-lib is to reduce the build time, you may consider using ccache. It will make subsequent builds blazing fast.

[techlearnerr]

@vikramdattu which IDF version you are using? Can you please share your static lib?

[vikramdattu]

@techlearnerr I have pushed the changes on the branch: test/microspeech_use_libs I have added libs for both ESP32 and ESP32-S3. I am able to reproduce the issue on ESP32.

[techlearnerr]

@vikramdattu Thanks for your reply. I used your test branch and build the code with esp32s3 with esp-idf version 5.0. I build, flashed, and run the example but the code stuck at the main_function.cc as I mentioned in the first comment.

if ((model_input->dims->size != 2) || (model_input->dims->data[0] != 1) || 
       (model_input->dims->data[1] != (kFeatureSliceCount * kFeatureSliceSize)) ||
       (model_input->type != kTfLiteInt8)) {
    MicroPrintf("Bad input tensor parameters in model");
    return;
 }

Its restarts the esp32S3 because of system panic,

ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0x8 (TG1WDT_SYS_RST),boot:0x8 (SPI_FAST_FLASH_BOOT)
Saved PC:0x420030ff
0x420030ff: panic_handler at D:/.espressif/frameworks/esp-idf-v5.0.1/components/esp_system/port/panic_handler.c:147 (discriminator 3)

SPIWP:0xee
mode:DIO, clock div:1
load:0x3fce3810,len:0x164c
load:0x403c9700,len:0xbe0
load:0x403cc700,len:0x2ef8
entry 0x403c9900