Static build of TfLiteArmnnDelegate results in "Requested unknown backend"

[desmoteo]

I have been trying to build static libarmnn.a and libarmnnDelegate.a to use on aarch64 devices (cortex-A53) in order to limit storage usage. I managed to compile the libraries and link the target executable, anyway when running on the target I get the following error:

Info: ArmNN v26.0.0

Info: Initialization time: 0.14 ms

INFO: TfLiteArmnnDelegate: Requested unknown backend GpuAcc INFO: TfLiteArmnnDelegate: Requested unknown backend CpuAcc INFO: TfLiteArmnnDelegate: Requested unknown backend CpuRef I>nfo: Shutdown time: 0.03 ms

exception: TfLiteArmnnDelegate: No known backend specified.

I built the libraries following the cross compilation documentation and modified the cmake command for armnn build as follows:

CXX=aarch64-linux-gnu-g++ CC=aarch64-linux-gnu-gcc cmake .. -DARMCOMPUTE_ROOT=$HOME/armnn-devenv/ComputeLibrary -DARMCOMPUTE_BUILD_DIR=$HOME/armnn-devenv/ComputeLibrary/build/ -DARMCOMPUTENEON=1 -DARMNNREF=1 -DBUILD_ARMNN_TFLITE_DELEGATE=1 -DTENSORFLOW_ROOT=$BASEDIR/tensorflow -DTFLITE_LIB_ROOT=/home/matteo/Documents/tensorflow/build-aarch64/ -DFLATBUFFERS_ROOT=$HOME/armnn-devenv/flatbuffers-arm64 -DFLATC_DIR=$HOME/armnn-devenv/flatbuffers-1.12.0/build -DPROTOBUF_ROOT=$HOME/armnn-devenv/google/arm64_pb_install/ -DPROTOBUF_LIBRARY_DEBUG=$HOME/armnn-devenv/google/arm64_pb_install/lib/libprotobuf.so.23.0.0 -DPROTOBUF_LIBRARY_RELEASE=$HOME/armnn-devenv/google/arm64_pb_install/lib/libprotobuf.so.23.0.0

I create the delegate as follows:

            // Create the Arm NN Delegate
            // std::vector<BackendId> backends = {Compute::CpuAcc}
            std::vector<armnn::BackendId> backends = {armnn::Compute::GpuAcc, armnn::Compute::CpuAcc, armnn::Compute::CpuRef};
            armnnDelegate::DelegateOptions delegateOptions(backends);
            delegateOptions.SetLoggingSeverity("debug");
            std::unique_ptr<TfLiteDelegate, decltype(&armnnDelegate::TfLiteArmnnDelegateDelete)>
                    theArmnnDelegate(armnnDelegate::TfLiteArmnnDelegateCreate(delegateOptions),
                                     armnnDelegate::TfLiteArmnnDelegateDelete);
            // Instruct the Interpreter to use the armnnDelegate
            if (m_interpreter->ModifyGraphWithDelegate(theArmnnDelegate.get()) != kTfLiteOk)
            {

                throw std::runtime_error(std::string("Failed to create armnn_delegate"));
            }

I don't understand what I am doing wrong. Any help is welcome.

[FrancisMurtagh-arm]

Hi @desmoteo,

Can you link the diff of your CMake? The backends (e.g. armnn/src/backends/neon/CMakeLists.txt) are normally statically linked to libarmnn.so, so I'd be interested to see how you included them as part of libarmnn.a.

Thanks, Francis.

[desmoteo]

Thanks for the quick answer! A few changes were sufficient to get some libarmnn.a and libarmnnDelegate.a.

diff --git a/CMakeLists.txt b/CMakeLists.txt
index 2b0c95254..99dda4c69 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -512,7 +512,7 @@ foreach(lib ${armnnLibraries})
     list(APPEND armnn_sources $<TARGET_OBJECTS:${lib}>)
 endforeach()

-add_library_ex(armnn SHARED ${armnn_sources})
+add_library_ex(armnn STATIC ${armnn_sources})

 target_compile_definitions(armnn PRIVATE "ARMNN_COMPILING_DLL")

diff --git a/delegate/CMakeLists.txt b/delegate/CMakeLists.txt
index b43feb7f9..c8e711b6e 100644
--- a/delegate/CMakeLists.txt
+++ b/delegate/CMakeLists.txt
@@ -50,7 +50,7 @@ list(APPEND armnnDelegate_sources
         src/Unpack.hpp
         src/Transpose.hpp)

-add_library(armnnDelegate SHARED ${armnnDelegate_sources})
+add_library(armnnDelegate STATIC ${armnnDelegate_sources})

 target_include_directories(armnnDelegate
         PUBLIC
diff --git a/delegate/cmake/Modules/FindTfLite.cmake b/delegate/cmake/Modules/FindTfLite.cmake
index 41f55e3e8..a99344739 100644
--- a/delegate/cmake/Modules/FindTfLite.cmake
+++ b/delegate/cmake/Modules/FindTfLite.cmake
@@ -17,6 +17,7 @@ find_library(TfLite_LIB
         NAMES
             "libtensorflow_lite_all.so"
             "libtensorflowlite.so"
+           "libtensorflow-lite.a"
         HINTS
             ${TFLITE_LIB_ROOT}
             ${TFLITE_LIB_ROOT}/tensorflow/lite)
@@ -24,7 +25,8 @@ find_library(TfLite_LIB
 find_path(TfLite_Schema_INCLUDE_PATH
             schema_generated.h
         HINTS
-            ${TFLITE_LIB_ROOT}/tensorflow/lite/schema)
+            ${TFLITE_LIB_ROOT}/tensorflow/lite/schema
+            ${TFLITE_LIB_ROOT}/../tensorflow/lite/schema)

 ## Set TFLITE_FOUND

[FrancisMurtagh-arm]

Hi @desmoteo,

Ah OK, I'm thinking you need to append the backend sources e.g the _${armnnNeonBackendsources} from armnn/src/backends/neon/CMakeLists.txt to the _armnnsources in armnn/CMakeLists.txt for each of the backends.

My assumption is you can't link a static library to a static library so you need to roll all the objects into one.

Can you use nm to check if the libarmnn.a contains any Neon specific functions? nm -C libarmnn.a | grep NeonLayerSupport

Hope that's of some help, Francis.

[desmoteo]

Thanks @FrancisMurtagh-arm

This is interesting. The command

nm -C libarmnn.a | grep NeonLayerSupport

produces a lot of output. But when I run the same on the target executable, no reference to "Neon" is found. I'll try to dig in, can it be related to the BackendRegistry inner workings?

Matteo

[desmoteo]

OK I managed to include all missing symbols by using the following linker option:

-Wl,-whole-archive -larmnn -Wl,-no-whole-archive

The statically linked executable is now sensibly smaller than the previous ex + shared libs bundle. ArmNN with CpuAcc backend seems more than 2x faster than XNNPack delegate.

Thanks again for the suggestions, @FrancisMurtagh-arm !

[FrancisMurtagh-arm]

Hi @desmoteo,

I wasn't aware of that option but it does make sense; thanks and nicely done.

Do you plan to publish these results anywhere? I'm sure others would be interested. @jimfly01

Thanks, Francis.

[desmoteo]

Hi @FrancisMurtagh-arm

Yes, I will write it down and post a link here.

Closing this