search

VeriSilicon / tflite-vx-delegate

Tensorflow Lite external delegate based on TIM-VX
MIT License
41 stars 23 forks source link

Unable to build the external delegate for the A311D processor on Android armv7-a #171

Open FrankdenUijl opened 1 year ago

FrankdenUijl commented 1 year ago

If I understand correctly I can build an External Delegate (for example: libvx_delegate.so) via this repository and run TensorFlow Lite models on the NPU of the A311D processor?

My first question, do I need to modify TensorFlow Lite to run this delegate, or can I swap this delegate with a different external delegate with a pre-build TensorFlow Lite plugin?

For example replace the external delegate on this location with the libvx_delegate.so (this runs TensorFlow Lite): https://github.com/asus4/tf-lite-unity-sample/tree/master/Packages/com.github.asus4.tflite/Plugins/Android/armeabi-v7a

My second question is, I'm unable to build this external delegate:

cmake .. -DCMAKE_TOOLCHAIN_FILE=C:/Users/Frank/AppData/Local/Android/Sdk/ndk/25.2.9519653/build/cmake/android.toolchain.cmake
-- Building for: Visual Studio 17 2022
-- Setting build type to Release, for debug builds use'-DCMAKE_BUILD_TYPE=Debug'.
-- ANDROID_PLATFORM not set. Defaulting to minimum supported version
19.
-- The C compiler identification is unknown
-- The CXX compiler identification is unknown
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - failed
-- Check for working C compiler: C:/Users/Frank/AppData/Local/Android/Sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/windows-x86_64/bin/clang.exe
-- Check for working C compiler: C:/Users/Frank/AppData/Local/Android/Sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/windows-x86_64/bin/clang.exe - broken
CMake Error at C:/Program Files/CMake/share/cmake-3.26/Modules/CMakeTestCCompiler.cmake:67 (message):
  The C compiler

    "C:/Users/Frank/AppData/Local/Android/Sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/windows-x86_64/bin/clang.exe"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: C:/Users/Frank/Documents/4Growth/wksp/tflite-vx-delegate/builds/CMakeFiles/CMakeScratch/TryCompile-ubk14s

    Run Build Command(s):C:/Program Files/Microsoft Visual Studio/2022/Community/MSBuild/Current/Bin/amd64/MSBuild.exe cmTC_86eea.vcxproj /p:Configuration=Debug /p:Platform=x64 /p:VisualStudioVersion=17.0 /v:n && MSBuild version 17.5.0+6f08c67f3 for .NET Framework
    Build started 15-3-2023 09:59:19.
    Included response file: C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Current\Bin\amd64\MSBuild.rsp

    Project "C:\Users\Frank\Documents\4Growth\wksp\tflite-vx-delegate\builds\CMakeFiles\CMakeScratch\TryCompile-ubk14s\cmTC_86eea.vcxproj" on node 1 (default targets).
    SetBuildDefaultEnvironmentVariables:
      ANDROID_HOME=C:\Program Files (x86)\Android\android-sdk
      ANDROID_SDK_ROOT=C:\Program Files (x86)\Android\android-sdk
      ANT_HOME=
      JAVA_HOME=C:\Program Files\Android\jdk\jdk-8.0.302.8-hotspot\jdk8u302-b08
      NDK_ROOT=C:\\Microsoft\AndroidNDK\android-ndk-r23c
    PrepareForBuild:
      Creating directory "cmTC_86eea.dir\Debug\".
      Creating directory "C:\Users\Frank\Documents\4Growth\wksp\tflite-vx-delegate\builds\CMakeFiles\CMakeScratch\TryCompile-ubk14s\Debug\".
      Creating directory "cmTC_86eea.dir\Debug\cmTC_86eea.tlog\".
    InitializeBuildStatus:
      Creating "cmTC_86eea.dir\Debug\cmTC_86eea.tlog\unsuccessfulbuild" because "AlwaysCreate" was specified.
    ClCompile:
      C:\\Microsoft\AndroidNDK\android-ndk-r23c\toolchains\llvm\prebuilt\windows-x86_64\bin\clang.exe -c -fdiagnostics-format=msvc --gcc-toolchain="C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\toolchains\\x86_64-4.9\\prebuilt\\windows-x86_64" -target "x86_64-none-linux-android1" --sysroot="C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\toolchains\\llvm\\prebuilt\\windows-x86_64\\sysroot" -isystem "C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\toolchains\\llvm\\prebuilt\\windows-x86_64\\sysroot\\usr\\include\\x86_64-linux-android" -isystem "C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\toolchains\\llvm\\prebuilt\\windows-x86_64\\lib64\\clang\\12.0.9\\include" -isystem "C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\sources\\android\\support\\include" -isystem "C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\toolchains\\llvm\\prebuilt\\windows-x86_64\\sysroot\\usr\\include" -I "C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\sources\\cxx-stl\\llvm-libc++\\include" -I "C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\sources\\cxx-stl\\llvm-libc++abi\\include" -I "C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\toolchains\\llvm\\prebuilt\\windows-x86_64\\sysroot\\usr\\include" -I "C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\toolchains\\llvm\\prebuilt\\windows-x86_64\\sysroot\\usr\\include\\x86_64-linux-android" -I "C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\sources\\android\\support\\include" -I "C:\\\\Microsoft\\AndroidNDK\\android-ndk-r23c\\toolchains\\llvm\\prebuilt\\windows-x86_64\\lib64\\clang\\12.0.9\\include" -gline-tables-only -o "cmTC_86eea.dir\\Debug\\testCCompiler.o" -Wall -O3 -fno-strict-aliasing -fomit-frame-pointer -fno-exceptions -ffunction-sections -fdata-sections -fstack-protector -fpic -fno-short-enums -D __ANDROID_API__=1 -D NDEBUG -D ANDROID -D _FORTIFY_SOURCE=2 -D "CMAKE_INTDIR=\"Debug\"" -x c  -g -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -march=armv7-a -mthumb -Wformat -Werror=format-security -fno-limit-debug-info "C:\\Users\\Frank\\Documents\\4Growth\\wksp\\tflite-vx-delegate\\builds\\CMakeFiles\\CMakeScratch\\TryCompile-ubk14s\\testCCompiler.c"
      testCCompiler.c
    clang : warning : argument unused during compilation: '-mthumb' [-Wunused-command-line-argument] [C:\Users\Frank\Documents\4Growth\wksp\tflite-vx-delegate\builds\CMakeFiles\CMakeScratch\TryCompile-ubk14s\cmTC_86eea.vcxproj]
    CLANGCOMPILE : error : unknown target CPU 'armv7-a' [C:\Users\Frank\Documents\4Growth\wksp\tflite-vx-delegate\builds\CMakeFiles\CMakeScratch\TryCompile-ubk14s\cmTC_86eea.vcxproj]
      note: valid target CPU values are: nocona, core2, penryn, bonnell, atom, silvermont, slm, goldmont, goldmont-plus, tremont, nehalem, corei7, westmere, sandybridge, corei7-avx, ivybridge, core-avx-i, haswell, core-avx2, broadwell, skylake, skylake-avx512, skx, cascadelake, cooperlake, cannonlake, icelake-client, icelake-server, tigerlake, sapphirerapids, alderlake, knl, knm, k8, athlon64, athlon-fx, opteron, k8-sse3, athlon64-sse3, opteron-sse3, amdfam10, barcelona, btver1, btver2, bdver1, bdver2, bdver3, bdver4, znver1, znver2, znver3, x86-64, x86-64-v2, x86-64-v3, x86-64-v4
    Done Building Project "C:\Users\Frank\Documents\4Growth\wksp\tflite-vx-delegate\builds\CMakeFiles\CMakeScratch\TryCompile-ubk14s\cmTC_86eea.vcxproj" (default targets) -- FAILED.

    Build FAILED.

    "C:\Users\Frank\Documents\4Growth\wksp\tflite-vx-delegate\builds\CMakeFiles\CMakeScratch\TryCompile-ubk14s\cmTC_86eea.vcxproj" (default target) (1) ->
    (ClCompile target) ->
      clang : warning : argument unused during compilation: '-mthumb' [-Wunused-command-line-argument] [C:\Users\Frank\Documents\4Growth\wksp\tflite-vx-delegate\builds\CMakeFiles\CMakeScratch\TryCompile-ubk14s\cmTC_86eea.vcxproj]

    "C:\Users\Frank\Documents\4Growth\wksp\tflite-vx-delegate\builds\CMakeFiles\CMakeScratch\TryCompile-ubk14s\cmTC_86eea.vcxproj" (default target) (1) ->
    (ClCompile target) ->
      CLANGCOMPILE : error : unknown target CPU 'armv7-a' [C:\Users\Frank\Documents\4Growth\wksp\tflite-vx-delegate\builds\CMakeFiles\CMakeScratch\TryCompile-ubk14s\cmTC_86eea.vcxproj]

        1 Warning(s)
        1 Error(s)

    Time Elapsed 00:00:00.20

  CMake will not be able to correctly generate this project.
Call Stack (most recent call first):
  CMakeLists.txt:31 (project)

-- Configuring incomplete, errors occurred!

Our A311D processor runs on Android 32-bit.

Any help is appreciated, thanks!

FrankdenUijl commented 1 year ago

I was able to build it via installing ubuntu via the windows store, and then run the following command: cmake .. -DCMAKE_TOOLCHAIN_FILE=~/wksp/android-ndk-r25c/build/cmake/android.toolchain.cmake -DANDROID_ABI=armeabi-v7a -DANDROID_PLATFORM=android-23 -DANDROID_STL=c++_static -DCONFIG=A311D

Now this worked! The problem I have is when running make vx_delegate:

make vx_delegate
[  0%] Building CXX object _deps/tim-vx-build/src/tim/CMakeFiles/tim-vx.dir/vx/builtin_op.cc.o
/bin/sh: 1: /home/frank/wksp/tflite-vx-delegate/build/_deps/tim-vx-build/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-g++: not found
make[3]: *** [_deps/tim-vx-build/src/tim/CMakeFiles/tim-vx.dir/build.make:76: _deps/tim-vx-build/src/tim/CMakeFiles/tim-vx.dir/vx/builtin_op.cc.o] Error 127
make[2]: *** [CMakeFiles/Makefile2:6734: _deps/tim-vx-build/src/tim/CMakeFiles/tim-vx.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:1320: CMakeFiles/vx_delegate.dir/rule] Error 2
make: *** [Makefile:169: vx_delegate] Error 2

Any idea how to solve this? Ofcours my first question still is valid, but if you can build vx_delegate myself I can test it myself :)

FrankdenUijl commented 1 year ago

When I download the requested package gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu myself and extract it, Im stuck at the next problem:

make vx_delegate
[  0%] Building CXX object _deps/tim-vx-build/src/tim/CMakeFiles/tim-vx.dir/vx/builtin_op.cc.o
aarch64-linux-gnu-g++: error: unrecognized command line option ‘--target=armv7-none-linux-androideabi28’
aarch64-linux-gnu-g++: error: unrecognized command line option ‘-mthumb’
make[3]: *** [_deps/tim-vx-build/src/tim/CMakeFiles/tim-vx.dir/build.make:76: _deps/tim-vx-build/src/tim/CMakeFiles/tim-vx.dir/vx/builtin_op.cc.o] Error 1
make[2]: *** [CMakeFiles/Makefile2:6734: _deps/tim-vx-build/src/tim/CMakeFiles/tim-vx.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:1320: CMakeFiles/vx_delegate.dir/rule] Error 2
make: *** [Makefile:169: vx_delegate] Error 2

seems like --target=armv7-none-linux-androideabi28 is not supported? Not sure what -mthumb is?

FrankdenUijl commented 1 year ago

I didn't get any further with make so I tried bazel, but also no luck:

frank@DESKTOP-1OFUK9C:~/wksp10/tflite-vx-delegate$ bazel build --config=android_arm :vx_delegate.so
Starting local Bazel server and connecting to it...
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=1 --terminal_columns=120
INFO: Reading rc options for 'build' from /home/frank/wksp10/tflite-vx-delegate/.bazelrc:
  Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /home/frank/wksp10/tflite-vx-delegate/.bazelrc:
  'build' options: --apple_platform_type=macos --define framework_shared_object=true --define open_source_build=true --java_toolchain=//third_party/toolchains/java:tf_java_toolchain --host_java_toolchain=//third_party/toolchains/java:tf_java_toolchain --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --noincompatible_prohibit_aapt1 --enable_platform_specific_config --config=v2
INFO: Found applicable config definition build:v2 in file /home/frank/wksp10/tflite-vx-delegate/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:android_arm in file /home/frank/wksp10/tflite-vx-delegate/.bazelrc: --config=android --cpu=armeabi-v7a --fat_apk_cpu=armeabi-v7a
INFO: Found applicable config definition build:android in file /home/frank/wksp10/tflite-vx-delegate/.bazelrc: --crosstool_top=//external:android/crosstool --host_crosstool_top=@bazel_tools//tools/cpp:toolchain --noenable_platform_specific_config --copt=-w --cxxopt=-std=c++17 --host_cxxopt=-std=c++17
DEBUG: Rule 'io_bazel_rules_docker' indicated that a canonical reproducible form can be obtained by modifying arguments shallow_since = "1556410077 -0400"
DEBUG: Repository io_bazel_rules_docker instantiated at:
  /home/frank/wksp10/tflite-vx-delegate/WORKSPACE:26:14: in <toplevel>
  /home/frank/.cache/bazel/_bazel_frank/2d94a7f5d8109712bec02b48c04f03b8/external/org_tensorflow/tensorflow/workspace0.bzl:105:34: in workspace
  /home/frank/.cache/bazel/_bazel_frank/2d94a7f5d8109712bec02b48c04f03b8/external/bazel_toolchains/repositories/repositories.bzl:37:23: in repositories
Repository rule git_repository defined at:
  /home/frank/.cache/bazel/_bazel_frank/2d94a7f5d8109712bec02b48c04f03b8/external/bazel_tools/tools/build_defs/repo/git.bzl:199:33: in <toplevel>
WARNING: /home/frank/.cache/bazel/_bazel_frank/2d94a7f5d8109712bec02b48c04f03b8/external/tim_vx/prebuilt-sdk/x86_64_linux/BUILD:11:11: in linkstatic attribute of cc_library rule @tim_vx//prebuilt-sdk/x86_64_linux:VIV_SDK_LIB: setting 'linkstatic=1' is recommended if there are no object files
INFO: Analyzed target //:vx_delegate.so (72 packages loaded, 2634 targets configured).
INFO: Found 1 target...
ERROR: /home/frank/.cache/bazel/_bazel_frank/2d94a7f5d8109712bec02b48c04f03b8/external/com_google_absl/absl/strings/BUILD.bazel:83:11: Compiling absl/strings/internal/escaping.cc failed: (Exit 1): false failed: error executing command /bin/false -MD -MF bazel-out/armeabi-v7a-opt/bin/external/com_google_absl/absl/strings/_objs/internal/escaping.pic.d ... (remaining 28 argument(s) skipped)
Target //:vx_delegate.so failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 11.980s, Critical Path: 2.85s
INFO: 105 processes: 93 internal, 12 local.
FAILED: Build did NOT complete successfully
sunshinemyson commented 1 year ago

I was able to build it via installing ubuntu via the windows store, and then run the following command: cmake .. -DCMAKE_TOOLCHAIN_FILE=~/wksp/android-ndk-r25c/build/cmake/android.toolchain.cmake -DANDROID_ABI=armeabi-v7a -DANDROID_PLATFORM=android-23 -DANDROID_STL=c++_static -DCONFIG=A311D

Now this worked! The problem I have is when running make vx_delegate:

make vx_delegate
[  0%] Building CXX object _deps/tim-vx-build/src/tim/CMakeFiles/tim-vx.dir/vx/builtin_op.cc.o
/bin/sh: 1: /home/frank/wksp/tflite-vx-delegate/build/_deps/tim-vx-build/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-g++: not found
make[3]: *** [_deps/tim-vx-build/src/tim/CMakeFiles/tim-vx.dir/build.make:76: _deps/tim-vx-build/src/tim/CMakeFiles/tim-vx.dir/vx/builtin_op.cc.o] Error 127
make[2]: *** [CMakeFiles/Makefile2:6734: _deps/tim-vx-build/src/tim/CMakeFiles/tim-vx.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:1320: CMakeFiles/vx_delegate.dir/rule] Error 2
make: *** [Makefile:169: vx_delegate] Error 2

Any idea how to solve this? Ofcours my first question still is valid, but if you can build vx_delegate myself I can test it myself :)

Please don't use "-DCONFIG=A311D" when you build for Android.

sunshinemyson commented 1 year ago

I was able to build it via installing ubuntu via the windows store, and then run the following command: cmake .. -DCMAKE_TOOLCHAIN_FILE=~/wksp/android-ndk-r25c/build/cmake/android.toolchain.cmake -DANDROID_ABI=armeabi-v7a -DANDROID_PLATFORM=android-23 -DANDROID_STL=c++_static -DCONFIG=A311D

Now this worked! The problem I have is when running make vx_delegate:

make vx_delegate
[  0%] Building CXX object _deps/tim-vx-build/src/tim/CMakeFiles/tim-vx.dir/vx/builtin_op.cc.o
/bin/sh: 1: /home/frank/wksp/tflite-vx-delegate/build/_deps/tim-vx-build/gcc-linaro-7.3.1-2018.05-x86_64_aarch64-linux-gnu/bin/aarch64-linux-gnu-g++: not found
make[3]: *** [_deps/tim-vx-build/src/tim/CMakeFiles/tim-vx.dir/build.make:76: _deps/tim-vx-build/src/tim/CMakeFiles/tim-vx.dir/vx/builtin_op.cc.o] Error 127
make[2]: *** [CMakeFiles/Makefile2:6734: _deps/tim-vx-build/src/tim/CMakeFiles/tim-vx.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:1320: CMakeFiles/vx_delegate.dir/rule] Error 2
make: *** [Makefile:169: vx_delegate] Error 2

Any idea how to solve this? Ofcours my first question still is valid, but if you can build vx_delegate myself I can test it myself :)

Please don't use "-DCONFIG=A311D" when you build for Android.