search

quic / qidk

Other
76 stars 16 forks source link

Cannot find -lSNPE in Android Studio Build #2

Closed karo-txs closed 5 months ago

karo-txs commented 12 months ago

Hi, I'm having trouble building the Sentiment Analysis demo. Here is every step by step I have done and the error I find.

Step-by-step:

  1. Create .DLC in Docker Container a. Dockerfile: FROM ubuntu:20.04

    WORKDIR /qidk COPY . /qidk RUN cd /qidk

    RUN apt-get update && apt-get -y install sudo

    RUN DEBIAN_FRONTEND=noninteractive TZ=America/Recife apt-get -y install tzdata RUN apt-get install ffmpeg libsm6 libxext6 lsb-release -y && \ apt-get install python3.8 python3-distutils libpython3.8 python3-pip -y && \ apt-get install python3.8-venv -y && \ python3 -m pip install --upgrade pip && \ sudo apt-get install make zip -y && \ sudo apt-get install systemd -y && \ apt-get clean && \ apt-get install git -y && \ apt-get install vim -y

    RUN apt-get update && \ apt-get install -y wget && \ rm -rf /var/lib/apt/lists/*

    RUN git clone https://github.com/quic/qidk.git

    RUN dpkg -i ./QualcommPackageManager3.3.0.82.4.Linux-x86.deb ENV SNPE_ROOT=../opt/qcom/aistack/snpe/2.12.0.230626 ENV PYTHONPATH="$SNPE_ROOT"/lib/python

    b. Commands to export model to .dlc

    b.1. Download SNPE

    qpm-cli --login login password
qpm-cli --license-activate qualcomm_neural_processing_sdk
qpm-cli --extract qualcomm_neural_processing_sdk

    b.2. Check Linux Dependencies

    sudo bash ${SNPE_ROOT}/bin/check-linux-dependency.sh

    b.3. Clang-9 To check if the environment is setup properly to use clang-9, the following command can be used:

    ${SNPE_ROOT}/bin/envcheck -c

    b.4. Python Dependencies

    python3.8 -m venv venv
source venv/bin/activate

pip install tensorflow==2.10.1
pip install tflite==2.3.0
pip install torch==1.8.1
pip install onnx==1.11.0
pip install transformers==4.18.0
pip install datasets==2.4.0

"$SNPE_ROOT"/bin/check-python-dependency

pip install numpy==1.21.0

    b.5. Environment Setup After Linux Platform Dependencies have been satisfied, the user environment can be set with the provided envsetup.sh script.

    source ${SNPE_ROOT}/bin/envsetup.sh

    b.6. Generate MobileBERT-SST2 model as Tensorflow Frozen Graph

    python qidk/Solutions/NLPSolution2-SentimentAnalysis/scripts/sa_model_gen.py

    b.7. Convert generated frozen graph into DLC (Deep Learning Container):

    snpe-tensorflow-to-dlc -i frozen_models/mobilebert_sst2.pb -d input_ids 1,128 -d attention_mask 1,128 --out_node Identity -o frozen_models/mobilebert_sst2.dlc
    Result:

==============================================================
Building Network
==============================================================
2023-07-22 22:17:02,922 - 235 - INFO - Resolving static sub-graphs in network...
2023-07-22 22:17:03,060 - 235 - INFO - Resolving static sub-graphs in network, complete.
2023-07-22 22:17:12,638 - 235 - INFO - INFO_INITIALIZATION_SUCCESS: 
2023-07-22 22:17:13,739 - 235 - INFO - INFO_CONVERSION_SUCCESS: Conversion completed successfully
2023-07-22 22:17:13,912 - 235 - INFO - INFO_WRITE_SUCCESS:

    b.8. Offline Preparation (caching) of DLC (for optimizing model loading/INIT time on DSP accelerator

    snpe-dlc-graph-prepare --input_dlc frozen_models/mobilebert_sst2.dlc --use_float_io --htp_archs v73

  2. Windows Setup 3.1. Install Qualcomm AI ENgine Direct SDK (SNEP) https://developer.qualcomm.com/software/qualcomm-ai-engine-direct-sdk 3.2. Verify Windows dependencies

    $ & "${SNPE_ROOT}/bin/check-windows-dependency.ps1"

    Result: WARNING: The version of VS BuildTools 14.36.32532.00 found has not been validated. Recommended to use known stable VS BuildTools version 14.34 WARNING: The version of CMake 3.26.0 found has not been validated. Recommended to use known stable CMake version 3.21 All Done

    3.3. Open Developer PowerShell for VS2022 as Administrator Visual Studio > Tools > Command Line > Developer PowerShell

    Set-ExecutionPolicy RemoteSigned
& "${SNPE_ROOT}\bin\envsetup.ps1"

    Result [INFO] AISW SDK environment set [INFO] SNPE_ROOT: C:\Qualcomm\AIStack\SNPE\2.12.0.230626 *Note: envsetup.ps1 is only required to be executed on Windows host

  3. Android Setup Android Studio: Android Studio Eletric Eel | 2022.1.1 JDK 11.0.15 NDK 21.4.7075529

    4.1. Copy .dlc to assets 4.2. Copy libSnpeHtpPrepare and libSNPE to jniLibs (Not found others files (i. e. libSnpeHtpV73Stub, libSnpeHtpV69Stub, libc++_shared) 4.3. Build app

    Error: C:/Users/USER/AppData/Local/Android/Sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/windows-x86_64/lib/gcc/aarch64-linux-android/4.9.x/../../../../aarch64-linux-android/bin\ld: skipping incompatible C:/Users/USER/Documents/bxbct/qualcomm/sidiqidk/qidk/Solutions/NLPSolution2-SentimentAnalysis/SentimentAnalysis/app/src/main/cpp/../jniLibs/arm64-v8a/libSNPE.so when searching for -lSNPE C:/Users/USER/AppData/Local/Android/Sdk/ndk/21.4.7075529/toolchains/llvm/prebuilt/windows-x86_64/lib/gcc/aarch64-linux-android/4.9.x/../../../../aarch64-linux-android/bin\ld: cannot find -lSNPE clang++: error: linker command failed with exit code 1 (use -v to see invocation) ninja: build stopped: subcommand failed.

Thanks for attention.

quic-rneti commented 5 months ago

Hello - We made a detailed course with hands on sessions : https://academy.qualcomm.com/course-catalog/AI-on-QIDK

Can you check Chapter-4 here (lab session), and let us know the exact issue you are facing ? From above description this looks like an environment setup issue.

quic-rneti commented 5 months ago

Hello - it looks like SNPE assets are not copied properly in Android build directory. Pls fix this.