search

ggerganov / whisper.cpp

Port of OpenAI's Whisper model in C/C++
MIT License
34.85k stars 3.55k forks source link

How to CrossCompile for aarch64 device #1821

Open 711suys opened 8 months ago

711suys commented 8 months ago

Hi,

I am new to this and need a little advice. How to cross compile whisper for aarch64 device? I need to make use opencl gpu support which means i also need to cross compile CLBlast library too. What changes do i need to make in my cmakelist/makefile to be able to cross-compile for aarch64 in an ubuntu machine?

some assistance would be amazing. thanks

gpokat commented 8 months ago
  1. Install Android studio
  2. Open project from examples https://github.com/ggerganov/whisper.cpp/tree/master/examples/whisper.android
  3. Then you can build it for aarch64 and deploy to emulator or device. This is for cpu mode only. To enable CLBLAST:
  4. Grab OpenCL.so from device (probably /vendor/lib64/OpenCL.so)
  5. In Khronos repo grab CL folder
  6. Checkout the CLBLAST repo and make for aarch64 example 
    
export NDK_PATH=/playground/sdk/ndk/25.2.9519653

cmake .. \ -DCMAKE_SYSTEM_NAME=Android \ -DCMAKE_SYSTEM_VERSION=33 \ -DCMAKE_ANDROID_ARCH_ABI=arm64-v8a \ -DCMAKE_ANDROID_NDK=$NDK_PATH \ -DCMAKE_ANDROID_STL_TYPE=c++_static \ -DOPENCL_ROOT=/playground/mediatek \ -DCMAKE_FIND_ROOT_PATH_MODE_LIBRARY=BOTH \ -DCMAKE_FIND_ROOT_PATH_MODE_INCLUDE=BOTH


7. Tune android project cmake file whisper.cpp/examples/whisper.android/lib/src/main/jni/whisper/CMakeLists.txt to include device 's opencl and clblast libs. Make sure to add links. src, headers and also add_compile_definitions(GGML_USE_CLBLAST)
8. Deploy.

But on my Helio G99 the enabling CLBLAST does not lead to any speed-up neither in benchmark and inferences times. Also the inference itself became broken. Probably need to run CLBLAST tuners or as mentioned before the openCL in android broken somehow. The CPU mode pretty good if you set 4-6 threads, the time processing drops to ~2secs in Release mode.
*for tiny models