Optimized OpenAI's Whisper TFLite Port for Efficient Offline Inference on Edge Devices
MIT License
134
stars
29
forks
source link
Compiled `whisper.android` for Android x86_64, it worked for about 30 mins but since then it crashes on processing with `Fatal signal 11 (SIGSEGV), code 128 (SI_KERNEL), fault addr 0x0 in tid 18464` #32
I compiled Android x86_64 build for running whisper.android on Android 11 x86_64. I was able to run it initially without any issues and it returned back with the transcribed text. But after about 30 mins, it mysteriously started crashing without any changes made to the java or cpp layer
Steps performed to build x86_64 Android
1) Clone tensorflow repo
2) Build with bazel using bazel build -c opt //tensorflow/lite:libtensorflowlite.so --linkopt="-s" --cpu=x86_64 --config=android_x86_64 --fat_apk_cpu=x86_64
3) Copied generated libtensorflowlite.so and replaced it @ cpp/tf-lite-api/generated-libs/x86_64
4) Updated tensorflow headers along with absl and flatbuffers
5) Run
Here are the logs
03:28:00.092 D Returned asset path: /data/user/0/com.whispertflite/files/jfk.wav
03:28:00.092 D WaveFile: /data/user/0/com.whispertflite/files/jfk.wav
03:28:00.092 D Update is received, Message: Processing...
03:28:02.735 D Skipping profile installation for com.whispertflite
03:28:23.831 A Fatal signal 11 (SIGSEGV), code 128 (SI_KERNEL), fault addr 0x0 in tid 18464 (Thread-3), pid 17926 (m.whispertflite)
03:28:23.846 A pid: 17926, tid: 18464, name: Thread-3 >>> com.whispertflite <<<
03:28:23.902 A #00 pc 0000000000beac5a /data/app/~~jUSctKS5U4J9GxcVh9Mxsw==/com.whispertflite-jOl8yuCGBpVIrjyiqNnSfw==/base.apk!libtensorflowlite.so (offset 0x70f000) (BuildId: 8687099524ac9c38f7c004e514867227)
03:28:23.902 A #01 pc 0000000000959178 /data/app/~~jUSctKS5U4J9GxcVh9Mxsw==/com.whispertflite-jOl8yuCGBpVIrjyiqNnSfw==/base.apk!libtensorflowlite.so (offset 0x70f000) (BuildId: 8687099524ac9c38f7c004e514867227)
03:28:23.902 A #02 pc 0000000000bec1b2 /data/app/~~jUSctKS5U4J9GxcVh9Mxsw==/com.whispertflite-jOl8yuCGBpVIrjyiqNnSfw==/base.apk!libtensorflowlite.so (offset 0x70f000) (BuildId: 8687099524ac9c38f7c004e514867227)
03:28:23.902 A #03 pc 0000000000959b62 /data/app/~~jUSctKS5U4J9GxcVh9Mxsw==/com.whispertflite-jOl8yuCGBpVIrjyiqNnSfw==/base.apk!libtensorflowlite.so (offset 0x70f000) (BuildId: 8687099524ac9c38f7c004e514867227)
03:28:23.902 A #04 pc 000000000091f02b /data/app/~~jUSctKS5U4J9GxcVh9Mxsw==/com.whispertflite-jOl8yuCGBpVIrjyiqNnSfw==/base.apk!libtensorflowlite.so (offset 0x70f000) (BuildId: 8687099524ac9c38f7c004e514867227)
03:28:23.902 A #05 pc 000000000043e5df /data/app/~~jUSctKS5U4J9GxcVh9Mxsw==/com.whispertflite-jOl8yuCGBpVIrjyiqNnSfw==/base.apk!libtensorflowlite.so (offset 0x70f000) (BuildId: 8687099524ac9c38f7c004e514867227)
03:28:23.902 A #06 pc 0000000000433cd8 /data/app/~~jUSctKS5U4J9GxcVh9Mxsw==/com.whispertflite-jOl8yuCGBpVIrjyiqNnSfw==/base.apk!libtensorflowlite.so (offset 0x70f000) (BuildId: 8687099524ac9c38f7c004e514867227)
03:28:23.902 A #07 pc 0000000000c5b3de /data/app/~~jUSctKS5U4J9GxcVh9Mxsw==/com.whispertflite-jOl8yuCGBpVIrjyiqNnSfw==/base.apk!libtensorflowlite.so (offset 0x70f000) (tflite::Subgraph::OpInvoke(TfLiteRegistration const&, TfLiteNode*)+270) (BuildId: 8687099524ac9c38f7c004e514867227)
03:28:23.902 A #08 pc 0000000000c5c097 /data/app/~~jUSctKS5U4J9GxcVh9Mxsw==/com.whispertflite-jOl8yuCGBpVIrjyiqNnSfw==/base.apk!libtensorflowlite.so (offset 0x70f000) (tflite::Subgraph::InvokeImpl()+1399) (BuildId: 8687099524ac9c38f7c004e514867227)
03:28:23.902 A #09 pc 0000000000c5baf8 /data/app/~~jUSctKS5U4J9GxcVh9Mxsw==/com.whispertflite-jOl8yuCGBpVIrjyiqNnSfw==/base.apk!libtensorflowlite.so (offset 0x70f000) (tflite::Subgraph::Invoke()+24) (BuildId: 8687099524ac9c38f7c004e514867227)
03:28:23.902 A #10 pc 0000000000891fe1 /data/app/~~jUSctKS5U4J9GxcVh9Mxsw==/com.whispertflite-jOl8yuCGBpVIrjyiqNnSfw==/base.apk!libtensorflowlite.so (offset 0x70f000) (tflite::ops::builtin::while_kernel::Eval_dynamic(TfLiteContext*, TfLiteNode*)+689) (BuildId: 8687099524ac9c38f7c004e514867227)
03:28:23.902 A #11 pc 0000000000893411 /data/app/~~jUSctKS5U4J9GxcVh9Mxsw==/com.whispertflite-jOl8yuCGBpVIrjyiqNnSfw==/base.apk!libtensorflowlite.so (offset 0x70f000) (tflite::ops::builtin::while_kernel::Eval(TfLiteContext*, TfLiteNode*)+305) (BuildId: 8687099524ac9c38f7c004e514867227)
03:28:23.902 A #12 pc 0000000000c5b3de /data/app/~~jUSctKS5U4J9GxcVh9Mxsw==/com.whispertflite-jOl8yuCGBpVIrjyiqNnSfw==/base.apk!libtensorflowlite.so (offset 0x70f000) (tflite::Subgraph::OpInvoke(TfLiteRegistration const&, TfLiteNode*)+270) (BuildId: 8687099524ac9c38f7c004e514867227)
03:28:23.902 A #13 pc 0000000000c5c097 /data/app/~~jUSctKS5U4J9GxcVh9Mxsw==/com.whispertflite-jOl8yuCGBpVIrjyiqNnSfw==/base.apk!libtensorflowlite.so (offset 0x70f000) (tflite::Subgraph::InvokeImpl()+1399) (BuildId: 8687099524ac9c38f7c004e514867227)
03:28:23.902 A #14 pc 0000000000c5baf8 /data/app/~~jUSctKS5U4J9GxcVh9Mxsw==/com.whispertflite-jOl8yuCGBpVIrjyiqNnSfw==/base.apk!libtensorflowlite.so (offset 0x70f000) (tflite::Subgraph::Invoke()+24) (BuildId: 8687099524ac9c38f7c004e514867227)
03:28:23.902 A #15 pc 0000000000c17ac2 /data/app/~~jUSctKS5U4J9GxcVh9Mxsw==/com.whispertflite-jOl8yuCGBpVIrjyiqNnSfw==/base.apk!libtensorflowlite.so (offset 0x70f000) (tflite::impl::Interpreter::Invoke()+114) (BuildId: 8687099524ac9c38f7c004e514867227)
03:28:23.902 A #16 pc 0000000000066482 /data/app/~~jUSctKS5U4J9GxcVh9Mxsw==/com.whispertflite-jOl8yuCGBpVIrjyiqNnSfw==/base.apk!libaudioEngine.so (offset 0x430000) (TFLiteEngine::transcribeBuffer(std::__ndk1::vector<float, std::__ndk1::allocator<float> >)+498) (BuildId: 61305ae1e4739848602842cf100b2e63d86d9044)
03:28:23.902 A #17 pc 00000000000667bc /data/app/~~jUSctKS5U4J9GxcVh9Mxsw==/com.whispertflite-jOl8yuCGBpVIrjyiqNnSfw==/base.apk!libaudioEngine.so (offset 0x430000) (TFLiteEngine::transcribeFile(char const*)+236) (BuildId: 61305ae1e4739848602842cf100b2e63d86d9044)
03:28:23.902 A #18 pc 0000000000068f02 /data/app/~~jUSctKS5U4J9GxcVh9Mxsw==/com.whispertflite-jOl8yuCGBpVIrjyiqNnSfw==/base.apk!libaudioEngine.so (offset 0x430000) (Java_com_whispertflite_engine_WhisperEngineNative_transcribeFile+66) (BuildId: 61305ae1e4739848602842cf100b2e63d86d9044)
03:28:23.902 A #19 pc 000000000002fc5b /data/app/~~jUSctKS5U4J9GxcVh9Mxsw==/com.whispertflite-jOl8yuCGBpVIrjyiqNnSfw==/oat/x86_64/base.odex (art_jni_trampoline+203)
03:28:23.902 A #21 pc 00000000008067e0 /data/app/~~jUSctKS5U4J9GxcVh9Mxsw==/com.whispertflite-jOl8yuCGBpVIrjyiqNnSfw==/oat/x86_64/base.vdex (com.whispertflite.engine.WhisperEngineNative.transcribeFile+4)
03:28:23.902 A #23 pc 00000000008065ee /data/app/~~jUSctKS5U4J9GxcVh9Mxsw==/com.whispertflite-jOl8yuCGBpVIrjyiqNnSfw==/oat/x86_64/base.vdex (com.whispertflite.asr.Whisper.threadFunction+126)
03:28:23.902 A #25 pc 0000000000806344 /data/app/~~jUSctKS5U4J9GxcVh9Mxsw==/com.whispertflite-jOl8yuCGBpVIrjyiqNnSfw==/oat/x86_64/base.vdex (com.whispertflite.asr.Whisper.lambda$start$0$com-whispertflite-asr-Whisper+12)
03:28:23.902 A #27 pc 00000000008061d0 /data/app/~~jUSctKS5U4J9GxcVh9Mxsw==/com.whispertflite-jOl8yuCGBpVIrjyiqNnSfw==/oat/x86_64/base.vdex (com.whispertflite.asr.Whisper$$ExternalSyntheticLambda0.run+4)
03:28:23.985 E channel 'ddeecf4 com.whispertflite/com.whispertflite.MainActivity (server)' ~ Channel is unrecoverably broken and will be disposed!
I compiled Android x86_64 build for running whisper.android on Android 11 x86_64. I was able to run it initially without any issues and it returned back with the transcribed text. But after about 30 mins, it mysteriously started crashing without any changes made to the java or cpp layer
Steps performed to build x86_64 Android 1) Clone tensorflow repo 2) Build with bazel using
bazel build -c opt //tensorflow/lite:libtensorflowlite.so --linkopt="-s" --cpu=x86_64 --config=android_x86_64 --fat_apk_cpu=x86_64
3) Copied generatedlibtensorflowlite.so
and replaced it @cpp/tf-lite-api/generated-libs/x86_64
4) Updated tensorflow headers along with absl and flatbuffers 5) RunHere are the logs