search

k2-fsa / sherpa-onnx

Speech-to-text, text-to-speech, and speaker recongition using next-gen Kaldi with onnxruntime without Internet connection. Support embedded systems, Android, iOS, Raspberry Pi, RISC-V, x86_64 servers, websocket server/client, C/C++, Python, Kotlin, C#, Go, NodeJS, Java, Swift
https://k2-fsa.github.io/sherpa/onnx/index.html
Apache License 2.0
928 stars 175 forks source link

Android TTS crash #823

Open jeffcraighead opened 2 weeks ago

jeffcraighead commented 2 weeks ago

I'm trying to use some updated versions of sherpa-onnx for TTS. I'm getting the following error with my own application and the example project included in the repo (android/SherpaOnnxTts folder). This is with the vits-vctk model, which has been downloaded from huggingface. I have tried several tags 1.9.4, 1.9.19, and 1.9.22 all with the same result. To test I'm grabbing the associated sherpa-onnx-<version>-android.tar.bz2 file and putting the entire jniLibs folder from the archive into the android\SherpaOnnxTts\app\src\main\ folder then building and running from AndroidStudio on a Galaxy Tab 7a. Any suggestions on what could be the issue? Seems like some data mismatch with the model maybe, but not sure where to start with that since I'm grabbing it from your online archive.

string is: test 2024-04-29 17:57:28.718 15243-15313 sherpa-onnx com.k2fsa.sherpa.onnx W Raw text: test 2024-04-29 17:57:28.718 15243-15313 libc++abi com.k2fsa.sherpa.onnx E terminating due to uncaught exception of type std::out_of_range: unordered_map::at: key not found 2024-04-29 17:57:28.718 15243-15313 libc com.k2fsa.sherpa.onnx A Fatal signal 6 (SIGABRT), code -1 (SI_QUEUE) in tid 15313 (Thread-5), pid 15243 (fsa.sherpa.onnx) 2024-04-29 17:57:29.024 15317-15317 DEBUG pid-15317 A Cmdline: com.k2fsa.sherpa.onnx 2024-04-29 17:57:29.024 15317-15317 DEBUG pid-15317 A pid: 15243, tid: 15313, name: Thread-5 >>> com.k2fsa.sherpa.onnx <<< 2024-04-29 17:57:29.024 15317-15317 DEBUG pid-15317 A #01 pc 00000000000d83c8 /data/app/~~OUhfaIyYdmT8xWe_dEAYRg==/com.k2fsa.sherpa.onnx-aZXVAxMdANOR3ijL3r4eJQ==/lib/arm64/libkaldi-native-fbank-core.so (BuildId: 1cc21a4c5ee3e85b21492ccea6fe24d092b296dc) 2024-04-29 17:57:29.024 15317-15317 DEBUG pid-15317 A #02 pc 00000000000d7cbc /data/app/~~OUhfaIyYdmT8xWe_dEAYRg==/com.k2fsa.sherpa.onnx-aZXVAxMdANOR3ijL3r4eJQ==/lib/arm64/libkaldi-native-fbank-core.so (BuildId: 1cc21a4c5ee3e85b21492ccea6fe24d092b296dc) 2024-04-29 17:57:29.024 15317-15317 DEBUG pid-15317 A #03 pc 00000000000d7b78 /data/app/~~OUhfaIyYdmT8xWe_dEAYRg==/com.k2fsa.sherpa.onnx-aZXVAxMdANOR3ijL3r4eJQ==/lib/arm64/libkaldi-native-fbank-core.so (BuildId: 1cc21a4c5ee3e85b21492ccea6fe24d092b296dc) 2024-04-29 17:57:29.024 15317-15317 DEBUG pid-15317 A #04 pc 00000000000f0e98 /data/app/~~OUhfaIyYdmT8xWe_dEAYRg==/com.k2fsa.sherpa.onnx-aZXVAxMdANOR3ijL3r4eJQ==/lib/arm64/libkaldi-native-fbank-core.so (BuildId: 1cc21a4c5ee3e85b21492ccea6fe24d092b296dc) 2024-04-29 17:57:29.024 15317-15317 DEBUG pid-15317 A #05 pc 00000000000f0e0c /data/app/~~OUhfaIyYdmT8xWe_dEAYRg==/com.k2fsa.sherpa.onnx-aZXVAxMdANOR3ijL3r4eJQ==/lib/arm64/libkaldi-native-fbank-core.so (__cxa_throw+128) (BuildId: 1cc21a4c5ee3e85b21492ccea6fe24d092b296dc) 2024-04-29 17:57:29.024 15317-15317 DEBUG pid-15317 A #06 pc 0000000000089eb4 /data/app/~~OUhfaIyYdmT8xWe_dEAYRg==/com.k2fsa.sherpa.onnx-aZXVAxMdANOR3ijL3r4eJQ==/lib/arm64/libsherpa-onnx-core.so (BuildId: eca87d6119782e2333de404ccfffcdb10507d5b3) 2024-04-29 17:57:29.025 15317-15317 DEBUG pid-15317 A #07 pc 00000000001398f8 /data/app/~~OUhfaIyYdmT8xWe_dEAYRg==/com.k2fsa.sherpa.onnx-aZXVAxMdANOR3ijL3r4eJQ==/lib/arm64/libsherpa-onnx-core.so (sherpa_onnx::Lexicon::ConvertTextToTokenIdsNotChinese(std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&) const+2688) (BuildId: eca87d6119782e2333de404ccfffcdb10507d5b3) 2024-04-29 17:57:29.025 15317-15317 DEBUG pid-15317 A #08 pc 0000000000140208 /data/app/~~OUhfaIyYdmT8xWe_dEAYRg==/com.k2fsa.sherpa.onnx-aZXVAxMdANOR3ijL3r4eJQ==/lib/arm64/libsherpa-onnx-core.so (sherpa_onnx::OfflineTtsVitsImpl::Generate(std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&, long, float, std::__ndk1::function<void (float const*, int)>) const+824) (BuildId: eca87d6119782e2333de404ccfffcdb10507d5b3) 2024-04-29 17:57:29.025 15317-15317 DEBUG pid-15317 A #09 pc 000000000014712c /data/app/~~OUhfaIyYdmT8xWe_dEAYRg==/com.k2fsa.sherpa.onnx-aZXVAxMdANOR3ijL3r4eJQ==/lib/arm64/libsherpa-onnx-core.so (sherpa_onnx::OfflineTts::Generate(std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&, long, float, std::__ndk1::function<void (float const*, int)>) const+172) (BuildId: eca87d6119782e2333de404ccfffcdb10507d5b3) 2024-04-29 17:57:29.025 15317-15317 DEBUG pid-15317 A #10 pc 000000000000af7c /data/app/~~OUhfaIyYdmT8xWe_dEAYRg==/com.k2fsa.sherpa.onnx-aZXVAxMdANOR3ijL3r4eJQ==/lib/arm64/libsherpa-onnx-jni.so (Java_com_k2fsa_sherpa_onnx_OfflineTts_generateWithCallbackImpl+400) (BuildId: a150266dbc6f22a6ee3f1eaedda6bba4d72066a0) 2024-04-29 17:57:29.025 15317-15317 DEBUG pid-15317 A #13 pc 00000000000031ee [anon:dalvik-classes3.dex extracted in memory from /data/app/~~OUhfaIyYdmT8xWe_dEAYRg==/com.k2fsa.sherpa.onnx-aZXVAxMdANOR3ijL3r4eJQ==/base.apk!classes3.dex] (com.k2fsa.sherpa.onnx.OfflineTts.generateWithCallback+34) 2024-04-29 17:57:29.025 15317-15317 DEBUG pid-15317 A #15 pc 0000000000001f68 [anon:dalvik-classes3.dex extracted in memory from /data/app/~~OUhfaIyYdmT8xWe_dEAYRg==/com.k2fsa.sherpa.onnx-aZXVAxMdANOR3ijL3r4eJQ==/base.apk!classes3.dex] (com.k2fsa.sherpa.onnx.MainActivity.onClickGenerate$lambda$3+72) 2024-04-29 17:57:29.025 15317-15317 DEBUG pid-15317 A #17 pc 0000000000001a28 [anon:dalvik-classes3.dex extracted in memory from /data/app/~~OUhfaIyYdmT8xWe_dEAYRg==/com.k2fsa.sherpa.onnx-aZXVAxMdANOR3ijL3r4eJQ==/base.apk!classes3.dex] (com.k2fsa.sherpa.onnx.MainActivity.$r8$lambda$GXwRBIBrY-N067P2cW25xsk89bs+0) 2024-04-29 17:57:29.025 15317-15317 DEBUG pid-15317 A #19 pc 0000000000001880 [anon:dalvik-classes3.dex extracted in memory from /data/app/~~OUhfaIyYdmT8xWe_dEAYRg==/com.k2fsa.sherpa.onnx-aZXVAxMdANOR3ijL3r4eJQ==/base.apk!classes3.dex] (com.k2fsa.sherpa.onnx.MainActivity$$ExternalSyntheticLambda1.run+16) 2024-04-29 17:57:29.171 1372-1575 WindowManager system_server E win=Window{897ebb5 u0 com.k2fsa.sherpa.onnx/com.k2fsa.sherpa.onnx.MainActivity EXITING} destroySurfaces: appStopped=false cleanupOnResume=false win.mWindowRemovalAllowed=true win.mRemoveOnExit=true win.mViewVisibility=0 caller=com.android.server.wm.ActivityRecord.destroySurfaces:6776 com.android.server.wm.ActivityRecord.destroySurfaces:6757 com.android.server.wm.WindowState.onExitAnimationDone:6224 com.android.server.wm.ActivityRecord$$ExternalSyntheticLambda11.accept:2 java.util.ArrayList.forEach:1262 com.android.server.wm.ActivityRecord.onAnimationFinished:8902 com.android.server.wm.ActivityRecord.postApplyAnimation:6489

csukuangfj commented 2 weeks ago

Please show the output of

git diff

jeffcraighead commented 2 weeks ago

This is the diff against tag 1.9.4.

PS C:\Users\jcraighead\Documents\sherpa-onnx> git diff
diff --git a/android/SherpaOnnxTts/app/src/main/java/com/k2fsa/sherpa/onnx/MainActivity.kt b/android/SherpaOnnxTts/app/src/main/java/com/k2fsa/sherpa/onnx/MainActivity.kt
index b2e4890..198a205 100644
--- a/android/SherpaOnnxTts/app/src/main/java/com/k2fsa/sherpa/onnx/MainActivity.kt
+++ b/android/SherpaOnnxTts/app/src/main/java/com/k2fsa/sherpa/onnx/MainActivity.kt
@@ -168,10 +168,10 @@ class MainActivity : AppCompatActivity() {
         lexicon = null
         dataDir = null

-        // Example 1:
-        // modelDir = "vits-vctk"
-        // modelName = "vits-vctk.onnx"
-        // lexicon = "lexicon.txt"
+         // Example 1:
+         modelDir = "vits-vctk"
+         modelName = "vits-vctk.onnx"
+         lexicon = "lexicon.txt"
jeffcraighead commented 2 weeks ago

OK, you got me thinking about the diff and I checked the line endings after checkout and I just needed to convert tokens.txt back to Unix line endings.

csukuangfj commented 2 weeks ago

So does it work now?