search

k2-fsa / sherpa-onnx

Speech-to-text, text-to-speech, speaker diarization, and VAD using next-gen Kaldi with onnxruntime without Internet connection. Support embedded systems, Android, iOS, Raspberry Pi, RISC-V, x86_64 servers, websocket server/client, C/C++, Python, Kotlin, C#, Go, NodeJS, Java, Swift, Dart, JavaScript, Flutter, Object Pascal, Lazarus, Rust
https://k2-fsa.github.io/sherpa/onnx/index.html
Apache License 2.0
3.5k stars 411 forks source link

[Flutter] Failed to load silero_vad or 3dspeaker_speech_eres2net_sv_en_voxceleb_16k from assets directory #1321

Closed keyur2maru closed 1 month ago

keyur2maru commented 2 months ago

Hi,

I am trying to use Silero VAD using the Sherpa-Onnx package for Flutter. I am not able to get the assets from Flutter to the native layer. My code is derived from the pure Dart API example and it works well when running directly on the host machine. But fails when using Flutter

Logs - 

flutter: AudioChatWidget: Copied assets to local storage: /Users/keyurstudent/Library/Developer/CoreSimulator/Devices/C8AF3178-C8BC-4A99-90A9-B0963D0DEEB1/data/Containers/Data/Application/9A993521-4159-483B-A937-CD430E235584/Documents/silero_vad/model.onnx, /Users/keyurstudent/Library/Developer/CoreSimulator/Devices/C8AF3178-C8BC-4A99-90A9-B0963D0DEEB1/data/Containers/Data/Application/9A993521-4159-483B-A937-CD430E235584/Documents/3dspeaker_speech_eres2net_sv_en_voxceleb_16k/model.onnx
flutter: AudioChatWidget: Copied AI bot audio files to local storage: [/Users/keyurstudent/Library/Developer/CoreSimulator/Devices/C8AF3178-C8BC-4A99-90A9-B0963D0DEEB1/data/Containers/Data/Application/9A993521-4159-483B-A937-CD430E235584/Documents/ai_bot_sample1.wav, /Users/keyurstudent/Library/Developer/CoreSimulator/Devices/C8AF3178-C8BC-4A99-90A9-B0963D0DEEB1/data/Containers/Data/Application/9A993521-4159-483B-A937-CD430E235584/Documents/ai_bot_sample2.wav, /Users/keyurstudent/Library/Developer/CoreSimulator/Devices/C8AF3178-C8BC-4A99-90A9-B0963D0DEEB1/data/Containers/Data/Application/9A993521-4159-483B-A937-CD430E235584/Documents/ai_bot_sample3.wav]
flutter: Failed to create VoiceActivityDetector
flutter: VAD initialized
flutter: SpeakerEmbeddingExtractor: SpeakerEmbeddingExtractorConfig(model: /Users/keyurstudent/Library/Developer/CoreSimulator/Devices/C8AF3178-C8BC-4A99-90A9-B0963D0DEEB1/data/Containers/Data/Application/9A993521-4159-483B-A937-CD430E235584/Documents/3dspeaker_speech_eres2net_sv_en_voxceleb_16k/model.onnx, numThreads: 1, debug: true, provider: cpu)
flutter: SpeakerEmbeddingExtractor: SpeakerEmbeddingExtractorConfig(model: /Users/keyurstudent/Library/Developer/CoreSimulator/Devices/C8AF3178-C8BC-4A99-90A9-B0963D0DEEB1/data/Containers/Data/Application/9A993521-4159-483B-A937-CD430E235584/Documents/3dspeaker_speech_eres2net_sv_en_voxceleb_16k/model.onnx, numThreads: 1, debug: true, provider: cpu)
flutter: SpeakerEmbeddingExtractor: ptr: Pointer: address=0x0
Lost connection to device.
csukuangfj commented 2 months ago

please have a.look at our flutter asr example.

We have code to process the assets there.

keyur2maru commented 1 month ago

Thank you, that helped resolve the issue!