Nailik / rhasspy_mobile

Rhasspy is a voice assistant software. This repository implements the functionality of a Rhasspy satellite, with local wake word recognition.
65 stars 4 forks source link

Advanced Porcupine Recording settings #384

Closed Nailik closed 1 year ago

Nailik commented 1 year ago

Add an option so users can choose whether porcupine also uses the advanced audio input settings.

This issue is a follow up to https://github.com/Nailik/rhasspy_mobile/issues/326

Use this function to process incoming data https://github.com/Picovoice/porcupine/blob/ec787ddcd69e54d6a54158125f0da5a4724e37d6/binding/android/Porcupine/porcupine/src/main/java/ai/picovoice/porcupine/Porcupine.java#L98

The only issue is, that it seems like they only support mono pcm audio with a special sample rate so it might be required to to record the audio and then process it depending on what the user has selected in the audio settings.

Here is an example how to convert stereo to audio: https://stackoverflow.com/questions/17363705/how-to-convert-audio-from-stereo-to-mono-in-android

Todos:

Nailik commented 1 year ago

After a first test it seems that porcupine needs ShortArrays of exactly 512 shorts. They use 12000Hz, AudioFormat.CHANNEL_IN_MONO, AudioFormat.ENCODING_PCM_16BIT and those values also work when using the custom recorder but not when using anotherone.

Seems like some conversion is needed:

Nailik commented 1 year ago

https://github.com/Nailik/AndroidResampler enables me to implement those changes.

However it does not support changing bit depth yet, which results in porcupine not working when setting other values than pcm16bit