Alexa with Respeaker core v1 & Respeaker-mic array v2 issue

[ikkaku2165]

• using board Respeaker Core v1 / Respeaker mic array v2

At first, I tried to run Alexa on the Respeaker Core v1 with the Respeaker mic array v1 but it hardly recognize wake word ( If I talk to the mic on Respeaker Core directly, then it sometimes recognize wake word). According to "#57 respeaker mic array issues", it seems that the mic array v1 has some bug, so I replaced it to the mic array v2, but another problem occurred(following).

root@ReSpeaker:~# alexa overrun!!! (at least 4444.227 ms long)

After that message, program did not show any response.To check where is the problem, I replaced back to mic array v1 and ran alexa command again, then alexa started after "overrun" message.

root@ReSpeaker:~# alexa overrun!!! (at least 4454.157 ms long) INFO:avs.alexa:Refreshing access_token INFO:hyper.http20.connection:Received unhandled event <RemoteSettingsChanged changed_settings:{ChangedSetting(setting=SettingCodes.MAX_CONCURRENT_STREAMS, original_value=None, new_value=100), ChangedSetting(setting=SettingCodes.INITIAL_WINDOW_SIZE, original_value=65535, new_value=2147483647), ChangedSetting(setting=SettingCodes._max_header_list_size, original_value=None, new_value=40960)}> INFO:hyper.http20.connection:Received unhandled event <WindowUpdated stream_id:0, delta:2147418112> INFO:hyper.http20.connection:Received unhandled event <SettingsAcknowledged changed_settings:{ChangedSetting(setting=SettingCodes.ENABLE_PUSH, original_value=1, new_value=0)}> INFO:hyper.http20.connection:Received unhandled event <SettingsAcknowledged changed_settings:{}> INFO:hyper.http20.connection:Received unhandled event <PriorityUpdated stream_id:1, weight:16, depends_on:0, exclusive:False> INFO:avs.alexa:wait for response INFO:hyper.http20.connection:Received unhandled event <PriorityUpdated stream_id:3, weight:16, depends_on:0, exclusive:False> INFO:avs.alexa:status code: 204 INFO:avs.alexa:on_ready

**there are so many unhandled events, but it works well(can talk back) if wake word is detected.

Do I need to change any setting to use the mic array v2? Actualy, mic array v2 always show the detected voice angle with indicater LED whereas mic array V1 not, so it seems that these two are in different mode.

[xiongyihui]

Did you change the firmware of the mic array to 1 channel firmware? See https://github.com/respeaker/usb_4_mic_array

[ikkaku2165]

Thank you for quick response! No I didn't. I 'll try again after change the firmware.

thanks,

[ikkaku2165]

Hi, I tried with 1 channel firmware, but result was exactly the same.

Also, I tried to run "respeaker_python_library/examples/offline_voice_assistant.py" to check PocketSphinx is working or not. Results were same as alexe command:

• with mic array v1: successfully started with bellow INFO but hardly recognize wake word.

  INFO:mic:Use ReSpeaker Microphone Array: USB Audio (hw:1,0)

• with mic array v2: program didn't show any response.

So it seems that Respeaker Core failed to detect mic array v2. Is there any setting item to solve this?

thanks,

[xiongyihui]

Could you try:

import pyaudio

pyaudio_instance = pyaudio.PyAudio()

for i in range(pyaudio_instance.get_device_count()):
    dev = pyaudio_instance.get_device_info_by_index(i)
    name = dev['name'].encode('utf-8')
    print(i, name, dev['maxInputChannels'], dev['maxOutputChannels'])
    if name.lower().find(b'respeaker') >= 0 and dev['maxInputChannels'] > 0:
        print('found')

Check if the mic array is found

The code to detect the mic array is at https://github.com/respeaker/respeaker_python_library/blob/master/respeaker/microphone.py#L80

[ikkaku2165]

Thanks for quick reply. I ran that code and got result below. The system seems to recognize mic array.

(0, 'MTK APSoC I2S: - (hw:0,0)', 2L, 2L)
(1, 'ReSpeaker 4 Mic Array (UAC1.0): USB Audio (hw:1,0)', 1L, 2L)
found
(2, 'sysdefault', 128L, 128L)
(3, 'default', 128L, 128L)
(4, 'dmix', 0L, 2L)

[ikkaku2165]

I found that "alexa" command is also working with mic array V2, without any command line response. It means that after overrun log, nothing was shown on CLI even if wake word was detected. (Why such thing happen...!?)

However, mic recognition is completely same as mic array V1 situation (If I say "alexa" with in 5 cm from the board, rarely success to detect about once in 5 times). After wake word was detected, Alexa's voice recognition was quite good and accurate (even 1~2 m distant from the board, command can be detected correctly). Is PocketSphinx successfully using the source from mic array?

Thanks,

[xiongyihui]

Maybe Adapting the acoustic model with the mic array will help. See https://cmusphinx.github.io/wiki/tutorialadapt/

Other option is to use snowboy instead of pocketsphinx when using Raspberry Pi

[ikkaku2165]

Thank you for information! I'll try to adapt the acoustic model.

Before your comment, I tried to modify some parameters, and recognition rate was improved.

• respeaker_python_library/respeaker/microphone.py L133: ...('-vad_threshold', 2.7) -> ('-vad_threshold', 0.7)
• respeaker_python_library/respeaker/pocketsphinx-data/keywords.txt L1: respeaker /1e-30/ -> respeaker /1e-50/

Of course this modification increase the miss-detection rate, but it works well in my environment :)

Thanks for your kind Support!!