search

openassistant / oa-core

Open Assistant Core
http://openassistant.org
GNU General Public License v3.0
291 stars 77 forks source link

no response for voice commands #46

Closed HeavyMoon closed 3 years ago

HeavyMoon commented 3 years ago

Describe the bug i'm trying oa on raspberry pi. no response for voice command 'boot mind'.

To Reproduce Steps to reproduce the behavior:

  1. start oa
  2. say 'boot mind' but no response.

Expected behavior sounds R2D2.

Additional context voice and quit looks working. but voice commands are no response.

pi@assistant:~/oa-core $ cat /etc/os-release 
PRETTY_NAME="Raspbian GNU/Linux 10 (buster)"
NAME="Raspbian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"
pi@assistant:~/oa-core $ uname -a
Linux assistant 5.10.17-v7+ #1403 SMP Mon Feb 22 11:29:51 GMT 2021 armv7l GNU/Linux
pi@assistant:~/oa-core $ python3 -m oa -d 
[2021-05-03 16:23:51,930] INFO MainThread __main__: Start Open Assistant
[2021-05-03 16:23:51,955] INFO MainThread oa.core.hub: Loading Modules <- /home/pi/oa-core/oa/modules
[2021-05-03 16:23:51,956] INFO MainThread oa.core.util: mind <- /home/pi/oa-core/oa/modules/mind
/usr/lib/python3/dist-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.4) or chardet (3.0.4) doesn't match a supported version!
  RequestsDependencyWarning)
[2021-05-03 16:23:52,672] INFO MainThread oa.core.util: ear <- /home/pi/oa-core/oa/modules/ear
[2021-05-03 16:23:53,588] INFO MainThread oa.core.util: speech_recognition <- /home/pi/oa-core/oa/modules/speech_recognition
[2021-05-03 16:23:53,655] INFO MainThread oa.core.util: sound <- /home/pi/oa-core/oa/modules/sound
[2021-05-03 16:23:53,660] INFO MainThread oa.core.util: voice <- /home/pi/oa-core/oa/modules/voice
[2021-05-03 16:23:53,673] DEBUG mind oa.core.hub: Starting
[2021-05-03 16:23:53,674] DEBUG ear oa.core.hub: Starting
[2021-05-03 16:23:53,676] DEBUG speech_recognition oa.core.hub: Starting
[2021-05-03 16:23:53,677] DEBUG sound oa.core.hub: Starting
[2021-05-03 16:23:53,679] DEBUG voice oa.core.hub: Starting
OA> [2021-05-03 16:23:53,684] INFO mind oa.modules.mind: Loading minds...
[2021-05-03 16:23:53,686] INFO mind oa.modules.mind: <- root.py
[2021-05-03 16:23:54,055] INFO mind oa.modules.mind: <- boot.py
[2021-05-03 16:23:54,059] INFO mind oa.modules.mind: Minds loaded!
[2021-05-03 16:23:54,059] INFO mind oa.modules.mind: Opening Mind: boot
[2021-05-03 16:23:54,060] DEBUG mind oa.modules.mind: "boot" is now listening. Say "Boot Mind!" to see if it can hear you.

Unrecognized command: 
OA> voice hello world
[2021-05-03 16:24:13,415] DEBUG MainThread oa.util.repl: voice <- hello world
OA> [2021-05-03 16:24:13,418] DEBUG voice oa.modules.voice: Saying: hello world
[2021-05-03 16:24:13,421] DEBUG speech_recognition oa.modules.speech_recognition: Muted
[2021-05-03 16:24:14,517] DEBUG speech_recognition oa.modules.speech_recognition: Unmuted

Unrecognized command: 
OA> quit
[2021-05-03 16:24:19,131] ERROR voice oa.core.hub: Error processing queue: Open Assistant closed.
[2021-05-03 16:24:19,134] DEBUG voice oa.core.hub: Stopped
[2021-05-03 16:24:19,135] ERROR speech_recognition oa.core.hub: Error processing queue: Open Assistant closed.
[2021-05-03 16:24:19,137] DEBUG speech_recognition oa.core.hub: Stopped
[2021-05-03 16:24:19,139] ERROR mind oa.core.hub: Error processing queue: Open Assistant closed.
[2021-05-03 16:24:19,140] DEBUG mind oa.core.hub: Stopped
[2021-05-03 16:24:19,165] ERROR sound oa.core.hub: Error processing queue: Open Assistant closed.
[2021-05-03 16:24:19,166] DEBUG sound oa.core.hub: Stopped
[2021-05-03 16:24:19,184] DEBUG ear oa.core.hub: ear -> speech_recognition
[2021-05-03 16:24:19,190] DEBUG ear oa.core.hub: Stopped

When i exit with q command, speech_recognition message appear sometimes.

...
[2021-05-03 16:40:07,686] INFO mind oa.modules.mind: Opening Mind: boot
[2021-05-03 16:40:07,687] DEBUG mind oa.modules.mind: "boot" is now listening. Say "Boot Mind!" to see if it can hear you.
q
[2021-05-03 16:40:39,142] DEBUG ear oa.core.hub: ear -> speech_recognition
[2021-05-03 16:40:39,147] DEBUG ear oa.core.hub: Stopped
[2021-05-03 16:40:39,186] ERROR mind oa.core.hub: Error processing queue: Open Assistant closed.
[2021-05-03 16:40:39,188] DEBUG mind oa.core.hub: Stopped
[2021-05-03 16:40:39,194] ERROR voice oa.core.hub: Error processing queue: Open Assistant closed.
[2021-05-03 16:40:39,195] ERROR sound oa.core.hub: Error processing queue: Open Assistant closed.
[2021-05-03 16:40:39,198] DEBUG sound oa.core.hub: Stopped
[2021-05-03 16:40:39,197] DEBUG voice oa.core.hub: Stopped
[2021-05-03 16:40:53,518] INFO speech_recognition oa.modules.speech_recognition: Heard: BOOT MIND BOOT BOOT MIND STOP OPEN ASSISTANT MIND OPEN ASSISTANT BOOT MIND BOOT MIND OPEN BOOT BOOT MIND
[2021-05-03 16:40:53,520] DEBUG speech_recognition oa.core.hub: Stopped

what can i do next? does anyone try on raspberry pi and work it fine?

joshuashort commented 3 years ago

It could be an audio problem; it's encouraging that you get the Heard: log entry sometimes.

That said, it's probably something about ear and your microphone.. if the stream is noisy enough, it might not stop listening.

robert478 commented 3 years ago

HeavyMoon, Jshort helped me today. I had the same problems as yourself. The problem lies in oa/modules/ear/init.py,

Had to change the timeout from none to 6.5 on my machine to get it to work, Its worth a try. It worked for me

HeavyMoon commented 3 years ago

thank you for comments. by changing timeout of ear, voice command 'boot mine' is working without R2D2 sound. and other voice commands not detect before voice command. 

pi@assistant:~/oa-core $ cat oa/modules/ear/__init__.py | grep \"timeout\":
    #"timeout": None,
    "timeout": 6.0,
pi@assistant:~/oa-core $ python3 -m oa -d 
[2021-05-17 21:45:42,660] INFO MainThread __main__: Start Open Assistant
[2021-05-17 21:45:42,684] INFO MainThread oa.core.hub: Loading Modules <- /home/pi/oa-core/oa/modules
[2021-05-17 21:45:42,686] INFO MainThread oa.core.util: mind <- /home/pi/oa-core/oa/modules/mind
/usr/lib/python3/dist-packages/requests/__init__.py:91: RequestsDependencyWarning: urllib3 (1.26.4) or chardet (3.0.4) doesn't match a supported version!
  RequestsDependencyWarning)
[2021-05-17 21:45:43,406] INFO MainThread oa.core.util: ear <- /home/pi/oa-core/oa/modules/ear
[2021-05-17 21:45:44,327] INFO MainThread oa.core.util: speech_recognition <- /home/pi/oa-core/oa/modules/speech_recognition
[2021-05-17 21:45:44,394] INFO MainThread oa.core.util: sound <- /home/pi/oa-core/oa/modules/sound
[2021-05-17 21:45:44,400] INFO MainThread oa.core.util: voice <- /home/pi/oa-core/oa/modules/voice
[2021-05-17 21:45:44,412] DEBUG mind oa.core.hub: Starting
[2021-05-17 21:45:44,414] DEBUG ear oa.core.hub: Starting
[2021-05-17 21:45:44,416] DEBUG speech_recognition oa.core.hub: Starting
[2021-05-17 21:45:44,417] DEBUG sound oa.core.hub: Starting
[2021-05-17 21:45:44,419] DEBUG voice oa.core.hub: Starting
OA> [2021-05-17 21:45:44,424] INFO mind oa.modules.mind: Loading minds...
[2021-05-17 21:45:44,425] INFO mind oa.modules.mind: <- root.py
[2021-05-17 21:45:44,790] INFO mind oa.modules.mind: <- boot.py
[2021-05-17 21:45:44,793] INFO mind oa.modules.mind: Minds loaded!
[2021-05-17 21:45:44,794] INFO mind oa.modules.mind: Opening Mind: boot
[2021-05-17 21:45:44,795] DEBUG mind oa.modules.mind: "boot" is now listening. Say "Boot Mind!" to see if it can hear you.
[2021-05-17 21:45:50,463] DEBUG ear oa.core.hub: ear -> speech_recognition
[2021-05-17 21:45:51,916] INFO speech_recognition oa.modules.speech_recognition: Heard: BOOT MIND
[2021-05-17 21:45:51,917] DEBUG speech_recognition oa.core.hub: speech_recognition -> mind
[2021-05-17 21:45:51,919] DEBUG mind oa.modules.mind: Input: BOOT MIND
[2021-05-17 21:45:51,924] DEBUG speech_recognition oa.modules.speech_recognition: Muted
[2021-05-17 21:45:56,480] DEBUG ear oa.core.hub: ear -> speech_recognition
[2021-05-17 21:46:02,495] DEBUG ear oa.core.hub: ear -> speech_recognition
[2021-05-17 21:46:08,510] DEBUG ear oa.core.hub: ear -> speech_recognition
[2021-05-17 21:46:14,526] DEBUG ear oa.core.hub: ear -> speech_recognition
[2021-05-17 21:46:20,542] DEBUG ear oa.core.hub: ear -> speech_recognition
voice a
[2021-05-17 21:46:22,772] DEBUG MainThread oa.util.repl: voice <- a
OA> [2021-05-17 21:46:22,774] DEBUG voice oa.modules.voice: Saying: a
[2021-05-17 21:46:22,776] DEBUG speech_recognition oa.modules.speech_recognition: Muted
[2021-05-17 21:46:23,355] DEBUG speech_recognition oa.modules.speech_recognition: Unmuted
[2021-05-17 21:46:26,559] DEBUG ear oa.core.hub: ear -> speech_recognition
[2021-05-17 21:46:27,601] INFO speech_recognition oa.modules.speech_recognition: Heard: HELP LIST
[2021-05-17 21:46:32,576] DEBUG ear oa.core.hub: ear -> speech_recognition
[2021-05-17 21:46:33,849] INFO speech_recognition oa.modules.speech_recognition: Heard: MIND
[2021-05-17 21:46:38,590] DEBUG ear oa.core.hub: ear -> speech_recognition
[2021-05-17 21:46:40,975] INFO speech_recognition oa.modules.speech_recognition: Heard: OPEN ASSISTANT
[2021-05-17 21:46:40,976] DEBUG speech_recognition oa.core.hub: speech_recognition -> mind
[2021-05-17 21:46:40,977] DEBUG mind oa.modules.mind: Input: OPEN ASSISTANT
[2021-05-17 21:46:40,979] INFO mind oa.modules.mind: Opening Mind: root
[2021-05-17 21:46:44,606] DEBUG ear oa.core.hub: ear -> speech_recognition
[2021-05-17 21:46:46,850] INFO speech_recognition oa.modules.speech_recognition: Heard: LIST COMMANDS
[2021-05-17 21:46:46,850] DEBUG speech_recognition oa.core.hub: speech_recognition -> mind
[2021-05-17 21:46:46,852] DEBUG mind oa.modules.mind: Input: LIST COMMANDS
[2021-05-17 21:46:46,853] DEBUG voice oa.modules.voice: Saying: - The currently available voice commands are:
ROOT MIND,
CLOSE ASSISTANT,
LIST COMMANDS,
WHAT CAN I SAY,
READ WORLD NEWS,
RUN DIAGNOSTICS,
SING A SONG,
WHAT DAY IS IT,
WHAT DID I SAY,
WHAT IS THE WEATHER,
WHAT TIME IS IT

Uhmm...

HeavyMoon commented 3 years ago

The first issue was solved by setting the timeout. And the lack of wav sounds seems to be a problem with the playsound library.

Mute after the "boot mind" command has been improved by adding a say function. 

@command("boot mind")
def response_sound():
  say('start boot mind')  ## add
  play('r2d2.wav')

thank you for your great help.