search

OpenVoiceOS / raspOVOS

Run ovos ontop of RaspberryPiOS
MIT License
36 stars 13 forks source link

OVOS is detecting my speech but no audio is playing #101

Closed Bulwagga closed 7 months ago

Bulwagga commented 11 months ago

@austin-b, can you post information from your audio.log and voice.log files? They are likely in $HOME/.local/state/mycroft. What happens when you enter

ovos-speak "hello"

Thanks

austin-b commented 11 months ago 
$ ovos-speak "hello"
2023-11-05 10:09:52.485 - OVOS - ovos_bus_client.session:from_message:447 - WARNING - No session context in message:connected

and in audio.log after my recent restart of all the services:

2023-11-05 10:05:55.241 - audio - ovos_audio.service:__init__:58 - INFO - Starting Audio Service
2023-11-05 10:05:55.242 - audio - ovos_audio.service:on_started:40 - INFO - Audio service started.
2023-11-05 10:05:55.258 - audio - ovos_bus_client.session:from_message:447 - WARNING - No session context in message:connected
2023-11-05 10:05:55.500 - audio - ovos_audio.service:_maybe_reload_tts:317 - INFO - (re)loading TTS engine
2023-11-05 10:05:55.516 - audio - ovos_audio.playback:run:194 - INFO - PlaybackThread started
2023-11-05 10:05:55.692 - audio - ovos_utils.configuration:get_xdg_base - WARNING - Deprecation version=0.1.0. Caller=ovos_tts_plugin_mimic:93. configuration moved to the `ovos_config` package.
2023-11-05 10:05:55.725 - audio - ovos_utils.configuration:read_mycroft_config - WARNING - Deprecation version=0.1.0. Caller=ovos_tts_plugin_mimic:106. configuration moved to the `ovos_config` package.
2023-11-05 10:05:55.726 - audio - ovos_plugin_manager.tts:create:197 - INFO - Found plugin ovos-tts-plugin-server
2023-11-05 10:05:55.767 - audio - mycroft:<module>:37 - WARNING - mycroft has been deprecated! please start importing from ovos_core and companion packages
mycroft module remains available for backwards compatibility and will be removed in version 0.2.0
2023-11-05 10:05:56.835 - audio - ovos_plugin_manager.tts:create:200 - INFO - Loaded plugin ovos-tts-plugin-server
2023-11-05 10:05:56.836 - audio - ovos_audio.service:_maybe_reload_tts:336 - INFO - (re)loading fallback TTS engine
2023-11-05 10:05:56.863 - audio - ovos_plugin_manager.tts:create:197 - INFO - Found plugin 
2023-11-05 10:05:57.057 - audio - ovos_plugin_manager.tts:create:200 - INFO - Loaded plugin 
2023-11-05 10:05:57.078 - audio - ovos_audio.audio:load_services:95 - INFO - Loading audio service plugin: ovos_audio_simple
2023-11-05 10:05:57.079 - audio - ovos_audio.audio:load_services:95 - INFO - Loading audio service plugin: ovos_common_play
2023-11-05 10:06:02.347 - audio - ovos_audio.audio:find_ocp:62 - INFO - OCP - OVOS Common Play set as default backend
2023-11-05 10:06:02.349 - audio - ovos_audio.service:on_alive:36 - INFO - Audio service is alive.
2023-11-05 10:06:06.030 - audio - ovos_plugin_common_play.ocp:register_ocp_intents:120 - INFO - OCP intents missing, registering for <ovos_plugin_common_play.ocp.OCP object at 0x7fa573c8aef0>
2023-11-05 10:06:23.341 - audio - ovos_audio.service:execute_tts:348 - INFO - Speak: ten oh six
2023-11-05 10:09:52.492 - audio - ovos_audio.service:execute_tts:348 - INFO - Speak: hello

The "ten oh six" is a response to "what time is it":

2023-11-05 10:05:54.632 - voice - ovos_dinkum_listener.service:__init__:125 - INFO - Starting Voice Service
2023-11-05 10:05:54.679 - voice - ovos_dinkum_listener.service:on_alive:60 - INFO - DinkumVoiceService is alive.
2023-11-05 10:05:54.683 - voice - ovos_dinkum_listener.service:_before_start:314 - INFO - Starting service...
2023-11-05 10:05:54.695 - voice - ovos_bus_client.session:from_message:447 - WARNING - No session context in message:connected
2023-11-05 10:05:54.847 - voice - ovos_dinkum_listener.service:_connect_to_bus:413 - INFO - Connected to Mycroft Core message bus
2023-11-05 10:05:55.407 - voice - ovos_dinkum_listener.service:on_alive:60 - INFO - DinkumVoiceService is alive.
2023-11-05 10:05:55.410 - voice - ovos_dinkum_listener.voice_loop.hotwords:load_hotword_engines:108 - INFO - creating hotword engines
2023-11-05 10:05:55.438 - voice - ovos_plugin_manager.wakewords:load_module:165 - INFO - Loading "hey_mycroft" wake word via ovos-ww-plugin-precise-lite with config: {'module': 'ovos-ww-plugin-precise-lite', 'model': 'https://github.com/OpenVoiceOS/precise-lite-models/raw/master/wakewords/en/hey_mycroft.tflite', 'expected_duration': 3, 'trigger_level': 3, 'sensitivity': 0.5, 'listen': True, 'fallback_ww': 'hey_mycroft_vosk', 'sound': 'snd/start_listening.wav'}
2023-11-05 10:05:55.756 - voice - ovos_plugin_manager.wakewords:load_module:172 - INFO - Loaded the Wake Word hey_mycroft with module ovos-ww-plugin-precise-lite
2023-11-05 10:05:55.760 - voice - ovos_dinkum_listener.voice_loop.hotwords:load_hotword_engines:158 - INFO - Loading hotword: hey_mycroft with engine: <ovos_ww_plugin_precise_lite.PreciseLiteHotwordPlugin object at 0x7f730ef0ae90>
2023-11-05 10:05:55.762 - voice - ovos_plugin_manager.wakewords:load_module:165 - INFO - Loading "wake_up" wake word via ovos-ww-plugin-pocketsphinx with config: {'module': 'ovos-ww-plugin-pocketsphinx', 'phonemes': 'W EY K . AH P', 'threshold': 1e-20, 'lang': 'en-us', 'wakeup': True}
2023-11-05 10:05:55.798 - voice - ovos_plugin_manager.wakewords:load_module:172 - INFO - Loaded the Wake Word wake_up with module ovos-ww-plugin-pocketsphinx
2023-11-05 10:05:55.843 - voice - ovos_dinkum_listener.voice_loop.hotwords:load_hotword_engines:158 - INFO - Loading hotword: wake_up with engine: <ovos_ww_plugin_pocketsphinx.PocketsphinxHotWordPlugin object at 0x7f730ef0aa10>
2023-11-05 10:05:55.844 - voice - ovos_dinkum_listener.voice_loop.hotwords:load_hotword_engines:191 - WARNING - No stop words loaded
2023-11-05 10:05:56.618 - voice - ovos_plugin_manager.ocp:load:54 - INFO - Loaded OCP plugin: ovos-ocp-news-plugin
2023-11-05 10:05:56.831 - voice - ovos_plugin_manager.ocp:load:54 - INFO - Loaded OCP plugin: ovos-ocp-m3u-plugin
2023-11-05 10:05:57.085 - voice - ovos_plugin_manager.ocp:load:54 - INFO - Loaded OCP plugin: ovos-ocp-files-plugin
2023-11-05 10:05:57.298 - voice - ovos_plugin_manager.ocp:load:54 - INFO - Loaded OCP plugin: ovos-ocp-rss-plugin
2023-11-05 10:05:57.343 - voice - ovos_dinkum_listener.voice_loop.voice_loop:start:169 - INFO - Listening mode: wakeword
2023-11-05 10:05:57.344 - voice - ovos_dinkum_listener.service:on_started:64 - INFO - DinkumVoiceService started.
2023-11-05 10:05:57.345 - voice - ovos_dinkum_listener.service:on_started:64 - INFO - DinkumVoiceService started.
2023-11-05 10:05:57.346 - voice - ovos_dinkum_listener.service:run:289 - INFO - Service ready
2023-11-05 10:05:57.347 - voice - ovos_dinkum_listener.voice_loop.voice_loop:run:194 - INFO - Starting loop in mode: wakeword
2023-11-05 10:06:13.682 - voice - ovos_dinkum_listener.voice_loop.voice_loop:run:222 - INFO - Wakeword detected
2023-11-05 10:06:16.975 - voice - ovos_dinkum_listener.voice_loop.voice_loop:run:252 - INFO - speech finished
2023-11-05 10:06:22.094 - voice - ovos_dinkum_listener.voice_loop.voice_loop:_after_cmd:669 - INFO - transcribed: "What time is it?"

I do not hear any listening sound/beep as I would expect. Volume is turned up and other things play with no issue.

builderjer commented 11 months ago

@austin-b what hardware are you using for a speaker? Is it listed with the command pactl list sinks

Also, try alsamixer and see if you can find and activate it there

austin-b commented 11 months ago

pactl list sinks gives 

Sink #1
    State: IDLE
    Name: alsa_output.pci-0000_00_1f.3.analog-stereo
    Description: Built-in Audio Analog Stereo
    Driver: module-alsa-card.c
    Sample Specification: s16le 2ch 44100Hz
    Channel Map: front-left,front-right
    Owner Module: 22
    Mute: no
    Volume: front-left: 65536 / 100% / 0.00 dB,   front-right: 65536 / 100% / 0.00 dB
            balance 0.00
    Base Volume: 65536 / 100% / 0.00 dB
    Monitor Source: alsa_output.pci-0000_00_1f.3.analog-stereo.monitor
    Latency: 40207 usec, configured 40000 usec
    Flags: HARDWARE HW_MUTE_CTRL HW_VOLUME_CTRL DECIBEL_VOLUME LATENCY 
    Properties:
        alsa.resolution_bits = "16"
        device.api = "alsa"
        device.class = "sound"
        alsa.class = "generic"
        alsa.subclass = "generic-mix"
        alsa.name = "ALC255 Analog"
        alsa.id = "ALC255 Analog"
        alsa.subdevice = "0"
        alsa.subdevice_name = "subdevice #0"
        alsa.device = "0"
        alsa.card = "0"
        alsa.card_name = "HDA Intel PCH"
        alsa.long_card_name = "HDA Intel PCH at 0xb1220000 irq 129"
        alsa.driver_name = "snd_hda_intel"
        device.bus_path = "pci-0000:00:1f.3"
        sysfs.path = "/devices/pci0000:00/0000:00:1f.3/sound/card0"
        device.bus = "pci"
        device.vendor.id = "8086"
        device.vendor.name = "Intel Corporation"
        device.product.id = "9d71"
        device.product.name = "Sunrise Point-LP HD Audio"
        device.form_factor = "internal"
        device.string = "front:0"
        device.buffering.buffer_size = "352800"
        device.buffering.fragment_size = "176400"
        device.access_mode = "mmap+timer"
        device.profile.name = "analog-stereo"
        device.profile.description = "Analog Stereo"
        device.description = "Built-in Audio Analog Stereo"
        module-udev-detect.discovered = "1"
        device.icon_name = "audio-card-pci"
    Ports:
        analog-output-speaker: Speakers (type: Speaker, priority: 10000, availability unknown)
        analog-output-headphones: Headphones (type: Headphones, priority: 9900, availability group: Legacy 2, not available)
    Active Port: analog-output-speaker
    Formats:
        pcm

alsamixer looks like so:

┌──────────────────────────────────────────────────────────── AlsaMixer v1.2.6 ────────────────────────────────────────────────────────────┐
│ Card: HDA Intel PCH                                                                                              F1:  Help               │
│ Chip: Realtek ALC255                                                                                             F2:  System information │
│ View: F3:[Playback] F4: Capture  F5: All                                                                         F6:  Select sound card  │
│ Item: Master [dB gain: 0.00]                                                                                     Esc: Exit               │
│                                                                                                                                          │
│                                                                                                                                          │
│                                                                                                                                          │
│      ┌──┐      ┌──┐      ┌──┐      ┌──┐                                    ┌──┐                ┌──┐      ┌──┐      ┌──┐                  │
│      │▒▒│      │  │      │▒▒│      │▒▒│                                    │  │                │  │      │  │      │  │                  │
│      │▒▒│      │  │      │▒▒│      │▒▒│                                    │  │                │  │      │  │      │  │                  │
│      │▒▒│      │  │      │▒▒│      │▒▒│                                    │  │                │  │      │  │      │  │                  │
│      │▒▒│      │  │      │▒▒│      │▒▒│                                    │  │                │  │      │  │      │  │                  │
│      │▒▒│      │  │      │▒▒│      │▒▒│                                    │  │                │  │      │  │      │  │                  │
│      │▒▒│      │  │      │▒▒│      │▒▒│                                    │  │                │  │      │  │      │  │                  │
│      │▒▒│      │  │      │▒▒│      │▒▒│                                    │  │                │  │      │  │      │  │                  │
│      │▒▒│      │  │      │▒▒│      │▒▒│                                    │  │                │  │      │  │      │▒▒│                  │
│      │▒▒│      │  │      │▒▒│      │▒▒│                                    │  │                │  │      │  │      │▒▒│                  │
│      │▒▒│      │  │      │▒▒│      │▒▒│                                    │▒▒│                │  │      │  │      │▒▒│                  │
│      │▒▒│      │  │      │▒▒│      │▒▒│                                    │▒▒│                │  │      │  │      │▒▒│                  │
│      │▒▒│      │  │      │▒▒│      │▒▒│                                    │▒▒│                │  │      │  │      │▒▒│                  │
│      │▒▒│      │  │      │▒▒│      │▒▒│                                    │▒▒│                │  │      │  │      │▒▒│                  │
│      │▒▒│      │  │      │▒▒│      │▒▒│                                    │▒▒│                │  │      │  │      │▒▒│                  │
│      │▒▒│      │  │      │▒▒│      │▒▒│                                    │▒▒│                │  │      │  │      │▒▒│                  │
│      ├──┤      ├──┤      ├──┤      └──┘      ┌──┐      ┌──┐      ┌──┐      ├──┤    Disabled    ├──┤      └──┘      └──┘    Disabled      │
│      │OO│      │MM│      │OO│                │MM│      │OO│      │OO│      │MM│                │MM│                                      │
│      └──┘      └──┘      └──┘                └──┘      └──┘      └──┘      └──┘                └──┘                                      │
│      100       0<>0    100<>100  100<>100                                 37<>37               0<>0      0<>0     53<>53                 │
│   < Master  >Headphone  Speaker     PCM     S/PDIF   S/PDIF 1  S/PDIF 2    Beep    Auto-Mute Headset M Headset M Internal  Loopback      │
│                                                                                                                                          │
│                                                                                                                                          │
│                                                                                                                                          │
└──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
builderjer commented 11 months ago

The only thing I see is the onboard audio. What speaker are you using?

austin-b commented 11 months ago

That's all I'm working with at the moment, my laptop's onboard audio. I will find a speaker or headphones to test with and see if there's any difference.

builderjer commented 11 months ago

You are using a x86. How did you install OVOS? This repo is optimized for a Rpi, so I am curious what you did.

austin-b commented 11 months ago

I cloned the repo and ran manual_user_install.sh. I had a few issues to clear up but those were mainly related to getting the services running.

builderjer commented 11 months ago

That is good to know that it can be installed on an x86 with that script. I would like to make this script work with an option for x86 and for OS's other than Ubuntu. It should be able to autodetect hardware and run the correct setup for each.

Bulwagga commented 11 months ago

I just installed using the newest manual_user_install.sh on my HP Intel laptop. ovos-admin-phal.service didn't start, but everything else did. ovos-speak "hello" gives me a hello with some deprecation messages. Still looking for the log files, but in a fuzzy sense it's working for me too.

Bulwagga commented 11 months ago

Logs were in the customary spot, I was chasing a command typo. All I needed to do was turn up my microphone volume and OVOS works as expected.

Ovos-speak gives me the same "WARNING - No session context in message:connected" output but does vocalize "hello".

If it helps, I am running Ubuntu 23.10, HDA Intel PCH, and the Driver listed in pactl list sinks is Pipewire.

Bulwagga commented 11 months ago

Both of these have been helpful to me in the past: https://openvoiceos.github.io/community-docs/011-hardware_audio/ https://mycroft-ai.gitbook.io/docs/using-mycroft-ai/troubleshooting/audio-troubleshooting

Bulwagga commented 11 months ago

On my Raspberry Pi 3B+ the pactl list sinks driver is module-alsa-card.c and the sound is working.

Bulwagga commented 11 months ago

Another place to look for odd output would be in /var/log/syslog.

austin-b commented 11 months ago

Thank you for the syslog recommendation, I have came across this which I'm trying to decipher right now that I'm hoping is part of my issue:

Nov  5 16:15:13 tommyboi-Swift-SF314-51 pipewire-media-session[9033]: ms.mod.policy-node: 0x55d3ad799950: no node found for 38, stream error
Nov  5 16:15:13 tommyboi-Swift-SF314-51 python3[9188]: remote error: id=3 seq:7 res:-2 (No such file or directory): no node available
Nov  5 16:15:13 tommyboi-Swift-SF314-51 python3[9188]: stream node 38 error: no node available
Bulwagga commented 11 months ago

For the past few weeks I have been having problems with OVOS occasionally dropping syllables, and this problem report talks about 'stuttering' as well as the ms.mod.policy-mode problem. https://bbs.archlinux.org/viewtopic.php?id=270610

Bulwagga commented 11 months ago

I see at https://github.com/QubesOS/qubes-issues/issues/8193 that pipewire is supposed to be a replacement for pulseaudio. So I tried deleting pulseaudio and pulseaudio-utils from my pc install (Ubuntu 13.10 - pipewire) and OVOS is now correctly pronouncing the words. I notice that Ubuntu has a process running called pipewire-pulse. You may be seeing competition between PW and PA. From the OVOS point of view I think you can try removing pulseaudio.

The intermittent speech problems have happened since I started running Debian bookworm on my Pi, and I know that bookworm has also switched to pipewire. I tried removing pulseaudio from my Pi, but there is no equivalent to pipewire-pulse, and the dinkum-listener failed after starting. So I've removed pipewire from my Pi and the pronunciation is correct. Let's see how long that lasts!

Bulwagga commented 11 months ago

About 15 minutes. Hope you have better luck.

austin-b commented 11 months ago

I tried a couple of things and have seemed to completely break my audio, no sound and no input or output devices are found at the moment. I'm not exactly sure what broke it so I will try again on a fresh Ubuntu install and will reinstall OVOS.

builderjer commented 9 months ago

I think that the bug with the strange voices has been fixed, and AFAK, USB devices are working pretty well with the new images. I am still working on i2c devices, eg respeaker devices.

please try out the new images and see if your issue is fixed

https://ovosimages.ziggyai.online/raspbian/newest

Bulwagga commented 9 months ago

I haven't been able to get the pulseaudio input to work for my RPi 4 and a plantronics calypso mic/speaker. I tried all of the pactl sources with no success. I was also unable to set the alsamixer input/output device (I think because I was logged into the venv. Is pactl the preferred tool? Should I be using pulseaudio? Thanks Good news is that garbled speech is gone, tested with ovos-say-to. All good now.

builderjer commented 9 months ago

Garbled speech was a bug with another package caching wrong, yes hope it was fixed.

Is your device usb? Or i2c? pactl is part of pulseaudio, so yes, that is preferred. Bigger question, if a usb device why won't it work.

Bulwagga commented 9 months ago

Yes, it is a USB device that worked without modification a week or so ago on a 3B install done with the script.

journalctl | grep -i pulse Last entry is: ... voice - ovos_microphone_plugin_sounddevice:find_input_device:42 - INFO - Searching for input device: pulse

$ pactl get-default-source OpenVoiceOS.monitor $ pactl get-default-sink OpenVoiceOS

ovos-say-to produces correct results Microphone looks like it's not recording: "arecord test.wav" records nothing, "hey mycroft" was not recognized as noted in journalctl. PulseAudio capture volume is up all the way.

Update: I was able to use "arecord" with one of the pactl sources, and I did the playback when the sink was OpenVoiceOS. None of the pactl sources was successful in responding to "hey mycroft".

builderjer commented 7 months ago

Please try the latest image. There have been some audio fixes

Bulwagga commented 7 months ago

Will do. I'm traveling now. Look for something by next weekend.

Bulwagga commented 7 months ago

Fixed.