search

MycroftAI / mycroft-core

Mycroft Core, the Mycroft Artificial Intelligence platform.
https://mycroft.ai
Apache License 2.0
6.53k stars 1.28k forks source link

VolumeSkill mutes microphone after wake-word is recognized #2884

Closed poohsen closed 3 years ago

poohsen commented 3 years ago

Describe the bug In my installation, in 99% of the cases, something makes the VolumeSkill mute my microphone once the wake-word is recognized and recording begins. The only way around it at present is to blacklist the VolumeSkill.

To Reproduce Steps to reproduce the behavior:

  1. Fire up mycroft with debug console
  2. say the wake-word
  3. observe the muting in the logs and the abysmal recognition rate of the STT engine

Expected behavior No muting by the VolumeSkill

Log files Once loglevel is set to DEBUG, I can see this in the logs:

18:43:18.218 | INFO | 3800 | main:handle_record_begin:41 | Begin Recording…
18:43:18.245 | DEBUG | 3794 | VolumeSkill | MUTING!
18:43:18.245 | DEBUG | 3794 | VolumeSkill | Finding Alsa Mixer for control…
18:43:18.284 | DEBUG | 3794 | VolumeSkill | Volume before mute: 100
18:43:18.284 | DEBUG | 3794 | VolumeSkill | 100
18:43:18.284 | DEBUG | 3794 | VolumeSkill | 0
18:43:21.407 | DEBUG | 3794 | VolumeSkill | Finding Alsa Mixer for control…
18:43:21.469 | DEBUG | 3794 | VolumeSkill | 100

(after 3s it finished recording and unmuted the mic again)

Environment (please complete the following information):

Additional context

This might be some setup problem as I’ve never paired my installation, but then again I played with the two most obvious settings that could affect this, “mute_during_output” and “duck_while_listening” and they had no effect here.

My user config:

{
  "max_allowed_core_version": 20.8,
  "time_format": "full",
  "date_format": "DMY",
  "location": {
    "city": {
      "code": "Amsterdam",
      "name": "Amsterdam",
      "state": {
        "code": "NH",
        "name": "Noord-Holland",
        "country": {
          "code": "NL",
          "name": "Netherlands"
        }
      }
    },
    "coordinate": {
      "latitude": 38.971669,
      "longitude": -95.23525
    },
    "timezone": {
      "code": "Europe/Amsterdam",
      "name": "Central European Summer Time",
      "dstOffset": 3600000,
      "offset":    3600000
    }
  },

  "listener": {
    "device_name": "Intel 82801AA-ICH",
    "wake_word": "hey marvin",
    "mute_during_output" : false,
    "duck_while_listening" : 0.3
  },
  "hotwords": {
    "hey marvin": {
      "module": "precise",
      "local_model_file": "/home/osboxes/.mycroft/precise/marvin.pb",
      "sensitivity": 0.8,
      "trigger_level": 3 
    }
  },
  "skills": {
    "blacklisted_skills": [
      "mycroft-configuration.mycroftai",
      "mycroft-weather.mycroftai",
      "mycroft-pairing.mycroftai"
    ],
  "priority_skills": ["mycroft-volume"]
  },
  "stt": {
    "deepspeech_server": {
      "uri": "http://192.168.2.25:8080/stt"
    },
    "module": "deepspeech_server"
  },
  "user": "",
  "log_level": "DEBUG"
}
forslund commented 3 years ago

That debug text is from the part that is intended to mute output. during listening. Do you think the case is that the volume skill doesn't find the master output mixer, instead finds the input mixer and mutes that?

Your config looks fine as far as I can tell.

The mute_during_output will affect if the mic is muted while Mycroft is talking (this is done internally in the speech client not using alsa), duck_while_listening was added to decide how much the output volume should be changed while recording STT...but I'm not sure it's used at all...(volume skill just triggers the mute, maybe the Mark-2 enclosure handles it separately in the future)

poohsen commented 3 years ago

doesn't find the master output mixer, instead finds the input mixer and mutes that?

maybe... I'll be deploying mycroft an a Raspberry Pi 4 soon. Maybe the sound setup will be slightly different and the issue will disappear

ChanceNCounter commented 3 years ago

Seems like that almost has to be it. Has anybody else tried it in VBox on a modern Mac? My Mac's far too old.

poohsen commented 3 years ago

my mac is fairly new, 2018. in VBox, I picked the "ICH AC97" controller but there are two others in the list. What would be the way to debug this? Some pulseaudio or alsamixer command I could run?

ChanceNCounter commented 3 years ago

I think an immediate step would be to experiment with whether the volume skill actually works. Does it alter the output volume, or does it dick with the mic? Even if you can't see the latter, you'd notice the volume thing.

(Issue the commands in the console, ofc. "set volume to #")

The difference between volume 3 and volume 7 or 8 should be pronounced. The difference between volume 1 and volume 10 should have an inverse relationship with how high you turn up your speakers to hear it =P

Somebody else can probably give you better information about pulseaudio than I can, but that test would probably be informative.

poohsen commented 3 years ago

The problem disappeared once I moved from the virtual machine to a raspberry pi 4.