Plugin doesn't work

[mvrius]

Hi,

Is this expected to work with pre-release ovos? It gets loaded but looks like it hangs the entire pipeline somehow. Not even wakewords work if this gets loaded.

Not sure how to debug yet, but I wanted to know if this is expected to work properly.

Thanks!

[mikejgray]

Hi @mvrius , we're going to need some more information to be able to troubleshoot this with you:

1. How did you install OVOS?
2. What hardware and OS are you using?
3. What are the contents of your audio.log file, or if it's Docker, can you share the STDOUT output of the audio service?
4. What version of Python are you running?

Upstream Piper forces Python 3.9 or higher so it will fail to install if it's lower than that. Wakewords are part of the speech service, which is entirely separate and usually decoupled from the audio service, so it seems that something unusual is happening. I just used Piper on a Mark 2 with near-latest OVOS pre-release code yesterday without an issue.

[mvrius]

Hi @mikejgray,

Thanks for you reply. Here's the log from ovos-audio, it looks like it's not getting loaded.

2024-06-07 09:51:52.853 - OVOS - ovos_plugin_manager.audio:<module>:7 - WARNING - Deprecation version=0.1.0. Caller=ovos_plugin_manager.plugin_entry:4. ovos_plugin_manager.audio has been deprecated on ovos-audio, move to ovos_plugin_manager.media
2024-06-07 09:51:52.902 - audio - ovos_audio.service:__init__:60 - INFO - Starting Audio Service
2024-06-07 09:51:52.902 - audio - ovos_audio.service:on_started:42 - INFO - TTS service started.
2024-06-07 09:51:53.636 - audio - ovos_audio.service:_maybe_reload_tts:349 - INFO - (re)loading TTS engine
2024-06-07 09:51:53.660 - audio - ovos_audio.playback:run:152 - INFO - PlaybackThread started
2024-06-07 09:51:53.708 - audio - ovos_plugin_manager.tts:create:205 - INFO - Found plugin ovos_tts_plugin_espeakng
2024-06-07 09:51:53.713 - audio - ovos_plugin_manager.templates.tts:__init__:171 - WARNING - Deprecation version=Unknown. Caller=ovos_tts_plugin_espeakng:12. lang argument for TTS has been deprecated! it will be ignored, pass lang to get_tts directly instead
eSpeak NG text-to-speech: 1.50  Data at: /usr/lib/x86_64-linux-gnu/espeak-ng-data
2024-06-07 09:51:54.050 - audio - ovos_plugin_manager.tts:create:209 - INFO - Loaded plugin ovos_tts_plugin_espeakng
2024-06-07 09:51:54.051 - audio - ovos_audio.service:on_alive:38 - INFO - TTS service is alive.
2024-06-07 09:51:54.051 - audio - ovos_audio.service:on_ready:34 - INFO - TTS service is ready.

This is what I have in mycroft.conf

"tts": {
        "module": "ovos-tts-plugin-piper",
        "ovos-tts-plugin-piper": {
          "voice": "southern_english_female-low"
        }
      },

I verified the module is installed correctly, by running this small script:

    from ovos_tts_plugin_piper import PiperTTSPlugin
    config = {}
    e = PiperTTSPlugin(config=config)
    e.get_tts("one oh clock? yes! it is one on the clock", "hello.wav")

Any thoughts on how to debug why it's not getting loaded?

[mikejgray]

Yes, would you please run cat ~/.config/mycroft/mycroft.conf | jq? You may need to install JQ first.

I'm not sure how you installed OVOS. espeakng hasn't been a default TTS engine for a long time.

[mvrius]

Appreciate your help!

My end goal is to run it on a system that is non-standard for OVOS, as such on my Ubuntu machine I started with the instructions of installing as modules https://openvoiceos.github.io/community-docs/042-install_ovos_core/ and for experimentation purposes, I am running each one of the required applications separately. ovos-core, ovos-audio, ovos-media, etc...

Because I had trouble with the released packages, I'm installing all pre-release versions. I realize this is not ideal, but for now it seems to be working OK.

python3 -m pip install --pre ovos_core ovos_audio ovos_media ovos_PHAL ovos_messagebus ovos_dinkum_listener ovos_tts_plugin_espeakng ovos-tts-plugin-piper ovos-stt-plugin-vosk ovos-ww-plugin-openwakeword ovos-stt-plugin-whispercpp ovos-microphone-plugin-alsa ovos-vad-plugin-silero

Here's the output of the command.

{
  "enable_old_audioservice": false,
  "tts": {
    "module": "ovos-tts-plugin-piper",
    "ovos-tts-plugin-piper": {
      "voice": "southern_english_female-low"
    }
  },
  "stt": {
    "module": "ovos-stt-plugin-vosk-streaming",
    "ovos-stt-plugin-vosk-streaming": {
      "model": "https://alphacephei.com/vosk/models/vosk-model-en-us-0.22-lgraph.zip",
      "model_small": "http://alphacephei.com/vosk/models/vosk-model-small-en-us-0.15.zip",
      "verbose": true
    },
    "ovos-stt-plugin-vosk": {
      "model": "https://alphacephei.com/vosk/models/vosk-model-en-us-0.22-lgraph.zip",
      "model_small": "http://alphacephei.com/vosk/models/vosk-model-small-en-us-0.15.zip"
    }
  },
  "stt_openai": {
    "module": "ovos-stt-plugin-whispercpp",
    "ovos-stt-plugin-whispercpp": {
      "model": "small.en",
      "model_tiny": "tiny.en",
      "model_base": "base.en"
    }
  },
  "listener": {
    "wake_word": "hey_jarvis",
    "microphone": {
      "module": "ovos-microphone-plugin-alsa",
      "device": "default",
      "period_size": 1024,
      "timeout": 5,
      "multiplier": 2,
      "audio_retries": 0,
      "audio_retry_delay": 0
    }
  },
  "hotwords": {
    "hey_jarvis": {
      "module": "ovos-ww-plugin-openwakeword",
      "sound": "snd/start_listening.wav",
      "listen": true,
      "debug": true
    }
  },
  "media": {
    "enable_mpris": false,
    "manage_external_players": false,
    "preferred_audio_services": [
      "gui",
      "vlc",
      "mplayer",
      "cli"
    ],
    "preferred_web_services": [
      "gui",
      "browser"
    ],
    "preferred_video_services": [
      "gui",
      "vlc"
    ],
    "audio_players": {
      "vlc": {
        "module": "ovos-media-audio-plugin-vlc",
        "aliases": [
          "VLC"
        ],
        "active": true
      },
      "cli": {
        "module": "ovos-media-audio-plugin-cli",
        "aliases": [
          "Command Line"
        ],
        "active": true
      },
      "gui": {
        "module": "ovos-media-audio-plugin-gui",
        "aliases": [
          "GUI",
          "Graphical User Interface"
        ],
        "active": true
      }
    },
    "video_players": {
      "vlc": {
        "module": "ovos-media-video-plugin-vlc",
        "aliases": [
          "VLC"
        ],
        "active": true
      },
      "gui": {
        "module": "ovos-media-video-plugin-gui",
        "aliases": [
          "GUI",
          "Graphical User Interface"
        ],
        "active": true
      }
    },
    "web_players": {
      "browser": {
        "module": "ovos-media-web-plugin-browser",
        "aliases": [
          "Browser",
          "Local Browser",
          "Default Browser"
        ],
        "active": true
      },
      "gui": {
        "module": "ovos-media-web-plugin-gui",
        "aliases": [
          "GUI",
          "Graphical User Interface"
        ],
        "active": true
      }
    }
  }
}

[mvrius]

From the docs, it seems that OVOS loads multiple configurations files with some precedence order and merges them. If I import ovos_config and dump the Configuration object, it looks wildly different. Is that expected?

Is it possible to have OVOS not load defaults, and rely on a single file, my file, so I have certainty of what's the configuration that is taking effect?

[mvrius]

Here's the Configuration object dumped

{
  "Audio": {
    "backends": {
      "OCP": {
        "active": true,
        "disable_mpris": true,
        "manage_external_players": false,
        "type": "ovos_common_play"
      },
      "simple": {
        "active": true,
        "type": "ovos_audio_simple"
      },
      "vlc": {
        "active": true,
        "type": "ovos_vlc"
      }
    },
    "native_sources": [
      "debug_cli",
      "audio",
      "mycroft-gui"
    ]
  },
  "confirm_listening": true,
  "coref": {
    "module": "ovos-coref-solver-heuristic"
  },
  "date_format": "MDY",
  "debug": false,
  "enable_old_audioservice": false,
  "g2p": {
    "module": "ovos-g2p-plugin-heuristic-arpa"
  },
  "gui": {
    "extension": "generic",
    "generic": {
      "homescreen_supported": true
    },
    "idle_display_skill": "skill-ovos-homescreen.openvoiceos"
  },
  "gui_websocket": {
    "base_port": 18181,
    "host": "0.0.0.0",
    "route": "/gui",
    "ssl": false
  },
  "hotwords": {
    "hey_jarvis": {
      "debug": true,
      "listen": true,
      "module": "ovos-ww-plugin-openwakeword",
      "sound": "snd/start_listening.wav"
    },
    "hey_mycroft": {
      "expected_duration": 3,
      "fallback_ww": "hey_mycroft_precise",
      "listen": true,
      "model": "https://github.com/OpenVoiceOS/precise-lite-models/raw/master/wakewords/en/hey_mycroft.tflite",
      "module": "ovos-ww-plugin-precise-lite",
      "sensitivity": 0.5,
      "trigger_level": 3
    },
    "hey_mycroft_pocketsphinx": {
      "lang": "en-us",
      "listen": true,
      "module": "ovos-ww-plugin-pocketsphinx",
      "phonemes": "HH EY . M AY K R AO F T",
      "threshold": 1e-90
    },
    "hey_mycroft_precise": {
      "expected_duration": 3,
      "fallback_ww": "hey_mycroft_vosk",
      "listen": true,
      "model": "https://github.com/MycroftAI/precise-data/raw/models-dev/hey-mycroft.tar.gz",
      "module": "ovos-ww-plugin-precise",
      "sensitivity": 0.5,
      "trigger_level": 3,
      "version": "0.3"
    },
    "hey_mycroft_vosk": {
      "fallback_ww": "hey_mycroft_pocketsphinx",
      "listen": true,
      "module": "ovos-ww-plugin-vosk",
      "rule": "fuzzy",
      "samples": [
        "hey mycroft",
        "hey microsoft",
        "hey mike roft",
        "hey minecraft"
      ]
    },
    "wake_up": {
      "fallback_ww": "wake_up_pocketsphinx",
      "lang": "en-us",
      "module": "ovos-ww-plugin-vosk",
      "rule": "fuzzy",
      "samples": [
        "wake up"
      ],
      "wakeup": true
    },
    "wake_up_pocketsphin": {
      "lang": "en-us",
      "module": "ovos-ww-plugin-pocketsphinx",
      "phonemes": "W EY K . AH P",
      "threshold": 1e-20,
      "wakeup": true
    }
  },
  "ignore_logs": [
    "enclosure.mouth.viseme",
    "enclosure.mouth.display"
  ],
  "intents": {
    "OCP": {
      "classifier_threshold": 0.4,
      "filter_SEI": true,
      "filter_media": true,
      "legacy": false,
      "min_score": 50,
      "playback_mode": 0,
      "search_fallback": true
    },
    "pipeline": [
      "stop_high",
      "converse",
      "ocp_high",
      "padatious_high",
      "adapt_high",
      "ocp_medium",
      "fallback_high",
      "stop_medium",
      "adapt_medium",
      "padatious_medium",
      "adapt_low",
      "common_qa",
      "fallback_medium",
      "fallback_low"
    ]
  },
  "keyword_extract": {
    "module": "ovos-keyword-extractor-heuristic"
  },
  "lang": "en-us",
  "language": {
    "detection_module": "ovos-lang-detector-plugin-server",
    "translation_module": "ovos-translate-plugin-server"
  },
  "listener": {
    "VAD": {
      "before_seconds": 0.5,
      "initial_energy_threshold": 1000,
      "max_current_ratio_threshold": 2,
      "min_seconds": 1,
      "module": "ovos-vad-plugin-silero",
      "ovos-vad-plugin-silero": {
        "threshold": 0.2
      },
      "ovos-vad-plugin-webrtcvad": {
        "vad_mode": 3
      },
      "silence_method": "vad_and_ratio",
      "silence_seconds": 0.5,
      "speech_seconds": 0.1
    },
    "continuous_listen": false,
    "energy_ratio": 1.5,
    "hybrid_listen": false,
    "instant_listen": false,
    "listen_timeout": 45,
    "microphone": {
      "audio_retries": 0,
      "audio_retry_delay": 0,
      "device": "default",
      "module": "ovos-microphone-plugin-alsa",
      "multiplier": 2,
      "period_size": 1024,
      "timeout": 5
    },
    "multiplier": 1,
    "mute_during_output": false,
    "phoneme_duration": 120,
    "record_wake_words": false,
    "recording_timeout": 10,
    "recording_timeout_with_silence": 3,
    "retry_mic_init": true,
    "sample_rate": 16000,
    "save_utterances": false,
    "silence_end": 0.7,
    "speech_begin": 0.3,
    "stand_up_word": "wake_up",
    "utterance_chunks_to_rewind": 2,
    "wake_word": "hey_jarvis",
    "wake_word_upload": {
      "disable": true,
      "url": ""
    },
    "wakeword_chunks_to_save": 15
  },
  "location": {
    "city": {
      "code": "Lawrence",
      "name": "Lawrence",
      "state": {
        "code": "KS",
        "country": {
          "code": "US",
          "name": "United States"
        },
        "name": "Kansas"
      }
    },
    "coordinate": {
      "latitude": 38.971669,
      "longitude": -95.23525
    },
    "timezone": {
      "code": "America/Chicago",
      "dstOffset": 3600000,
      "name": "Central Standard Time",
      "offset": -21600000
    }
  },
  "media": {
    "audio_players": {
      "cli": {
        "active": true,
        "aliases": [
          "Command Line"
        ],
        "module": "ovos-media-audio-plugin-cli"
      },
      "gui": {
        "active": true,
        "aliases": [
          "GUI",
          "Graphical User Interface"
        ],
        "module": "ovos-media-audio-plugin-gui"
      },
      "vlc": {
        "active": true,
        "aliases": [
          "VLC"
        ],
        "module": "ovos-media-audio-plugin-vlc"
      }
    },
    "enable_mpris": false,
    "manage_external_players": false,
    "preferred_audio_services": [
      "gui",
      "vlc",
      "mplayer",
      "cli"
    ],
    "preferred_video_services": [
      "gui",
      "vlc"
    ],
    "preferred_web_services": [
      "gui",
      "browser"
    ],
    "video_players": {
      "gui": {
        "active": true,
        "aliases": [
          "GUI",
          "Graphical User Interface"
        ],
        "module": "ovos-media-video-plugin-gui"
      },
      "vlc": {
        "active": true,
        "aliases": [
          "VLC"
        ],
        "module": "ovos-media-video-plugin-vlc"
      }
    },
    "web_players": {
      "browser": {
        "active": true,
        "aliases": [
          "Browser",
          "Local Browser",
          "Default Browser"
        ],
        "module": "ovos-media-web-plugin-browser"
      },
      "gui": {
        "active": true,
        "aliases": [
          "GUI",
          "Graphical User Interface"
        ],
        "module": "ovos-media-web-plugin-gui"
      }
    }
  },
  "metadata_transformers": {},
  "microservices": {
    "email": {
      "recipient": "",
      "smtp": {
        "host": "smtp.mailprovider.com",
        "password": "",
        "port": 465,
        "username": ""
      }
    },
    "geolocation_provider": "auto",
    "owm_key": "",
    "weather_provider": "auto",
    "wolfram_key": "",
    "wolfram_provider": "auto"
  },
  "network_tests": {
    "captive_portal_text": "NetworkManager is online",
    "captive_portal_url": "http://nmcheck.gnome.org/check_network_status.txt",
    "dns_primary": "1.1.1.1",
    "dns_secondary": "8.8.8.8",
    "ip_url": "https://api.ipify.org",
    "web_url": "http://nmcheck.gnome.org/check_network_status.txt",
    "web_url_secondary": "https://checkonline.home-assistant.io/online.txt"
  },
  "opt_in": false,
  "padatious": {
    "fuzz": true,
    "regex_only": false
  },
  "postag": {
    "module": "ovos-classifiers-postag-plugin"
  },
  "ready_settings": [
    "skills"
  ],
  "secondary_langs": [],
  "segmentation": {
    "module": "ovos-segmentation-plugin-quebrafrases"
  },
  "server": {
    "backend_type": "offline",
    "metrics": true,
    "sync_skill_settings": true,
    "update": false,
    "url": "",
    "version": "v1"
  },
  "session": {
    "ttl": -1
  },
  "skills": {
    "autogen_meta": false,
    "blacklisted_skills": [
      "skill-ovos-stop.openvoiceos"
    ],
    "converse": {
      "converse_activation": "accept_all",
      "converse_blacklist": [],
      "converse_mode": "accept_all",
      "converse_priorities": {},
      "converse_whitelist": [],
      "cross_activation": true,
      "cross_deactivation": true,
      "max_activations": -1,
      "skill_activations": {},
      "skill_timeouts": {},
      "timeout": 300
    },
    "directory": "skills",
    "fallbacks": {
      "fallback_blacklist": [],
      "fallback_mode": "accept_all",
      "fallback_priorities": {},
      "fallback_whitelist": []
    },
    "sync2way": true,
    "upload_skill_manifest": false
  },
  "sounds": {
    "acknowledge": "snd/acknowledge.mp3",
    "end_listening": "snd/end_listening.wav",
    "error": "snd/error.mp3",
    "start_listening": "snd/start_listening.wav"
  },
  "stt": {
    "fallback_module": "",
    "module": "ovos-stt-plugin-vosk-streaming",
    "ovos-stt-plugin-vosk": {
      "model": "https://alphacephei.com/vosk/models/vosk-model-en-us-0.22-lgraph.zip",
      "model_small": "http://alphacephei.com/vosk/models/vosk-model-small-en-us-0.15.zip"
    },
    "ovos-stt-plugin-vosk-streaming": {
      "model": "https://alphacephei.com/vosk/models/vosk-model-en-us-0.22-lgraph.zip",
      "model_small": "http://alphacephei.com/vosk/models/vosk-model-small-en-us-0.15.zip",
      "verbose": true
    }
  },
  "stt_openai": {
    "module": "ovos-stt-plugin-whispercpp",
    "ovos-stt-plugin-whispercpp": {
      "model": "small.en",
      "model_base": "base.en",
      "model_tiny": "tiny.en"
    }
  },
  "system": {
    "disable_remote_config": false,
    "disable_user_config": false,
    "protected_keys": {
      "remote": [
        "enclosure",
        "server",
        "system",
        "websocket",
        "gui_websocket",
        "network_tests",
        "listener:wake_word_upload:disable",
        "listener:channels",
        "listener:sample_rate",
        "listener:multiplier",
        "listener:energy_ratio",
        "skills:upload_skill_manifest",
        "skills:auto_update",
        "skills:priority_skills",
        "skills:blacklisted_skills",
        "opt_in"
      ],
      "user": []
    }
  },
  "system_unit": "metric",
  "time_format": "half",
  "tokenization": {
    "module": "ovos-tokenization-plugin-quebrafrases"
  },
  "tts": {
    "fallback_module": "",
    "module": "ovos-tts-plugin-piper",
    "ovos-tts-plugin-mimic": {
      "min_free_percent": 75,
      "persist_cache": false,
      "persist_thresh": 1,
      "preloaded_cache": "",
      "voice": "ap"
    },
    "ovos-tts-plugin-piper": {
      "voice": "southern_english_female-low"
    },
    "pulse_duck": false
  },
  "utterance_transformers": {
    "ovos-utterance-corrections-plugin": {},
    "ovos-utterance-normalizer": {},
    "ovos-utterance-plugin-cancel": {}
  },
  "websocket": {
    "host": "127.0.0.1",
    "max_msg_size": 25,
    "port": 8181,
    "route": "/core",
    "shared_connection": true,
    "ssl": false
  }
}

[JarbasAl]

have you tried the installer? https://github.com/OpenVoiceOS/ovos-installer

[mvrius]

I have, it works on Ubuntu, but doesn't work on the custom hardware, so I want to locally experiment with the modules path, which maps exactly to what works on the device.

In both deployments, Ubuntu and the device - everything works well with modules if I select any other TTS plugin but piper which is very strange.

Is there a module loader that I could put breakpoints in or such?

[JarbasAl]

your logs dont match your config :thinking:

the config you posted doesnt look quite right, and FYI ovos-media is not yet usable. I also dont see espeak anywhere in there, but it is loading in your logs?

Is it possible to have OVOS not load defaults, and rely on a single file, my file, so I have certainty of what's the configuration that is taking effect?

no, otherwise missing values cause things to fail all over the place, default values are shipped with ovos-config package. You can then have your own changes in the user mycroft.conf file

[JarbasAl]

but doesn't work on the custom hardware,

I'm a bit confused, what does this mean exactly? what hardware are we talking about, with what base OS / python version?

[mvrius]

I'm a bit confused, what does this mean exactly? what hardware are we talking about, with what base OS / python version?

It's an ARM64 with Ubuntu 19 and Python 3.8 - the default scripts don't work on this configuration. I've also tried miniconda with Python 3.9 and also the manual scripts don't work. The autoinstaller requires Python 3.11 for which on ARM64 there are several packages not published for.

The only setup I was able to get working is the manual packages path.

[mvrius]

Good news though, I'm not sure why really, but solving other errors that were showing up in core and audio specifically installing pandatious and fixing sklearn version, now piper starting working.

2024-06-07 11:30:22.435 - audio - ovos_audio.service:execute_tts:381 - INFO - Speak: Hello
2024-06-07 11:30:22.437 - audio - ovos_plugin_manager.utils.tts_cache:load_persistent_cache:263 - INFO - Persistent TTS cache files loaded successfully.

/run/user/1000/tts/ovos-tts-plugin-piper/southern_english_female-low/en-us/8b1a9953c4611296a827abf8c47804d7.wav:

 File Size: 19.0k     Bit Rate: 257k
  Encoding: Signed PCM    
  Channels: 1 @ 16-bit   
Samplerate: 16000Hz      
Replaygain: off         
  Duration: 00:00:00.59  

In:0.00% 00:00:00.00 [00:00:00.59] Out:0     [      |      ]        Clip:0    
In:86.5% 00:00:00.51 [00:00:00.08] Out:8.19k [!=====|=====!]        Clip:0    2024-06-07 11:30:23.438 - audio - ovos_audio.service:execute_tts:381 - INFO - Speak: hello english folks

In:100%  00:00:00.59 [00:00:00.00] Out:9.47k [      |      ] Hd:0.0 Clip:0    
In:100%  00:00:00.59 [00:00:00.00] Out:9.47k [      |      ] Hd:0.0 Clip:0    
Done.

/run/user/1000/tts/ovos-tts-plugin-piper/southern_english_female-low/en-us/f4e7d6928b1af02be2c5683bb7bdc9a6.wav:

 File Size: 42.0k     Bit Rate: 256k
  Encoding: Signed PCM    
  Channels: 1 @ 16-bit   
Samplerate: 16000Hz      
Replaygain: off         
  Duration: 00:00:01.31  

In:0.00% 00:00:00.00 [00:00:01.31] Out:0     [      |      ]        Clip:0    
In:39.0% 00:00:00.51 [00:00:00.80] Out:8.19k [!=====|=====!]        Clip:0    
In:78.0% 00:00:01.02 [00:00:00.29] Out:16.4k [======|======] Hd:0.0 Clip:0    
In:100%  00:00:01.31 [00:00:00.00] Out:21.0k [  -===|===-  ] Hd:0.0 Clip:0    
In:100%  00:00:01.31 [00:00:00.00] Out:21.0k [      |      ] Hd:0.0 Clip:0    
Done.

[mvrius]

Closing this off.

@JarbasAl if there's a forum where I can post about the custom HW I'm trying to target, I'll gladly do so, in help of others perhaps trying the same.

I'm also trying to write a custom skill and not fully groking the developer workflow. Posted here https://github.com/orgs/OpenVoiceOS/discussions/269

Really appreciate your replies folks!

[JarbasAl]

the github discussions are our "official forums", but the community is also active in https://community.openconversational.ai/c/community-projects/ovos/35

those used to be the old Mycroft forums, now maintained by Neon, all related projects are welcome there

for completeness, we also have https://www.reddit.com/r/OpenVoiceOS/ and our matrix chatrooms