search

NeonGeckoCom / skill-device_controls

Device Control Center skill
Other
1 stars 2 forks source link

feat: new voices and wakewords #88

Open mikejgray opened 6 months ago

mikejgray commented 6 months ago

Description

Allows users to use one of three presets for voice/wakeword combinations:

  1. Neon ("Hey Neon", custom Neon Coqui voice model)
  2. Classic Mycroft ("Hey Mycroft", Piper Alan Pope voice model)
  3. The Butler ("Hey Jarvis", Piper Alan Pope voice model)

Issues

87

Other Notes

N/A

NeonDaniel commented 5 months ago

Regarding the test failure, can you log wake_word_config when this assertion fails? I suspect something with the test listeners is causing the config to not update properly in this test case, though I don't see an obvious cause in your changes

NeonDaniel commented 5 months ago

I got a "sorry, something has gone wrong. I will continue using-" followed by services restarting.

audio.log

2024-07-02 15:42:10.046 - audio - neon_audio.tts.neon:execute:373 - DEBUG - responses={'en-us': {'sentence': "Okay, I'll be back shortly.", 'translated': False, 'phonemes': None, 'genders': ['female'], 'female': '/home/neon/.cache/neon/tts/CoquiRemoteTTS/en-us/female/8f18e5d69575daf28d21830251e1a820.wav'}}
2024-07-02 15:42:10.249 - audio - ovos_bus_client.session:get:605 - DEBUG - No message, use default session
2024-07-02 15:42:12.954 - audio - neon_audio.tts.neon:_play:185 - INFO - Played 1719960128.2668295
2024-07-02 15:42:12.957 - audio - neon_audio.tts.neon:_sort_timing_metrics:149 - DEBUG - Parsed timing context: {'timestamps': {'handle_utterance': 1719960107.7861273, 'speech_start': 1719960128.266869, 'audio_begin': 1719960130.2339814, 'audio_end': 1719960132.8814085}, 'durations': {'transform_audio': 3.1948089599609375e-05, 'get_stt': 5.132228374481201, 'save_transcript': 1.049041748046875e-05, 'text_parsers': 0.01896810531616211, 'transform_utterance': 0.01896810531616211, 'get_tts': 1.5334551334381104}}
2024-07-02 15:42:12.986 - audio - neon_audio.service:handle_speak:148 - DEBUG - Playback completed for: 1719960128.2668295
2024-07-02 15:42:38.631 - audio - ovos_audio.service:execute_tts:381 - INFO - Speak: Sorry, something went wrong. I will continue using the old wake word.
2024-07-02 15:42:38.726 - audio - neon_audio.tts.neon:get_requested_tts_languages:79 - DEBUG - Got profiles: [{'user': {'first_name': '', 'middle_name': '', 'last_name': '', 'preferred_name': '', 'full_name': '', 'dob': 'YYYY/MM/DD', 'age': '', 'email': 'daniel@neon.ai', 'username': 'local', 'password': '', 'picture': '', 'about': '', 'phone': '', 'phone_verified': False, 'email_verified': False}, 'brands': {'ignored_brands': {}, 'favorite_brands': {}, 'specially_requested': {}}, 'speech': {'stt_language': 'en-us', 'alt_languages': ['en-us', 'uk-ua'], 'tts_language': 'en-us', 'tts_gender': 'female', 'neon_voice': '', 'secondary_tts_language': '', 'secondary_tts_gender': 'male', 'secondary_neon_voice': '', 'speed_multiplier': 1.0}, 'units': {'time': 12, 'date': 'MDY', 'measure': 'imperial'}, 'location': {'lat': 47.6765382, 'lng': -122.2070775, 'city': 'Kirkland', 'state': 'Washington', 'country': 'United States', 'tz': 'America/Los_Angeles', 'utc': -8.0, 'country_code': 'us'}, 'response_mode': {'speed_mode': 'quick', 'hesitation': False, 'limit_dialog': False}, 'privacy': {'save_audio': True, 'save_text': True}}]
2024-07-02 15:42:38.729 - audio - neon_audio.tts.neon:get_requested_tts_languages:87 - DEBUG - local requesting female en-us
2024-07-02 15:42:38.731 - audio - neon_audio.tts.neon:get_requested_tts_languages:132 - DEBUG - Got 1 TTS Voice Requests
2024-07-02 15:42:38.734 - audio - neon_audio.tts.neon:get_multiple_tts:300 - DEBUG - tts_requested=[{'speaker': 'Neon', 'language': 'en-us', 'gender': 'female', 'voice': None}]
2024-07-02 15:42:38.739 - audio - neon_audio.tts.neon:get_multiple_tts:304 - DEBUG - utterance_lang=en-us
2024-07-02 15:42:38.799 - audio - ovos_config.models:load_local:112 - DEBUG - Configuration /home/neon/.config/neon/neon.yaml loaded
2024-07-02 15:42:38.801 - audio - ovos_config.config:_on_file_change:315 - INFO - /home/neon/.config/neon/neon.yaml changed on disk
2024-07-02 15:42:38.804 - audio - neon_audio.tts.neon:_get_tts:268 - INFO - Legacy Neon TTS signature found (CoquiRemoteTTS)
2024-07-02 15:42:38.825 - audio - ovos_config.config:_on_file_change:316 - DEBUG - Calling 1 callbacks
2024-07-02 15:42:38.834 - audio - ovos_audio.service:_maybe_reload_tts:360 - DEBUG - Skipping fallback TTS init
2024-07-02 15:42:39.037 - audio - ovos_audio.service:execute_tts:381 - INFO - Speak: Wake word hey neon is still enabled. Check the logs.
2024-07-02 15:42:39.352 - audio - ovos_config.models:load_local:112 - DEBUG - Configuration /home/neon/.config/neon/neon.yaml loaded
2024-07-02 15:42:39.412 - audio - ovos_config.config:_on_file_change:315 - INFO - /home/neon/.config/neon/neon.yaml changed on disk
2024-07-02 15:42:39.419 - audio - ovos_config.config:_on_file_change:316 - DEBUG - Calling 1 callbacks
2024-07-02 15:42:39.504 - audio - neon_audio.tts.neon:execute:373 - DEBUG - responses={'en-us': {'sentence': 'Sorry, something went wrong. I will continue using the old wake word.', 'translated': False, 'phonemes': None, 'genders': ['female'], 'female': '/home/neon/.cache/neon/tts/CoquiRemoteTTS/en-us/female/ea369b8d5507d03365d8a4c806a801bb.wav'}}
2024-07-02 15:42:39.617 - audio - neon_audio.tts.neon:get_requested_tts_languages:79 - DEBUG - Got profiles: [{'user': {'first_name': '', 'middle_name': '', 'last_name': '', 'preferred_name': '', 'full_name': '', 'dob': 'YYYY/MM/DD', 'age': '', 'email': 'daniel@neon.ai', 'username': 'local', 'password': '', 'picture': '', 'about': '', 'phone': '', 'phone_verified': False, 'email_verified': False}, 'brands': {'ignored_brands': {}, 'favorite_brands': {}, 'specially_requested': {}}, 'speech': {'stt_language': 'en-us', 'alt_languages': ['en-us', 'uk-ua'], 'tts_language': 'en-us', 'tts_gender': 'female', 'neon_voice': '', 'secondary_tts_language': '', 'secondary_tts_gender': 'male', 'secondary_neon_voice': '', 'speed_multiplier': 1.0}, 'units': {'time': 12, 'date': 'MDY', 'measure': 'imperial'}, 'location': {'lat': 47.6765382, 'lng': -122.2070775, 'city': 'Kirkland', 'state': 'Washington', 'country': 'United States', 'tz': 'America/Los_Angeles', 'utc': -8.0, 'country_code': 'us'}, 'response_mode': {'speed_mode': 'quick', 'hesitation': False, 'limit_dialog': False}, 'privacy': {'save_audio': True, 'save_text': True}}]
2024-07-02 15:42:39.621 - audio - neon_audio.tts.neon:get_requested_tts_languages:87 - DEBUG - local requesting female en-us
2024-07-02 15:42:39.628 - audio - neon_audio.tts.neon:get_requested_tts_languages:132 - DEBUG - Got 1 TTS Voice Requests
2024-07-02 15:42:39.658 - audio - neon_audio.tts.neon:get_multiple_tts:300 - DEBUG - tts_requested=[{'speaker': 'Neon', 'language': 'en-us', 'gender': 'female', 'voice': None}]
2024-07-02 15:42:39.662 - audio - neon_audio.tts.neon:get_multiple_tts:304 - DEBUG - utterance_lang=en-us
2024-07-02 15:42:39.668 - audio - neon_audio.tts.neon:_get_tts:268 - INFO - Legacy Neon TTS signature found (CoquiRemoteTTS)
2024-07-02 15:42:39.737 - audio - ovos_bus_client.session:get:605 - DEBUG - No message, use default session
2024-07-02 15:42:41.984 - audio - neon_audio.tts.neon:execute:373 - DEBUG - responses={'en-us': {'sentence': 'Wake word hey neon is still enabled. Check the logs.', 'translated': False, 'phonemes': None, 'genders': ['female'], 'female': '/home/neon/.cache/neon/tts/CoquiRemoteTTS/en-us/female/7f492da5f1189046714154697d04426e.wav'}}
2024-07-02 15:42:41.988 - audio - ovos_audio.service:_maybe_reload_tts:349 - INFO - (re)loading TTS engine
2024-07-02 15:42:41.991 - audio - ovos_plugin_manager.utils.config:get_plugin_config:40 - DEBUG - Loaded configuration: {'lang': 'en-us', 'voice': 'alan-low', 'module': 'ovos-tts-plugin-piper', 'pulse_duck': False, 'preload_fallback': False, 'fallback_module': 'coqui'}
2024-07-02 15:42:42.150 - audio - ovos_plugin_manager.utils:load_plugin:161 - WARNING - Could not find the plugin mycroft.plugin.tts.ovos-tts-plugin-piper
2024-07-02 15:42:42.155 - audio - neon_audio.tts:create:59 - ERROR - Could not find plugin: ovos-tts-plugin-piper
2024-07-02 15:42:42.159 - audio - ovos_config.config:_on_file_change:321 - ERROR - Error in config update callback handler
Traceback (most recent call last):
  File "/home/neon/venv/lib/python3.10/site-packages/ovos_config/config.py", line 319, in _on_file_change
    handler()
  File "/home/neon/venv/lib/python3.10/site-packages/ovos_audio/service.py", line 351, in _maybe_reload_tts
    self.tts.init(self.bus, self.playback_thread)
AttributeError: 'NoneType' object has no attribute 'init'
2024-07-02 15:42:43.525 - audio - neon_audio.tts.neon:_play:185 - INFO - Played 1719960158.5661793
2024-07-02 15:42:43.528 - audio - neon_audio.tts.neon:_sort_timing_metrics:149 - DEBUG - Parsed timing context: {'timestamps': {'handle_utterance': 1719960107.7861273, 'speech_start': 1719960158.5661988, 'audio_begin': 1719960159.7325459, 'audio_end': 1719960163.4334536}, 'durations': {'transform_audio': 3.1948089599609375e-05, 'get_stt': 5.132228374481201, 'save_transcript': 1.049041748046875e-05, 'text_parsers': 0.01896810531616211, 'transform_utterance': 0.01896810531616211, 'get_tts': 0.7698671817779541}}
2024-07-02 15:42:43.566 - audio - neon_audio.service:handle_speak:148 - DEBUG - Playback completed for: 1719960158.5661793
2024-07-02 15:42:49.745 - audio - ovos_utils:wait_for_exit_signal:183 - DEBUG - Exiting on KeyboardInterrupt
2024-07-02 15:42:50.137 - audio - ovos_config.config:_on_file_change:312 - DEBUG - Ignoring non-config file change: /home/neon/.config/neon/schedule.json
2024-07-02 15:42:51.317 - audio - ovos_bus_client.client.client:on_error:122 - ERROR - === RuntimeError('cannot schedule new futures after shutdown') ===
Traceback (most recent call last):
  File "/home/neon/venv/lib/python3.10/site-packages/websocket/_app.py", line 672, in _callback
    callback(self, *args)
  File "/home/neon/venv/lib/python3.10/site-packages/ovos_bus_client/client/client.py", line 160, in on_message
    self.emitter.emit(parsed_message.msg_type, parsed_message)
  File "/home/neon/venv/lib/python3.10/site-packages/pyee/_base.py", line 113, in emit
    handled = self._call_handlers(event, args, kwargs)
  File "/home/neon/venv/lib/python3.10/site-packages/pyee/_base.py", line 96, in _call_handlers
    self._emit_run(f, args, kwargs)
  File "/home/neon/venv/lib/python3.10/site-packages/pyee/_executor.py", line 50, in _emit_run
    future = self._executor.submit(f, *args, **kwargs)
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 167, in submit
    raise RuntimeError('cannot schedule new futures after shutdown')
RuntimeError: cannot schedule new futures after shutdown
2024-07-02 15:42:51.375 - audio - ovos_bus_client.client.client:on_error:126 - ERROR - Failed to emit error event: cannot schedule new futures after shutdown
Traceback (most recent call last):
  File "/home/neon/venv/lib/python3.10/site-packages/ovos_bus_client/client/client.py", line 124, in on_error
    self.emitter.emit('error', error)
  File "/home/neon/venv/lib/python3.10/site-packages/pyee/_base.py", line 116, in emit
    self._emit_handle_potential_error(event, args[0] if args else None)
  File "/home/neon/venv/lib/python3.10/site-packages/pyee/_base.py", line 86, in _emit_handle_potential_error
    raise error
  File "/home/neon/venv/lib/python3.10/site-packages/websocket/_app.py", line 672, in _callback
    callback(self, *args)
  File "/home/neon/venv/lib/python3.10/site-packages/ovos_bus_client/client/client.py", line 160, in on_message
    self.emitter.emit(parsed_message.msg_type, parsed_message)
  File "/home/neon/venv/lib/python3.10/site-packages/pyee/_base.py", line 113, in emit
    handled = self._call_handlers(event, args, kwargs)
  File "/home/neon/venv/lib/python3.10/site-packages/pyee/_base.py", line 96, in _call_handlers
    self._emit_run(f, args, kwargs)
  File "/home/neon/venv/lib/python3.10/site-packages/pyee/_executor.py", line 50, in _emit_run
    future = self._executor.submit(f, *args, **kwargs)
  File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 167, in submit
    raise RuntimeError('cannot schedule new futures after shutdown')
RuntimeError: cannot schedule new futures after shutdown
2024-07-02 15:42:51.835 - audio - ovos_bus_client.client.client:on_error:134 - WARNING - Message Bus Client will reconnect in 5.0 seconds.
2024-07-02 15:42:56.850 - audio - ovos_bus_client.client.client:on_open:93 - DEBUG - Connected

skills.log

2024-07-02 15:42:08.292 - skills - neon_utils.skills.neon_skill:speak:646 - DEBUG - Skill speak! {'utterance': "Okay, I'll be back shortly.", 'lang': 'en-us', 'expect_response': False, 'meta': {'dialog': 'confirm_change', 'data': {}, 'skill': 'DeviceControlCenterSkill'}, 'speaker': None, 'speak_ident': '1719960128.2668295'}
2024-07-02 15:42:08.295 - skills - neon_utils.skills.neon_skill:speak:647 - DEBUG - speak
2024-07-02 15:42:08.303 - skills - skill_device_controls:_set_user_tts_settings:565 - DEBUG - Patching user ngi config for persona jarvis
2024-07-02 15:42:08.308 - skills - skill_device_controls:_retrieve_ngi_config:583 - DEBUG - Patching user ngi config for Neon TTS
2024-07-02 15:42:08.339 - skills - ovos_bus_client.client.client:on_default_session_update:166 - DEBUG - synced default_session
2024-07-02 15:42:08.538 - skills - neon_utils.configuration_utils:_write_yaml_file:344 - DEBUG - YAML updated ngi_user_info
2024-07-02 15:42:08.540 - skill-device_controls.neongeckocom - DEBUG - Attempting to enable WW: hey_jarvis
2024-07-02 15:42:10.389 - skills - ovos_bus_client.util.scheduler:check_state:186 - DEBUG - Call scheduled event: skill-ovos-homescreen.openvoiceos:skill-ovos-homescreen.openvoiceosupdate_dt
2024-07-02 15:42:20.408 - skills - ovos_bus_client.util.scheduler:check_state:186 - DEBUG - Call scheduled event: skill-ovos-homescreen.openvoiceos:skill-ovos-homescreen.openvoiceosupdate_dt
2024-07-02 15:42:30.429 - skills - ovos_bus_client.util.scheduler:check_state:186 - DEBUG - Call scheduled event: skill-ovos-homescreen.openvoiceos:skill-ovos-homescreen.openvoiceosupdate_dt
2024-07-02 15:42:38.548 - skills - skill_device_controls:_emit_enable_ww_message:594 - ERROR - No response to WW enable request for hey_jarvis!
2024-07-02 15:42:38.551 - skills - skill_device_controls:_enable_wake_word:435 - ERROR - No response to WW enable request for hey_jarvis!
2024-07-02 15:42:38.555 - skills - neon_utils.skills.neon_skill:speak_dialog:680 - DEBUG - data={}
2024-07-02 15:42:38.568 - skills - neon_utils.skills.neon_skill:speak:646 - DEBUG - Skill speak! {'utterance': 'Sorry, something went wrong. I will continue using the old wake word.', 'lang': 'en-us', 'expect_response': False, 'meta': {'dialog': 'error_ww_change_failed', 'data': {}, 'skill': 'DeviceControlCenterSkill'}, 'speaker': None, 'speak_ident': '1719960158.5661793'}
2024-07-02 15:42:38.571 - skills - neon_utils.skills.neon_skill:speak:647 - DEBUG - speak
2024-07-02 15:42:38.752 - skill-device_controls.neongeckocom - DEBUG - Found available WW: {'hey_mycroft': {'active': False, 'module': 'ovos-ww-plugin-precise-lite', 'listen': True, 'sound': 'snd/start_listening.wav', 'version': '0.3', 'model': 'https://github.com/OpenVoiceOS/precise-lite-models/raw/master/wakewords/en/hey_mycroft.tflite', 'expected_duration': 3, 'trigger_level': 3, 'sensitivity': 0.5}, 'hey_neon': {'module': 'ovos-ww-plugin-vosk', 'listen': True, 'sound': 'snd/start_listening.wav', 'debug': False, 'rule': 'fuzzy', 'lang': 'en', 'samples': ['hey neon', '[unk]'], 'model_folder': '/home/neon/.local/share/neon/vosk-model-small-en-us-0.15', 'active': True}, 'hey_jarvis': {'module': 'ovos-ww-plugin-openwakeword', 'active': True, 'listen': True}}
2024-07-02 15:42:38.754 - skill-device_controls.neongeckocom - DEBUG - Found enabled WWs: ['hey_neon', 'hey_jarvis']
2024-07-02 15:42:38.755 - skill-device_controls.neongeckocom - DEBUG - Disabling WW: hey_neon
2024-07-02 15:42:38.927 - skills - skill_device_controls:_disable_wake_word:466 - WARNING - WW disable failed with response: {'error': 'only one active ww', 'active': True, 'wake_word': 'hey_neon'}
2024-07-02 15:42:38.933 - skills - neon_utils.skills.neon_skill:speak_dialog:680 - DEBUG - data={'ww': 'hey neon'}

It seems there's something going on with disabling the old WW before enabling the new one which causes an error to avoid a state where there is no active WW.

UPDATE: After restarting, it looks like the Audio module is failing to load due to a missing plugin:

2024-07-02 15:46:18.874 - audio - neon_audio.tts:create:59 - ERROR - Could not find plugin: ovos-tts-plugin-piper
2024-07-02 15:46:18.880 - audio - ovos_audio.service:__init__:95 - ERROR - 'NoneType' object has no attribute 'init'
AttributeError: 'NoneType' object has no attribute 'init'
2024-07-02 15:47:41.265 - audio - ovos_utils.process_utils:exists:353 - ERROR - Failed to kill PID <_io.TextIOWrapper name='/tmp/neon/audio.pid' mode='r' encoding='UTF-8'>: [Errno 3] No such process
2024-07-02 15:47:57.173 - audio - neon_audio.tts:create:59 - ERROR - Could not find plugin: ovos-tts-plugin-piper
2024-07-02 15:47:57.176 - audio - ovos_audio.service:__init__:95 - ERROR - 'NoneType' object has no attribute 'init'
AttributeError: 'NoneType' object has no attribute 'init'

I'll install that and try again

mikejgray commented 4 months ago

@NeonDaniel https://github.com/NeonGeckoCom/NeonCore/pull/702