STT / TTS not working

[superflewis]

Specs

• Leon version: 1.0.1
• OS (or browser) version: Brave 1.34.81
• Node.js version: 17.1.0
• Complete "leon check" (or "npm run check") output:

pi@alchemyvoice:~/.leon $ npm run check

leon@1.0.0-beta.5 check babel-node scripts/run-check.js

⠋

.: CHECKING :.

➡ node --version ✔ v17.1.0

➡ npm --version ✔ 8.1.2

➡ pipenv --version ✔ pipenv, version 2022.1.8

➡ pipenv --where ✔ /home/pi/.leon/bridges/python

➡ pipenv run python --version ✔ Python 3.7.3

➡ pipenv run python bridges/python/main.py scripts/assets/query-object.json ✔ {"package": "leon", "module": "randomnumber", "action": "run", "lang": "en", "input": "Give me a random number", "entities": [], "output": {"type": "end", "codes": ["success"], "speech": 23, "options": {}}}

➡ NLP model state ✔ Found and valid

➡ Amazon Polly TTS ✔ Configured

➡ Google Cloud TTS/STT ✔ Configured

➡ Watson TTS ❗ Watson TTS is not yet configured

➡ Offline TTS ❗ Cannot find bin/flite/flite. You can setup the offline TTS by running: "npm run setup:offline-tts"

➡ Watson STT ❗ Watson STT is not yet configured

➡ Offline STT ❗ Cannot find bin/deepspeech/lm.binary. You can setup the offline STT by running: "npm run setup:offline-stt"

---

.: REPORT :.

➡ Here is the diagnosis about your current setup ✔ Run ✔ Run modules ✔ Reply you by texting ✔ Amazon Polly text-to-speech ✔ Google Cloud text-to-speech ❗ Watson text-to-speech ❗ Offline text-to-speech ✔ Google Cloud speech-to-text ❗ Watson speech-to-text ❗ Offline speech-to-text

✔ Hooray! Leon can run correctly

Expected Behavior

1. When I type in the browser, should I expect an audio response?
2. How can I confirm or test if Leon can access my mic and speaker?
3. I uttered a hotword, "Leon", no response from device.

Actual Behavior

I typed a greeting, Leon responded with text. But did not produce an audio response.

How Do We Reproduce?

My setup is a RaspberryPi with an Adafruit Voice Bonnet

Extra (like a sample repo to reproduce the issue, etc.)

[louistiti]

Hi @superflewis 👋,

Thanks for opening this issue.

• In your .env file, do you confirm that LEON_STT and LEON_TTS are set to true?
• In the browser, when you click the mic icon, does it activate it (color animation)?
• In the browser, can you please open the DevTools and report what you see in the console when you click the mic icon?

Now to answer your questions:

1. When I type in the browser, should I expect an audio response?

Yes, if it is correctly configured you should expect an audio response.

2. How can I confirm or test if Leon can access my mic and speaker?

As long as you allow your browser to access your audio input, it should work.

[superflewis]

Hi @louistiti, thanks for your fast reply!

Browser console was showing: MediaDevices.getUserMedia() is not supported on your browser. I've fixed that. Currently...

1.) No audio from Pi or browser when this shows: .: LEON :. ➡ Talking...

2.) The mic icon activates. When it stops, only this is showing: .: ASR :. ➡ Encoding WebM file to WAVE file... ✔ Encoding done ➡ Parsing WAVE file...

No errors in console, tested browsers in Win/Linux. Mic tested at mic-test.com

Just to confirm, is this not a correct expected behavior?:

• Leon is installed on my Pi. It has a working mic and speaker.
• I can say the 'Leon' hotword and it will be detected by the Pi, and audio output from the Pi speaker.
• And I can type in browser on a PC, with audio output from Pi.

[louistiti]

Thanks for sharing more details!

Leon is installed on my Pi. It has a working mic and speaker. I can say the 'Leon' hotword and it will be detected by the Pi, and audio output from the Pi speaker. And I can type in browser on a PC, with audio output from Pi.

This should work. Can you please try to install Leon on your PC with the current configuration and see if it works by using the PC only? Maybe there is an issue about the audio source/output when streaming from a Pi.

Also, can you please share the content of your .env file?

[kotnibf]

Thanks