Add text-to-speech functionality

duwudi commented 3 years ago

Use pi-top's onboard speaker to interact with the user via speech - particularly useful/interesting for robotics applications but can also have a lot of value for simpler Foundation Kit projects.

See old library created here that uses espeak.

Possibly make this part of the Pitop class so it can be used by default without the user specifically adding it manually
Would be nice to have multiple implementations available to use, i.e. user can choose between a high-quality cloud-based version (Google/Amazon/Microsoft) or a lower quality local version (espeak etc)

Options

Google TTS

Based on Google translate so decent voice quality
Has a python package pip3 install gTTS
Requires internet connection

Usage:

from gtts import gTTS
import pygame
from io import BytesIO

pygame.init()

tts = gTTS(text=text, lang='en')
fp = BytesIO()
tts.write_to_fp(fp)
fp.seek(0)

pygame.mixer.init()
pygame.mixer.music.load(fp)
pygame.mixer.music.play()

eSpeak

Offline
Poor quality voice
See pyttsx3 for python library that uses this under the hood

Pico TTS

Offline
Better voice quality than espeak
Supports British English, American English, Spanish, Dutch, French, and Italian (all women voices)
No python package available, would have to run terminal commands to use binary from our own sdk

Installation

wget http://ftp.us.debian.org/debian/pool/non-free/s/svox/libttspico0_1.0+git20130326-9_armhf.deb
wget http://ftp.us.debian.org/debian/pool/non-free/s/svox/libttspico-utils_1.0+git20130326-9_armhf.deb
sudo apt-get install -f ./libttspico0_1.0+git20130326-9_armhf.deb ./libttspico-utils_1.0+git20130326-9_armhf.deb

Usage:

pico2wave -w lookdave.wav "Hello, world!" && aplay lookdave.wav

pyttsx3

Offline
Can use different engines under the hood, defaults to espeak on linux
Poor voice quality
Has a fairly well maintained and documented python package available

Installation

sudo apt install espeak
pip3 install pyttsx3

Usage

import pyttsx3
engine = pyttsx3.init()
engine.say("Hello, world!")
engine.runAndWait()

Microsft TTS

Has python package
Requires Azure subscription key to use which is a bit of a hassle

Amazon Polly

Cloud
Requires an account with API keys
This python package is a TTS broker compatible with google cloud, amazon polly and ibm (all these require credentials)

Festival

Offline
Similar quality to espeak
Has a python lib but just uses subprocess under the hood

Installation

sudo apt-get install festival
pip3 install pyfestival

Mozilla TTS

Very good quality voices
pip installable but lots of dependencies, including pytorch which isn't available via pypy on armv7
This TTS can be installed on a RPi 4 but the process is lengthy and requires compiling pytorch
Speech is not real-time either (6x real-time to produce audio)

Mimic1 from Mycroft.ai

Compiled binary, instructions
Relatively lightweight but much better voice than most other offline solutions

Usage

./mimic -t "Hello"

# change voice
./mimic -t "Hello" -voice kal16

angusjfw commented 2 years ago

I got pretty good results using festival with a plug-in voice as described here.

sudo apt-get install -y festival festvox-us-slt-hts
festival -b '(voice_cmu_us_slt_arctic_hts)' \
    '(SayText "The temperature is 22 degrees centigrade and there is a slight breeze from the west.")'

The python lib actually binds to the c++ lib, subprocesses are only used for converting to mp3 I think. It gives me an ImportError when I try to use it though.

angusjfw commented 2 years ago

Similar results with flite, which mimic is based on:

sudo apt install flite
flite -voice slt -t "The Raspberry Pi is a great Maker platform!"

Didn't try the python bindings

duwudi commented 2 years ago

I got pretty good results using festival with a plug-in voice as described here.

This one sounds pretty good actually, let's just use that!

duwudi commented 2 years ago

The python lib actually binds to the c++ lib, subprocesses are only used for converting to mp3 I think. It gives me an ImportError when I try to use it though.

Can we still use the plugin if using the python library?

duwudi commented 2 years ago

It gives me an ImportError when I try to use it though.

Yeah I've got the same thing, the pypy package is not actually up to date with the repo though - importing from the pip-installed version gives this error:

Traceback (most recent call last):
  File "/home/pi/.local/lib/python3.7/site-packages/festival.py", line 3, in <module>
    from . import _festival
ImportError: attempted relative import with no known parent package

Installing from the repo directly gives this error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/pi/.local/lib/python3.7/site-packages/festival.py", line 6, in <module>
    import _festival
ImportError: /lib/arm-linux-gnueabihf/libestools.so.2.5: undefined symbol: omp_get_thread_num

Not sure how to fix this, any ideas?

duwudi commented 2 years ago

Didn't try the python bindings

Can't get this one building either, fails on python3 setup.py build even after installing swig

duwudi commented 2 years ago

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/pi/.local/lib/python3.7/site-packages/festival.py", line 6, in <module>
    import _festival
ImportError: /lib/arm-linux-gnueabihf/libestools.so.2.5: undefined symbol: omp_get_thread_num

Adding this fix https://github.com/techiaith/pyfestival/pull/9 gets the python bindings working. The plan is to fork this repo and manage it ourselves since it's relatively simple

pi-top / pi-top-Python-SDK