Implement verbal interaction

[vvasco]

Examples of potential conversation can be found here:

• the robot has to explain the test;
• the robot has to reply to potential questions related to the test, such as "at which speed should I move?", "can I use the walking stick?", "how many time should I repeat the test?";
• speech can be triggered by the "hand-raised-up" option.

Given these considerations, we have to understand which tool to use and start to create a potential conversation for a meaningful verbal interaction.

cc @vtikha

[vtikha]

:thought_balloon: :speech_balloon:

The best option then for this is to use the architecture developed for the IBM JL. Do you confirm that we will be using English? If yes, then:

• [ ] Use IBM speech to text - Backup can be google speech to text

• [ ] Use IBM services :cloud: to branch out all the possible responses

• [ ] Speech can be trigger by hand raise :raising_hand: or even hey R1:speech_balloon:

• [ ] Decide what mic to use, handheld or the imbedded ones on R1 :microphone:

[pattacini]

Unfortunately, the first language we've got to use is Italian 🤔

[vtikha]

we therefore need to use google services for speech to text.

[vvasco]

Ok so we would have:

• Google speech to text for dealing with Italian;
• IBM services cloud for all possible responses;
• speech triggered by hand raise or "hey R1"
• I would say R1 embedded mics with some constraints (interaction within some distance and no noise), to avoid equipping users with microphones?

[vtikha]

I completely agree @vvasco , but the mics are terrible on R1. In any case, the mics are plug and play therefore I can switch and test both easily.

[vvasco]

We decided to use Google services for interpreting potential questions. As we don't plan to implement a conversation for now, but rather a simple natural question and answer mechanism, we can avoid IBM services and eventually resort to them later to include a more complex interaction.

The speech interpreter will manage questions (triggered by "hey robot" or hand raise), interpret them and send a corresponding keyword to managerTUG (port name: /managerTUG/speech-interp:i), which will formulate the answer accordingly and send it to iSpeak. For now, managerTUG manages the following keywords:

• speed: "At which speed should I move?"
• aid: "Can I use a walking stick?"
• repetition: "How many repetitions should I do?"
• feedback: "How good am I in doing this?"
• unclear: the question is not understood;
• unknown: the question is not in the list of possible questions and the answer is unknown.

Important: we have to think of how to manage that the question generates an event, which can occur at any time and which the modules involved must be aware of.

cc @vtikha @pattacini

[vtikha]

The following PR deals with this by adding:

• googleSpeech: a module that receives sound from a microphone and retrieves the speech transcript from google speech cloud services
• googleSpeechProcess: a module that receives the speech transcript and analyses the phrase to retrieve its structure and meaning to interpret the questions. This uses google language cloud services. This then sends out the required keywords for use in the managetTUG

[vvasco]

Done in #225.