search

WEKIT-ECS / MIRAGE-XR

MirageXR is a reference implementation of an XR training system. MirageXR enables experts and learners to share experience via XR and wearables using ghost tracks, realtime feedback, and anchored instruction.
Other
28 stars 4 forks source link

Refactor keyword manager #481

Closed wekitecs closed 2 years ago

wekitecs commented 3 years ago

In GitLab by @wild on Jun 24, 2021, 15:44

Remove all the ballast that does not exist anymore (carefully check what "show sensors" does - possibly leave it there or comment out for future use). We definitely use "move actions" / "move task list" and "move activity list" on worldspace UIs. We also definitely use "back" and "next". Users have asked for new ones like "create new step" / "add step", "start recording a video", "take a picture", ...

Outcome of the discussion below:

This way the user gets confirmation of what's happening and can learn the voice commands available.

Related ticket: See it, say it: https://platform.xr4all.eu/wekit-ecs/mirage-xr/-/issues/72

wekitecs commented 3 years ago

In GitLab by @wild on Jun 24, 2021, 15:52

Also add a prompt (like "abracadabra - next" or "mirage - next")?

wekitecs commented 3 years ago

In GitLab by @a85jafari on Jul 12, 2021, 09:11

Carlo from EAC: It’d be nice to call the different features with voice commands (i.e. create new step, start recording a video, take a pic, etc.)

wekitecs commented 3 years ago

In GitLab by @wild on Aug 31, 2021, 16:03

Or "Alakazam", "hocus pocus", ...? The vote is out :)

wekitecs commented 3 years ago

In GitLab by @wild on Aug 31, 2021, 16:06

Here is the full list: https://platform.xr4all.eu/wekit-ecs/mirage-xr/-/blob/master/Assets/MirageXR/Common/Scripts/Services/KeywordService.cs

So far, I think we are only using a few of them really ("move actions", "move activity list", not sure whether "reset" or "start" even work still?).

Refactor?

wekitecs commented 3 years ago

In GitLab by @BenediktHensen on Sep 2, 2021, 15:59

The prompt should be something short and easy to pronounce. A long or complicated phrase will become annoying and feel inefficient to people, especially if they need to use speech commands often.

wekitecs commented 3 years ago

In GitLab by @robhillman97 on Sep 2, 2021, 17:24

based on what @william.guest was saying about harsh consonants for reliable recognition we could just use "Wekit" or shortened down to "Kit" as the prompt? (Although that abbreviation might feel like talking to an AI car from the 80's at times...)

wekitecs commented 3 years ago

In GitLab by @a85jafari on Sep 2, 2021, 20:06

But it should ne something special, daily used English words can be recognized as prompt when users try to record an audio or dialog.

@BenediktHensen The short word should follows by a specific word to avoid the issue I mentioned above. As Google or Amazon use OK and Hi after their name/device name. (Hi Alexa, Ok Google, Hello Siri, Hi Bixby)

wekitecs commented 3 years ago

In GitLab by @wild on Sep 3, 2021, 17:06

Here are a few more (https://en.wikipedia.org/wiki/Magic_word): "Hey Presto" "Sim Sala Bim" "Shazam" "Skadoosh" "Hex Hex"

They all seem bit childish, though (imagine a CEO of Boeing standing on stage saying "Abrakadabra"). I quite like the neutrality of "Hey Presto" - and it has a plosive as well, so should be good to recognise?

https://www.minigrey-blog.com/wp-content/uploads/2017/10/Hypnotism-poster-small.jpg

Maybe we could even add a few of them to allow for variety (Hey Presto, Abracadabra, Alakazam, Hex Hex, Simsalabim)?

wekitecs commented 3 years ago

In GitLab by @wild on Sep 7, 2021, 15:10

Or quite simply "Hey Mirage"?

wekitecs commented 3 years ago

In GitLab by @wild on Sep 7, 2021, 15:15

To provide feedback to the user that things are happening, how about the following: when users speaks the prompt ("Hey Presto", "Hey Mirage", "Sim Sala Bim"), we display the prompt with think white type face in screen space, fading in quickly, and we list the available voice commands below. When the user then continues to speak "move action panel", then we blink the words in bold, and fade out the whole dialogue again quickly. This way the user gets confirmation of what's happening and can learn the voice commands available.

wekitecs commented 3 years ago

In GitLab by @BorisJov on Sep 16, 2021, 15:50

marked this issue as related to #72