Immersive assistant that helps you stay organized and responds with care and concern for your well being. Immersiveness is achieved by creating a room where you are situated in a desk - every time you log on - but are allowed to roam around your room while music plays in the back or, if you would prefer, view a virtual scenery through your windows.
Using the Google Cloud Speech-to-Text API, the user can speak to Mischa, who then responds with Google text-to-speech.
Using the Google DialogFlow API, users can trigger different intents when speaking to Mischa, as well as hold general conversation. These intents include scheduling with Google Calendar, a meditation intent, and a music intent.
Using the Google Cloud Natural Language API, Mischa analyzes the user's language to determine their sentiment and keeps track of this value over time. Based on the user's sentiment, Mischa may suggest that the user uses our stress-reduction activities.
Mischa fully supports seated users with dynamic height tracking and snap turns for an optimal viewing angle. Also features teleportation capabilities to help reduce motion sickness. The selection visualizer (pointer) and teleportation controls are accessible with both controllers.
Users can freely explore different scenes, practice archery, stream videos in a mini theater, and even create 3D sculptures using Mischa's built-in paintbrush. Users can also ask Mischa to play a meditation tape or music of their choice from YouTube.
Unity Editor 2.1.3
Unity Version: 2018.4.9f1 (LTS)
Python 3.7
Python packages: speech_recognition, playsound, gtts, pyaudio, google.cloud, dialogflow, selenium
Ideas for future implementation:
See also the list of contributors who participated in this project.