NIMBUS: The CSAI Voice Assistant
The "Cal Poly Computer Science & Artificial Intelligence" club is designing a voice assistant to provide the Cal Poly Computer Science department with relevant information about the department and university. The intent is for users to ask for information about the department such as professor's office hours & expertise, course prerequisites, etc. Users can also ask the assistant for general information about the university such as the history, mascott, and more. Our project team was awarded the "Google Cloud Platform Educational Grant" to fund the usage of server hosting, database management, and speech application needs.
Leads
Chidi Ewenike - Project Lead
Miles Aikens - Software/Cloud Infrastructure Lead
Jason Ku - Software/Test Lead
Michael Fekadu - Data Lead
John Waidhofer - Application Development Lead
Richa Gadgil - Natural Language Processing Lead
Cameron Toy - Software Lead
Tyler Herzog - Hardware Lead
Austin Silveria - Information Infrastructure Lead
Active Developers:
Faculty Advisors
Dr. Fooad Khosmood - Natural Language Processing/Machine Learning
Dr. Wayne Pilkington - Audio Processing
Dr. Franz Kurfess - Machine Learning
Spring 2019: Wake Word
Wake Word Phase Developers: Kalen Goo, Tyler Herzog, Max Mohammadi, Brett Nelson, Viet Nguyen, Kevin Nottberg, Austin Silveria, Cameron Toy, John Waidhofer
The CSAI Voice Assistant Project Team is currently designing the wake-word solution for the voice assistant. This is to create a speech recognition application layer for the assistant which will activate our voice assistant by saying "Nimbus". We have collected wake word audio data on the name of the voice assistant "Nimbus" and non wake word which would result in false activations. With this audio data, we extract MFCCs as input features to feed the Gated Recurrent Unit. We are currently building and optimizing our Gated Recurrent Unit model and designing infrastructure to implement our current model.
Summer 2019: Data Acquisition/Optimization
Data Acquisition/Optimization Phase Developers: Kalen Goo, Tyler Herzog, Max Mohammadi, Brett Nelson, Viet Nguyen, Austin Silveria, Cameron Toy
Natural Language Processing data acquisition and optimization of the current wake word solution. We are currently developing data collection applications for iOS, Android, and Web for members to access. These applications will allow for both wake-word and NLP intent collection and validation.
Fall 2019: Natural Language Processing Domain-Specific Implementation
Natural Language Processing Phase Developers: Jacob Burghgraef , Tyler Campanile, Daniel DeFoe, Allen Deng, Kalen Goo, Yale Hone, Max Mohammadi, Viet Nguyen, Kevin Nottberg, Sai Pamidi ,Adam Perlin, Austin Silveria
Applying the NLP layer and optimizing to make accurate queries for accurate predictions. Also scripting a site scraping program to obtain necessary information and adding such information to a look-up table for future access.
Winter 2020: Infrastructure - Hardware/Mobile Application Design
Finalzing the NLP layer and implementing Google Cloud Platform Speech-To-Text and Text-To-Speech. The hardware and mobile application design will occur to complete the project for the end-to-end product.
Project Progress
