search

Azure-Samples / azure-search-openai-demo

A sample app for the Retrieval-Augmented Generation pattern running in Azure, using Azure AI Search for retrieval and Azure OpenAI large language models to power ChatGPT-style and Q&A experiences.
https://azure.microsoft.com/products/search
MIT License
6.02k stars 4.12k forks source link

Custom Keyword Recognition for Voice Activation #2031

Open ram-atea opened 1 week ago

ram-atea commented 1 week ago

This issue is for a: (mark with an x)

- [ ] bug report -> please search issues before submitting
- [x] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Hii ✨

I would love to see keyword recognition/voice activation integrated into the chatbot! With the chatbot continuously listen for custom trigger phrases like "Hey chatbot" to initiate transcription.

This feature would significantly enhance accessibility, especially for users with disabilities, by allowing handsfree use of the app and making it more user friendly for more people.

Given that speech functionality is already implemented, I believe this could be built upon to introduce keyword recognition. However, I’ve attempted to implement this myself without success so far. Is there an "easy" way to incorporate it into the existing code? I feel this addition would really benefit many users and open up new ways to interact with the chatbot! 😊

pamelafox commented 2 days ago

Have you looked into https://github.com/Azure-Samples/aisearch-openai-rag-audio ? I wonder if that would be more the UI you're interested in. I'm working on a PR that makes it super easy to use with the search index from this app.