search

OpenASR / idiolect

🎙️ Handsfree Audio Development Interface
https://arxiv.org/pdf/2305.03089.pdf
Apache License 2.0
93 stars 10 forks source link

Feature/settings panel and xml #36

Closed nalbion closed 7 years ago

nalbion commented 7 years ago

This is a work in progress, but welcome feedback as this is my first IntelliJ plugin. I think the form would eventually have more options - maybe a separate panel for Recognition & TTS - locale, voice, credentials...

breandan commented 7 years ago

Nice idea to abstract over ASR/TTS providers! I should be able to review and accept these over the weekend. Thank you for contributing!

nalbion commented 7 years ago

I started on an AWS Lex ASR recognition implementation. Think an NLP abstraction is also required to convert a recognised string (and context) into actions. For Lex and Google Voice API I think the ASR mic needs a Voice Activity Detector so that only the interesting audio snippets are sent. The current implementation saves 15 seconds of audio to disk, what happens when the voice command spans 2 connective recordings? Perhaps CustomMicrophone also needs a VAD?

breandan commented 7 years ago

Thanks, merged. If your project configuration gets corrupted after updating (it happened to me), you will need to remove all .idea related configuration files (or kill it completely rm -rf idear && git clone https://github.com/OpenASR if that fails) and Import Project as a Gradle project in IntelliJ IDEA.

breandan commented 7 years ago

FYI the Minecraft Dev project has some good examples of using the IntelliJ Platform SDK in Kotlin. Particularly the settings implementation. Kotlin has some language features to make things easier, ie. late initialization lets us avoid using !! and ?everywhere. I can handle the plugin API side going forward if you want to focus on integrating these providers.