omlins / JustSayIt.jl

Software and high-level API for offline, low latency and secure translation of human speech to computer commands or text on Linux, MacOS and Windows
BSD 3-Clause "New" or "Revised" License
86 stars 12 forks source link

Add support for interaction with Visual Studio Code #30

Open omlins opened 2 years ago

damian-666 commented 1 year ago

i would love this. i have been trying to get a MSFT execs to use this a as a replacement for all the UI in Start, ( after its fixed) and to all Copilot to code against APIs it was not trained on.. Voice rec on DSL or App or OS doms , to make a plugin for Chat GPI, so i can say , "Computer, sort all my files by date" sho a query, in logical 90s like symbols , and do it. Don't ask but type what i said. If any data might be lost of its not go high confidence, i've never done it, or can delete something, ask confirmation.. Yes and No against just yes and no and not the would English's language should be used.. Synonyms should be used, generated, then customizable. But noone is in charge. How much money does bad software really make? I know the game.. Engagement, addiction. abuse tolerance. spying. So UI is a contentious issue but if you agree with one mic, one hotword and my description, that is the best , most agreeable for everyone, then the UI arguments about screen real estate or duplicate functionality might just not apply as the NLS will become the front end and subsume much of the contention tabs and menus and mouse click and distances and ergonomics, and will be predictive

The new DEveloper Studio, has a code/ feature indexed search that combines the discoverability of UI, with Random access of command line.c.. Right how as an assistant , or driving my UI, its no better than cortana. This and its low latency combined with the transformers of Chat GPI could set the standard for all apps, a Ribbon with a Mic icon this glows and gives signal info clipping and level + speech to text.. confirmation levels should be tweakable , and if its read you the prompt, and yo say "proceed", "do it" its gets added to its dossier on you , how to talk and fixies it when you said no, i meant t this, but raised confidence when you keep going, or say thanks, If you type a correction it can learn..

The basic ui, with a one line google like text box. thas is the whole default UI. Command and Query for the OS and each App, with a sort of template for common apps, with UI in NetCore.. I would love it julia was as deployable as .net is and hope they get IL -> or jullia into the dotnet umbrella somehow. If you think this is feasible I can find someone with leadership ability in MSFT, relentlessly because all the UI product owners resigned.. One when they ruined the start menu into a ad platforms or whatever . It Bings you how to do something its should be able to do. Ist just awful.. Google assistant is closer put windows should have this.. Linux is too fragmented. Or ill hry s Mac and run Jet Brains..

Physicists need engineers, AIs an get us to space, and im held back by crappy tooling ants 95% there . I've worked with your an Olegs Cs-Scripts scripting in C# and custom IDEs for physics for over a decade. If you agree ill push i can be very persuasive. I need to know if this is feasible and your opinion

omlins commented 1 year ago

Thanks for your comment, the JustSayIt recipes I'm going to share soon (https://github.com/omlins/JustSayItRecipes.jl), allow to work very well with Visual Studio Code. Stay tuned!