dictation-toolbox / Caster

Dragonfly-Based Voice Programming and Accessibility Toolkit
Other
336 stars 122 forks source link

Add Vim support #534

Open lexxish opened 5 years ago

lexxish commented 5 years ago

This will require a significant amount of refactoring due to how Vim handles modes. Exact method TBD but prefer to keep Vim in command mode or allow either command or insert mode.

kendonB commented 5 years ago

I think the simple answer is correct for vim, which is to have voice commands for all the common keystroke combinations. e.g. something like

"insert mode":
    R(Key("i")),
"save and close":
    R(Key("escape, colon, x, enter")),

@lexxish what complications do you envision? For every command that requires command mode, prefix with escape. For caster commands that require you to already be in insert mode it's fine to assume the user already is I think.

lexxish commented 5 years ago

I was considering assuming the user spends most the time in command mode which means Caster commands would have to be overridden. Also, after doing some testing it looks like some Caster commands such as "shin" and "queue" don't even work in insert mode.

RyanQuey commented 4 years ago

I'm having a hard time using vim with caster also, it looks like commands that should be used in insert mode that use Text("some text here") don't work well, though Key("some key") works fine.

Pretty much renders rules for programming (eg the html rule) useless

LexiconCode commented 4 years ago

I'm having a hard time using vim with caster also, it looks like commands that should be used in insert mode that use Text("some text here") don't work well, though Key("some key") works fine.

Pretty much renders rules for programming (eg the html rule) useless

It would be nice to have a proper grammar of the modes and so forth. Until that time consider the following.

Key is slow. However you could try utilizing the Paste action instead in html. To make it simple without heavy edits to existing grammar use from dragonfly import Paste as Text

lexxish commented 4 years ago

I have basic vim mode tracking and commands here https://gist.github.com/lexxish/1a9582f175119f36a508282a81761d2a

I haven't had a chance to merge them into caster yet. You'd want the files vim_helper, vim_mode, vim_rule and you'd replace InheritedRule with MappingRule

LexiconCode commented 4 years ago

Awesome and of course right after I removed the WIP tag :)

RyanQuey commented 4 years ago

This is great! Thanks guys, this is super helpful!

LexiconCode commented 4 years ago

I have basic vim mode tracking and commands here gist.github.com/lexxish/1a9582f175119f36a508282a81761d2a

I haven't had a chance to merge them into caster yet. You'd want the files vim_helper, vim_mode, vim_rule and you'd replace InheritedRule with MappingRule

@lexxish function context https://github.com/dictation-toolbox/Caster/issues/747 will go a long way to make modes much easier to implement.

LexiconCode commented 4 years ago

All right @lexxish FuncContext is now supported in Caster.

FuncContext a dragonfly class that evaluates a given function with optional parameters for window title/executable, whose return value is interpreted as a bool, determining whether the context is active.

lubyrex commented 2 years ago

Hi, I was wondering if there is any update on this. Or if anyone knows of good software on speech recognition for vim. Thanks.

Talos878 commented 2 years ago

Is there documentation for a standardized required set of commands for an IDE to be accepted? I use vim, and wood like to help if possible.

LexiconCode commented 2 years ago

https://www.vim.org/

I'm willing to do this in collaboration with help. Someone could create the individual grammars consisting of one mode per file. Then I would be willing to do the plumbing to create the modes using function context.

My recommendation would to develop a grammar for a specific vim editor 1st. After that we can figure out how generalizable it is to other vim like Programs like vs code in vim mode.

Vim Cheat Sheet https://vim.rtorr.com/

Using the above cheat cheat we can create different modes function context. A final pass would probably need to be done to review the commands.

Talos878 commented 2 years ago

Using contexts, is it possible for a rule to know what mode it is in, and if a rule is for another mode switch to the proper mode and run the command from the other file?

LexiconCode commented 2 years ago

Yes, function context gives us that functionality. What function context does is evaluate a function to true or false every utterance. If true, then the context and therefore rule becomes active. This is what I did with the 4 mouse grids utilizing one function.

An example of the function https://github.com/dictation-toolbox/Caster/blob/6b0eb6825511ea77ce5f8e46662d1d30e2d8950e/castervoice/lib/navigation.py#L37

How a Function Context is defined https://github.com/dictation-toolbox/Caster/blob/5d16e4c2e7dd57c358599aa5fdb636c9c8624fb6/castervoice/rules/apps/mouse_grids/gridlegion.py#L86

Commands to change which rule is active based on that Function context https://github.com/dictation-toolbox/Caster/blob/a39d02dd9e2ea0e9ddd4119c3eb420cc03fab838/castervoice/rules/core/utility_rules/mouse_alts_rules.py#L10

LexiconCode commented 2 years ago

In a couple of days here, I'll come up with a minimum implementation that you guys can use to expand.

LexiconCode commented 2 years ago

@Talos878 @lubyrex Here you go vim_rules.zip. a minimum implementation that you guys can use to expand.

Test from Run_Caster_Test_Engine.bat

enable vim
ex # mode name eg. "normal mode, visual mode, replace mode, command mode, select mode, ex mode"
test mode # prints active mode name eg "ex-mode"
ex-mode
Talos878 commented 2 years ago

Thanks.