dictation-toolbox / Caster

Dragonfly-Based Voice Programming and Accessibility Toolkit
https://dictation-toolbox.github.io/Caster/
Other
340 stars 121 forks source link

Add grammar for github website keyboard shortcuts #311

Open LexiconCode opened 5 years ago

LexiconCode commented 5 years ago

Nearly every page on GitHub has a keyboard shortcut to perform actions faster.

Typing ? in GitHub brings up a dialog box that lists the keyboard shortcuts available for that page. You can use these keyboard shortcuts to perform actions across the site without using your mouse to navigate.

This should work with any browser that Cater supports. For example multiple grammars and contexts spanning over different programs

Based on the URL certain grammar should be according to the context of Available keyboard shortcuts.

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/65403555-add-grammar-for-github-website-keyboard-shortcuts?utm_campaign=plugin&utm_content=tracker%2F1825907&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F1825907&utm_medium=issues&utm_source=github).
sovoid commented 3 years ago

@LexiconCode I'd love to work on this. Is there specific portion of the documentation I can refer to so that I have more context. For example check the URL

LexiconCode commented 3 years ago

Thank you for your interest! This issue is more involved than I originally thought. Caster being built on top of dragonfly relies simply isn't aware of the HTML/CSS/Address URL in a web browser. dragonfly uses EXE, window title\class and function context(A function that must return the Bool to enable context). Without some sort of lower level integration with the browser I don't think there's much that we can do about that. There are a few different ways this could be tackled.

Leveraging the OS accessibility APIs to expose the application's functionality. Windows would be Microsoft Active Accessibility (MSAA) UI Automation (UIA). This is probably the most difficult out of the 3 however also provides the most benefit.