NVDA Command Palette

[SaschaCowley]

Is your feature request related to a problem? Please describe.

Currently, there are several commands in NVDA that can only be executed if an input gesture is assigned to them. This can present a problem where a user wishes to execute a script that does not have an input gesture assigned, but which they do not wish to give an input gesture, for example because it is rarely used. This may also be problematic for users who have difficulty remembering or executing complex input gestures, for whom there are a limited number of available key combinations to choose from.

Describe the solution you'd like

Introduce a "command palette" that the user can open with a single input gesture (or via the NVDA menu), which presents all scripts that are possible in the current context. Allow the user to type to search for gestures (preferably with some sort of "fuzzy logic" to allow for imprecise matches). Allow the user to activate the selected script, with the effect of the script taking place in the context from which they invoked the command palette. Optionally, display the input gesture(s) associated with the script (according to the method used to open the command palette (desktop layout/laptop layout/touch)) alongside the script.

Describe alternatives you've considered

Add an "Activate" button to the input gestures dialog, which performs the selected script in the context from which the user opened the input gestures dialog.

Additional context

The command palette paradigm is present in many software products. For instance, Visual Studio Code and Windows Terminal both offer such functionality (by default both invoked with Ctrl+Shift+P). Some other screen readers, including Voiceover on the Mac, offer a "Commands Menu" which is functionally equivalent to a command palette.

Example user stories:

• As a user with a cognitive impairment, I want to be able to use natural language to search for NVDA commands so that I can use my computer without struggling to remember a lot of keystrokes.
• As a new NVDA user, I want to be able to search for the command I need so that I can carry out tasks without having to switch to and from NVDA’s documentation.
• As a user with arthritis, I want to be able to execute NVDA scripts without complex keyboard shortcuts that are painful to input.
• As a power user, I want to be able to search for NVDA scripts to help me quickly activate less frequently used actions.

[CyrilleB79]

A similar experience, to some extent, is implemented in Command Helper add-on by @javidominguez, present in the Add-on Store.

I think the "Activate" button in the input gesture dialog is also implemented in an add-on, NVDA global commands extension; maybe @paulber19 can confirm.

[javidominguez]

Hello, indeed, Command Helper implements that feature. I was actually inspired by the VoiceOver palette to develop it. It works through a Virtual Menu, with no visible interface. When the menu is active, a keyboard overlay captures all keystrokes. It also includes voice command search. All of this is intended to ensure that the addon interferes as little as possible with the system state. The problem with displaying a visible dialog with a actívate button is that it attracts the system focus and even if you hide it before launching the execution, the system focus or NVDA focus may not return exactly to the same place it was and that would alter the result of some scripts.

[nvdaes]

I think that the CommandHelper add-on is very valuable. Also, it would be great if this could be implemented with a visible interface. Years ago, when I created JAWS scripts, a dialog could be created in JAWS to select the command to be run with a visible interface. I think that there are people complaining about NVDA keystrokes, difficult to press or to remember, and hope this issue can be prioritized accordingly. I'm very interested, mostly for new users or even people with low vision, so they can see the available keystrokes easily.

[nvdaes]

Thinking about this, I think that the suggested alternative of using the input gestures dialog would be nice, to avoid the creation of a different command. Also, some people may be used to this dialog and the search feature is already implemented. If the option is expanded to select a gesture, it may be passed to the script. The problem may arise with commands without a description, intended to send a gesture, not shown in this dialog. A predefined description may be used in these cases, like: "Sends x gesture", or these commands may be excluded. Another problem is caused by commands which require to repeat a gesture, but I think that these commands should have their own scripts. I've tested in the Python console, and seems that gui.mainFrame.prevFocus can be used to ensure that the focus is set to that object, thought I'm not completely sure if this would work in all cases. Though generally, when destroying the input gestures dialog, the focus seems to be preserved, though I would need to perform more testing to be sure.

[paulber19]

YES, that's correct.

Le 24/09/2024 08:45, Cyrille Bougot a écrit :

A similar experience, to some extent, is implemented in Command Helper https://github.com/javidominguez/commandHelper/ add-on by @javidominguez https://github.com/javidominguez, present in the Add-on Store.

I think the "Activate" button in the input gesture dialog is also implemented in an add-on, NVDA global commands extension; maybe @paulber19 https://github.com/paulber19 can confirm.

— Reply to this email directly, view it on GitHub https://github.com/nvaccess/nvda/issues/17209#issuecomment-2370329873, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADZLFFF3WRY5OKE3DNB57TDZYEC65AVCNFSM6AAAAABOXNJE4KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNZQGMZDSOBXGM. You are receiving this because you were mentioned.Message ID: @.***>

[RuturajL]

This is a valid use-case and a helpful feature. The Blindpandas team had released an addon for this a while back, I don't think it's compatible with latest versions of NVDA anymore. https://github.com/blindpandas/command_palette

[CyrilleB79]

Command Helper use a virtual interface: we can navigate in menus / commands with speech (and maybe braille) feedback, but no dialog is visible at all. On the other side, NVDA global commands extension add-on and Command Palette add-on use real dialogs.

The advantage of a virtual interface with respect to a real dialog is that we have no side effect when calling commands related to the current focus, the navigator object or the review cursor. On the other side, the drawback is that nothing is visible, what may be an issue for visually impaired people or for sighted collaborators.

My personal preference goes for the virtual interface, and I am an occasional user of Command Helper, because I am comfortable enough to work without a visual feedback (even if I have a little vision left.

[paulber19]

The problem posed is to activate a script without a input gesture. Today, To find out which scripts can be activated, the user uses the "Input gesture" dialog. It is then that he notices that the script does not have an associated gesture. Two solutions: -, a button in the dialog allows him to activate the script.

• he must use another dialog to activate it.

I think that apart from a technical problem, the choice between the two solutions is obvious.

Le 26/09/2024 13:16, Cyrille Bougot a écrit :

Command Helper use a virtual interface: we can navigate in menus / commands with speech (and maybe braille) feedback, but no dialog is visible at all. On the other side, NVDA global commands extension add-on and Command Palette add-on use real dialogs.

The advantage of a virtual interface with respect to a real dialog is that we have no side effect when calling commands related to the current focus, the navigator object or the review cursor. On the other side, the drawback is that nothing is visible, what may be an issue for visually impaired people or for sighted collaborators.

My personal preference goes for the virtual interface, and I am an occasional user of Command Helper, because I am comfortable enough to work without a visual feedback (even if I have a little vision left.

— Reply to this email directly, view it on GitHub https://github.com/nvaccess/nvda/issues/17209#issuecomment-2376655053, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADZLFFG5R7TIDAQO4G3WTQ3ZYPUHRAVCNFSM6AAAAABOXNJE4KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNZWGY2TKMBVGM. You are receiving this because you were mentioned.Message ID: @.***>

[CyrilleB79]

@paulber19 wrote:

The problem posed is to activate a script without a input gesture. Today, To find out which scripts can be activated, the user uses the "Input gesture" dialog. It is then that he notices that the script does not have an associated gesture. Two solutions: -, a button in the dialog allows him to activate the script. - he must use another dialog to activate it. I think that apart from a technical problem, the choice between the two solutions is obvious.

Even if you are replying to my message, you seem to totally ignore the virtual interface proposal.

Have you tested Command Helper add-on? If yes, why have you preferred the dialog solution in your add-on? You have surely had good reasons, but it would be interesting to make them all explicit here so that they are shared with everyone.

Re the choice between input gesture dialog and another dialog, again explicit is better than implicit. So what seem obvious to me is that if we go forward with a dialog solution, your solution is the best one, i.e. add one more button in the input gesture dialog rather than creating a brand new dialog.

[paulber19]

Sorry, but I think I expressed myself incorrectly. Speaking of another dialog, I meant to force the user when he is in the "Input gesture" dialog to use another means to activate a script without having to assign a command gesture , whether this dialogue is visible or invisible. I think this answers all the questions.Le 26/09/2024 18:02, Cyrille Bougot a écrit :

@paulber19 https://github.com/paulber19 wrote:

The problem posed is to activate a script without a input gesture.
Today, To find out which scripts can be activated, the user uses
the "Input gesture" dialog. It is then that he notices that the
script does not have an associated gesture. Two solutions: -, a
button in the dialog allows him to activate the script. - he must
use another dialog to activate it. I think that apart from a
technical problem, the choice between the two solutions is obvious.

Even if you are replying to my message https://github.com/nvaccess/nvda/issues/17209#issuecomment-2376655053, you seem to totally ignore the virtual interface proposal.

Have you tested Command Helper add-on? If yes, why have you preferred the dialog solution in your add-on? You have surely had good reasons, but it would be interesting to make them all explicit here so that they are shared with everyone.

Re the choice between input gesture dialog and another dialog, again explicit is better than implicit. So what seem obvious to me is that if we go forward with a dialog solution, your solution is the best one, i.e. add one more button in the input gesture dialog rather than creating a brand new dialog.

— Reply to this email directly, view it on GitHub https://github.com/nvaccess/nvda/issues/17209#issuecomment-2377371489, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADZLFFAQWDEONBKFUPAJA2LZYQV2TAVCNFSM6AAAAABOXNJE4KVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDGNZXGM3TCNBYHE. You are receiving this because you were mentioned.Message ID: @.***>