give the user a graphical interface that allows the creation of automated routines

[fernando-jose-silva]

Is your feature request related to a problem? Please describe.

to explain my suggestion i would like to site another screen reader and platform. I resentfully had contact with the jieshuo screen reader that runs on the android, and a screen reader feature caught my attention a lot. In this screen reader it is possible to copy button names, several of these names, each name in a line, then it is possible to associate a word to these names. This screen reader has a voice assistant, where you speak the command associated with button names, and the screen reader clicks these buttons. To give a better example, the person showed me: in the window where we write the names of the controls she entered: most playback speed 2x as a word to assign these clicks she put it fast. Now when she enters youtube she plays the voice assistant and automatically the video is put at 2x playback speed. Optionally, the user can put a number after the button names to indicate how many milliseconds they want the reader to wait to press each button. I'm not a programmer and I have no contact with the developers of this screen reader, but I believe that the feature works more or less like this. The screen reader searches the screen for the name written by the user in the field to tap. Upon finding this word in the current window, the screen reader performs a tap on this word. You can do this on various buttons that appear on the screen. I would like to suggest to nvda a feature based on this functionality. Since I think voice recognition is very difficult for nvda to implement, we can replace this part with a keystroke. So the user would have an interface something like this: we pretend that the user wants to skip the youtube ads. He would then write in a text interface skip ads in another field would press a set of keys that he wants to associate with this command for example ctrl + nvda + l now the user can enter a youtube page, he waits the 5 seconds that youtube requires for you to watch the ad, and now he presses the shortcut ctrl + nvda + l Nvda will search the page for the sequence of words to skip ad and will click the mouse on this button. This was just an example, now we think about how many tasks the user can automate.

I believe that it is possible to do something like this by developing an addon, but it is necessary to learn how to program an addon, and this can be easy, but for non-experts it can represent a big barrier. The simplicity that the developers of jieshuo implemented this feature caught my attention, and it works, I wanted to do it myself on a cell phone and it magically works, I was delighted.

Describe the solution you'd like

allow the user, through a simple interface and without the need to know how to program, at most, some intervention of execution time and other very basic things, to be able to assemble small scripts to perform actions on the computer.

Describe alternatives you've considered

Additional context

it may seem silly, but I've had contact with people, an 89-year-old blind user for 6 years that I'm teaching him to go on youtube and listen to his old songs that bring him so many memories. I tried to tell him to keep typing the letter b until he found the skip ad button, plus the hearing barrier, hearing loss due to age, and a little lack of motor coordination make this activity difficult, because sometimes he prefers to wait for all the ads until Final. I could use this tool myself, I use a program at work where I need to do a series of button clicks to access a specific screen, I would use this feature to speed up this task. We think that not all programmers put shortcut keys on every button. Thanks.

[feerrenrut]

Not to discount this idea, there are a few alternatives that partially solve this:

• Use the elements list
• Try Searching for text

In particular searching for text seems to be pretty close to what you are describing, but there is no ability to assign to a gesture (keyboard input). Other applications would call this a "macro" (a series of manual input tasks that can saved and assigned a name)

One concern is that (specifically web applications) change often, the macros would need to be recreated occasionally.

[mltony]

@fernando-jose-silva, I implemented this half way in my BrowserNav add-on. In particular, you can set up a regex of all the words you might want to find and then you can jump to them by pressing J or Shift+J. I think your idea of also adding an option to click some buttons automatically sounds very interesting, I will try to find some time to add it into the next version of my add-on.

[fernando-jose-silva]

thanks! in the future we can make this feature available in other Windows arias, as well as integrate into the core of NVDA

[mltony]

Added in BrowserNav v2.2 . You can create either a QuickJump bookmark that you can jump to by pressing J. Or QuickClick bookmark - by pressing Alt+J you can click a button or link automatically. This is still not macro language, but I believe it might solve the problem you're describing.

[fernando-jose-silva]

thank you very much, for adding it to your add-on, I'm already using it, I've set it for example to skip the YouTube ad by pressing alt j And I'm also going to teach my students how to use this add-on for this activity, this is a complicated task for novice users, or those who have difficulties, once again congratulations, exactly what I imagined. Now as a suggestion, I would still like that in the future, nv access would accept this feature in nvda's core, since you were able to prove that an implementation of this one works, and it works very well. This add-on, for me, is already at the level of essential add-ons for nvda congratulations, thank you very much.

[cary-rowen]

Really great, I really wish Tony's many add-ons were added to the NVDA core.