Click to move focus & system carat in web pages

[nvaccessAuto]

Reported by steverep80 on 2015-02-08 22:20 When in browse mode (document review), an editable document such as an email message, Word document, etc. has the desired property within NVDA that a left mouse button click moves the carat and focus to that position, and thus the user can just arrow down from there to continue editing and reading. For low vision users, especially on webpages with no or poor semantics or landmarks, this would be a desired effect of a left mouse click. In this way, a lot of junk could be skipped and flat reviewing via the arrow keys or tabbing from that point forward could be achieved.

Right now, there is technically a workaround, which is to:

1. Place the mouse at the desired position, making sure to be in "document review"
2. Use NVDA+numpad-multiply to move the navigator object to the mouse
3. Use NVDA+shift+numpad-minus twice to move the focus and carat

This is a bit of a lengthy process clearly, and simply pointing and clicking would be much easier.

Ticket #3004 is similar to this request, but I believe I've provided a more concise description of the feature request here.

Summary The discussion on this issue is quite long, consider jumping straight to a summary comment: https://github.com/nvaccess/nvda/issues/4896#issuecomment-325895839

[nvaccessAuto]

Comment 1 by jteh on 2015-02-09 02:21 Implementing this using mouse clicks would be potentially problematic. We'd have to override mouse clicks and we can't know for certain whether clicking there would normally activate something. I agree that the current solution is tedious, but I don't think clicking is the answer.

[nvaccessAuto]

Comment 2 by steverep80 on 2015-02-09 03:18 I definitely see your point on many users, especially those who are totally blind, running into a dangerous situation of not knowing whether the click is to link to or activate something or not. I can't say I fully understand the scenarios you'd be afraid of though since numpad-divide and numpad-multiply are already linked to mouse clicks. Could you give a couple examples? I'm sure there's a compromise here.

If the problem is just mouse vs. keyboard control, then a first step might just be a new NVDA+something command to combine steps 2 and 3 after placing the mouse (e.g. NVDA+shift+numpad-multiply might make sense for the desktop scheme).

[nvaccessAuto]

Comment 3 by jteh on 2015-02-09 03:33 Currently, we don't ever override the function of the mouse buttons. If I understand you correctly, you're suggesting that we do; i.e. clicking should move the browse mode cursor. In a web browser, clicking normally activates things. The only time it moves the caret is in an editable text field. If we overrode this, clicking might not activate something when it was intended to by the author.

[nvaccessAuto]

Comment 4 by steverep80 on 2015-02-09 04:03 Ah, there might be a misunderstanding here. I'm not suggesting changing what a click would do in any way for a webpage (it should activate a button, link, or whatever as the author intends). In fact, NVDA will already do what I'm suggesting in those cases. For example, if I go to the NVDA User Guide page and left-click and hold on a link, but then move the mouse away so that Firefox doesn't actually open the page, the result is that the focus and cursor are now moved to the link location, and I can arrow down the page from there.

What I'm suggesting is that the same thing should happen when I left-click on a paragraph, heading, graphic, or any other inactive element, analogous to being in an editable field or document. It would allow low vision users to pick a focus point to start reading from on particularly difficult to navigate pages.

[nvaccessAuto]

Comment 5 by jteh on 2015-02-09 19:59 I understand what you're suggesting. The problem is that clicking just about anything can activate it. That is, anything can be "clickable", even some things that don't report as clickable for various reasons. This is why enter will always click in browse mode if all else fails.

So, doing as you suggest could break some of these edge cases. It might also unintentionally cause problems for other mouse functionality; e.g. scrolling or dragging.

Aside from using keyboard commands, the only thing we could perhaps consider is something like NVDA+click; i.e. hold down the NVDA key and click. That way, we can be certain it was intended to affect NVDA and only override in that case.

[nvaccessAuto]

Comment 6 by steverep80 on 2015-02-10 00:05 I may not fully understand those edge cases, but I'm the beginner at understanding the NVDA core and you're obviously the expert, so I'll heed to your judgement. Let's focus on the possible compromises. So we've got two so far:

1. NVDA+mouse-click overriding to move the focus/carat to the clicked position. In fact, you might extend that and say NVDA+left-click moves the focus/carat, and NVDA+right-click moves the navigator position.
2. Move the mouse to the desired position, and rely on a new keyboard shortcut to move the focus/carat in less steps.

I really like option #1, but I'll play devil's advocate and say it's sort of a conflicting paradigm considering NVDA+numpad-divide is already mapped to move the mouse to the current navigator object. On the other hand, it makes perfect sense since NVDA+numpad-multiply already moves the navigator to the mouse (and numpad-multiply is a right click. What do you think?

[nvaccessAuto]

Comment 7 by steverep80 on 2015-02-10 17:17 Sorry to not wait for a reply, but I was thinking about this overnight and I think a really consistent set of commands would be:

Command	Desktop Key	Comment
Right click	numpad-multiply	No change
Move navigator object to mouse	NVDA+numpad-multiply or NVDA+right-mouse-click	Same keyboard command with addition of the mouse clicking ability
Move mouse to current navigator position	NVDA+shift+numpad-multiply	This would be a change from the current NVDA+numpad-divide command
Left click	numpad-divide	No change
Move focus/carat to mouse	NVDA+numpad-divide or NVDA+left-mouse-click	Would be a change from current keyboard command and addition of clicking ability
Move mouse to current focus/carat position	NVDA+shift+numpad-divide	New command

I guess the only factor here is whether you guys would be willing to change the function of NVDA+numpad-divide, but this table seems like the most consistent approach and would really help out users who can still see and/or use the mouse.

[nvaccessAuto]

Comment 8 by jteh on 2015-02-10 22:29 Changing these commands like this would be fairly dangerous, since users who are already used to them would end up clicking when they didn't intend to, which could have disasterous results. I also don't follow how adding a mouse click solves your problem. You want to move the browse mode cursor, and as I've pointed out, clicking cannot and will not do this in many cases.

As I understand your request, what you want is just a single command to route the caret to the mouse, preferably something you can activate from the mouse itself.

[nvaccessAuto]

Comment 9 by steverep80 on 2015-02-10 23:31 Yes, I believe you understand the request to quickly move the carat to the mouse. I do not at all understand how you think the table of commands I proposed could have "disastrous" results. The only command that currently has a function that I'm proposing to change is NVDA+numpad-divide, and the worst that happens is someone moves the carat instead of moving the mouse, which might result in confusion unless the user read's the release notes, updated user guide, and/or pop-up after installing the version with the change, and simply adapts (something the visually-impaired are quite good at). The commands in that table are mostly as they are today, with functionality changes as follows:

1. Remap current NVDA+numpad-divide to NVDA+shift+numpad-multiply (an unassigned key)
2. Enhance current NVDA+numpad-multiply functionality by also allowing NVDA+right-mouse-click to do the same thing
3. Add a new function to move the carat to the mouse position, either via NVDA+numpad-divide (change) or NVDA+left-mouse-click (new)
4. Add new function to move the mouse to the carat/focus via NVDA+shift+numpad-divide (an unassigned key)

I would suggest sharing this ticket and our long list of comments with other users and developers to get additional perspectives, perhaps by posting to the mailing list?

[nvaccessAuto]

Comment 10 by nvdakor on 2015-02-11 00:11 Hi, A few thoughts:

• Some users could have reassigned these commands and may wish to do the same for the new commands, making the proposed commands invalid - invalid as in becoming invalid or unassigned.
• Some add-ons may use these commands in the future (I know one add-on which took over NVDA+Numpad Minus command for itself), potentially opening doors for more confusion - even more so if a user wishes to reassign commands for them on the main keyboard section, particularly on small laptops. One major concern we as NVDA contributors have is the fact that we're running out of commands, even more so when more add-ons are developed which doesn't check to see if a command is taken by the core or another add-on. Lastly, different languages have different commands for some NVDA core features, so it might not be a good idea to assume that the proposed commands will be accepted for now.
• What if the user may need to click (or route the mouse to) a control that is off-screen or invisible? This is more so on a site where a visitor may need to scroll to the invisible or object located off screen, particularly for controls which require mouse interactions nad if the interaction causes side effects (which isn't desirable (however, more and more sites are using dynamic contents and mouse-based interactions, especially gesture and touch-based interactions are gaining in popularity). One thing that caught my attention was the phrase which may imply (not to me for now) that visually impaired computer users are good at adopting to change. I'd like to kindly disagree: with limited resources and with "limited view and knowledge", visually impaired computer users have harder time adjusting to change, as our recent discussion on the NVDA users list regarding operating systems demonstrate. Part of the reason for some people to not adopt latest versions of software (particularly for professional applications) is the tendency for quicker changes - both outward (user interface, layout, etc.) and inwards (code, taking advantage of accessibility frameworks, etc.), and with current trends, this will accelerate, putting visually impaired users at a disadvantage. Although it may seem that changing the behavior of a command that seems trivial to implement is a good thing, when we go deeper into this issue (based on Jamie's comments above), it is a major change (take a look at ticket 804 (I believe) where users and developers had a heated debate on new laptop commands). But I can see why the proposed change might be good for low vision users (if we look at the current blindness statistics, we'll be able to serve more and more low vision users), but because of the fact that NVDA is primarily coded by two blind programmers with help from other visually impaired programmers around the world, I'd say that, for now, let NVDA be good at what it does now (screen reading), as well as have some room for collaboration from low vision programmers and users in hopes of asking NVDA and perhaps a magnification tool (built-in or third party ones) to cooperate with each other in the future. Thanks.

[nvaccessAuto]

Comment 11 by jteh (in reply to comment 9) on 2015-02-11 00:19 I think I misunderstood your intention. You noted "Same keyboard command with addition of the mouse clicking ability". I thought you meant you wanted the command to click the mouse as well, when I now think you just meant an additional (mouse) gesture for the same command.

Can you explain the need to change NVDA+numpadDivide rather than just using new keys for new commands?

It's worth noting that too many routing commands may actually confuse things further. Right now, everything goes via the NVDA review cursor. That is, you can route the review cursor to any other cursor and vice versa. That does sometimes mean multiple commands, but it also makes for a consistent experience.

It's also worth noting that you can set the review cursor to follow the mouse cursor (NVDA menu -> Preferences -> Follow mouse cursor). That way, you should just be able to move the mouse and then route the caret to the mouse, since the review cursor will always be at your mouse position unless you explicitly move the review cursor after you move the mouse.

[nvaccessAuto]

Comment 12 by steverep80 on 2015-02-11 03:30

To nvdakor

I'm not really sure how your comments contribute to the actual feature being discussed. The comments about addons or users changing commands applies so generally and is a separate topic if you want a feature to prevent or better handle this. And I'm not sure what off-screen or dynamic content has to do with this.

I do, however, understand the constrained number of key commands, and, yes, I certainly understand how it can be difficult to adapt to new versions of software, operating systems, etc. Give me a break though and be a little more positive.

Lastly, and this is where we really are going to disagree, is that "letting NVDA be" is not exactly a fantastic sales pitch to gain more users and grow the community into a compromised set of visually-impaired users, from starting to lose sight to never having known it at all. So, yes, I'm proposing a change that is primarily for low vision users, but so what? That may mean more donations and faster adaption.

Now, the one thing you did say that has to do specifically with this request is the mention of keys in other languages or on a laptop layout. I made no assumptions there, I'm simply not an expert in other language's keyboard layouts and don't use the laptop layout nearly as much as the desktop one. Please do chime in on how my proposal breaks those though. I'm an engineer and programmer by trade, and would like to solve that problem if there is one.

To Jaime

I think you hit a misunderstanding on the money again. I was not proposing a click anymore than already exists in the current command set.

The reason for changing the function of NVDA+numpad-divide rather than a new set of keys is what I was trying to illustrate with the wiki table. Right now, numpad-divide and numpad-multiply are left and right clicks, respectively, and NVDA+numpad-multiply moves the navigator to the mouse. To me, it makes perfect sense to add shift to that to do the reverse, i.e. move the mouse to the navigator. Then, you'd have numpad-divide to do the same things for the carat/focus, i.e. by itself is a click, adding NVDA moves the carat/focus to the mouse, and adding shift reverses it by moving the mouse to the focus/carat. Having NVDA plus actual mouse clicks be the same as NVDA+numpad-divide or multiply then follows directly since those are left and right clicks by themselves.

Your point about solely using the review cursor as the mover and move-to thing (for lack of a better term) is a good one, but I strongly feel it breaks down significantly in browse mode. If I move the navigator, single letter navigation and tabbing doesn't care until I get the carat there too. I just tested turning on having the review cursor follow the mouse. On the bright side, this does actually make it one less step, combining steps 1 and 2 from my original request description, but I still think my proposal is much better and obviously shorter. Keep in mind there's always a step 4 which is to either begin a "say all" or continue other arrowing or first-letter navigation from that point.

[nvaccessAuto]

Comment 13 by jteh (in reply to comment 12) on 2015-02-11 03:39 Replying to steverep80: Thanks for the explanation regarding the movement of commands. While it was never our intention that NVDA+numpadMultiply be associated with a left mouse click, etc., I understand that you're saying this would help with remembering the commands. I'm pretty reluctant to change commands users already know, though.

I just tested turning on having the review cursor follow the mouse. On the bright side, this does actually make it one less step, combining steps 1 and 2 from my original request description, but I still think my proposal is much better and obviously shorter.

I don't follow why it's much shorter. With what you're proposing, you'd move the mouse, then hit NVDA+numpadDivide or NVDA+leftMouse. With things as they currently are and follow mouse enabled, you move the mouse, then hit NVDA+shift+numpadMinus twice. It's a double press instead of a single press, but it's the same command, so I'd argue that's negligible.

[nvaccessAuto]

Comment 14 by steverep80 on 2015-02-11 05:47 Well, it must be getting too late here in the east USA, because I cannot seem to repeat my previous statement, or maybe I never truly tested it. I have the checkbox to have the review cursor follow the mouse enabled right now, and it it seems to be following correctly, i.e. I can use the text review commands to see that the points are the same. However, when performing a NVDA+shift+numpad-minus double press, the carat doesn't move. The only way I can get it to move is by doing a NVDA+numpad-multiply first, as in the original description. Could someone else please test this? I've tried it on 3 different websites and same lack of result. I even resorted to restarting my PC thinking I royally screwed up something.

[nvaccessAuto]

Comment 15 by steverep80 on 2015-02-15 23:54 Have we given up on discussing this? Regarding having the review cursor follow the mouse, and then double pressing NVDA+shift+numpad-minus not working, I'm prepared to open that as a separate issue as a bug instead of a feature request. That is, unless someone else tells me I'm crazy and it works fine. It seems to be a bug to me at this point.

Regarding the original feature request, if you are totally against the table of commands I proposed because it would change one existing command, that's fine I guess. I still think there is value in adding a feature to more quickly move the carat in webpages by clicking though, so what about just implementing the NVDA+mouse-click commands?

[bhavyashah]

This is a friendly reminder for @jcsteh, and maybe @Qchristensen and @feerrenrut also (as sighted individuals), to kindly consider responding to https://github.com/nvaccess/nvda/issues/4896#issuecomment-155334398.

[feerrenrut]

Since its been so long since there was activity on this issue, and it's quite long and time consuming to get familiar with, I'll attempt to summarise:

Use case:

For low vision nvda users who do not use mouse tracking and are familiar with using nvda with the keyboard, being able to easily (with a shortcut like nvda+click) move the focus and caret to the current mouse location could provide an improved (and faster) navigation UX.

Currently this is possible and requires the following:

1. Place the mouse at the desired position, making sure to be in "document review"
2. Use NVDA+numpad-multiply to move the navigator object to the mouse
3. Use NVDA+shift+numpad-minus twice to move the focus and carat

Benefits

Improved navigation of pages with poor semantics or few landmarks. It's worth noting that mouse users (mostly) have their right hand on the mouse, and their left hand free to access the keyboard. This should be considered when picking the keys that should be pressed in combination with mouse use.

Related issues

• 3004 (Move virtual buffer position to mouse position in browse mode feature)

• 6213 (Use braille display keys as modifier keys)

• 7106 (Add "report formatting under cursor routing key" command)

Technical considerations

• We would have to override mouse clicks
• We would have to allow gestures made by multiple input sources, this is something that has been requested for braille as well (EG NVDA+Braille Key)

  • 6213 (Use braille display keys as modifier keys)

  • 7106 (Add "report formatting under cursor routing key" command)

• There is a risk that clicking would activate something unintentional.

Further work

• Also providing NVDA+right mouse to move the Navigator object to the mouse.
• Provide a keyboard shortcut to move the mouse to the current focus/caret position.

This work would result in the following behaviour:

Command	Input reguired to trigger	Comment
Right click	numpad-multiply	No change
Left click	numpad-divide	No change
Move navigator object to mouse	NVDA+right-mouse-click	New
Move focus/caret to mouse	NVDA+left-mouse-click	New
Move mouse to current focus/caret position	??	New

This ignores some of the more controversial new / changed keyboard shortcuts suggested in https://github.com/nvaccess/nvda/issues/4896#issuecomment-155334389 :

• Add a keyboard shortcut for "Move mouse to current focus/caret position" as NVDA+shift+numpad-divide
• Change keyboard shortcut for "Move mouse to current navigator position" to NVDA+shift+numpad-multiply
  • This would be a change from the keyboard shortcut which is NVDA+numpad-divide
  • This provides nice symmetry between this and moving focus/caret with the same keyboard shortcut except using divide instead of multiply.
• Change keyboard shortcut for "Move focus/caret to mouse" to NVDA+numpad-divide
• Keep existing keyboard shortcut for "Move navigator object to mouse" as NVDA+numpad-multiply

There are some benefits to changing these keyboard shortcuts, especially in terms of the symmetry. The major disadvantage is that its a subtle change that is likely to confuse existing users.

Questions

What is the disadvantage to using mouse tracking? Or to put it another way, what is the advantage of this solution over mouse tracking? A low vision user could turn on mouse tracking and move the mouse to have things reported by NVDA.

[Adriani90]

@feerrenrut, I think if NVDA could provide a key command to move the focus to the mouse (i.e. NVDA + left mouse click), maybe this would simplify and solve this problem. I hope I didn't miss anything important from this discussion. In browse mode, the virtual cursor would be moved with NVDA + left mouse click. In focus mode, NVDA + left mouse click would move the focus.

[Adriani90]

Comming back to this, I think it would be much easier now to implement since we can split the virtual cursor from system focus with NVDA+8. cc: @leonardder

[Qchristensen]

Revisiting this again, this is definitely something we get asked about regularly. What I thought should work:

1) EITHER In NVDA's review cursor settings, have "Follow mouse cursor checked" OR move to desired location on page and press NVDA+numpad multiply (NVDA+shift+n) to set the navigator object to the object under the mouse.

2) Press NVDA+shift+numpad minus (or NVDA+shift+backspace) twice to move the system focus or caret to the position of the review cursor.

3) Press NVDA+down arrow (NVDA+a) to say all.

Step 1 works, but step 2 doesn't seem to do anything.

I found setting the review cursor to follow the mouse, hovering the mouse over the text to read and then pressing numpad plus (NVDA+shift+a) to say all in review works - but only in the current element, because the next element is a new object. I wonder if we could make that a command you could press twice? EG: "Pressed once, reads from the current position of the review cursor to the end of the current object, moving the review cursor as it goes. Pressed twice, reads from the current position of the review cursor to the end of the document, moving the system caret as it goes"

That way the workflow for someone wanting to read the text under the mouse would be (having setup the review cursor to follow the mouse): 1) Hover the mouse over the text to read 2) Press numpad plus twice to read to the ned of the document

I believe an argument could be made for changing the numpad plus command to simply move the caret and do a regular say all with one press (most times when people initiate any kind of "read from here" command, they generally mean "read from here to the end of the document or until I stop it reading".

[motobojo]

FWIW I'm a new user of NVDA and am finding this a significant stumbling block in my adoption of NVDA. You can find lots of detail and discussion on the groups.io forum for NVDA.
https://nvda.groups.io/g/nvda/topic/positioning_system_caret_with/106007331 I just wanted to provide support for some action to be taken on this topic. Thanks.

[pranavlal]

Hi all, I am adding my voice for support as well. Action needs to be taken since this would help partly sighted users as well as totally blind users when they are collaborating with sighted users. In addition, narrator already does this which means that there has to be a way to accomplish this.

[Adriani90]

@pranavlal what do you mean exactly? Actually you can enable navigator follows system focus, carret and mouse in NVDA review cursor settings. If all three are enabled, then you can move the focus / carret / virtual cursor in browse mode directly to the mouse by pressing nvda+shift+backspace once (focus) or twice (carret) on the fly, (latopt keyboard layout) or you can use the nvda+shift+numpadminus command in desktop keyboard layout. This works because when you move the mouse, the navigator already follows the mouse. So step 2 from the description is actually obsolete.

I think this is confortable enough for people, it is just a single keystroke.

Can you elaborate why developers should still invest effort in overriding mouse click or bringing the nvda+mouse click command? Is the nvda+shift+numpadminus or nvda+shift+backspace command not confortable? Note that you can change this keystroke to what ever you like in input gestures dialog.

[XLTechie]

@Adriani90 Did you read https://github.com/nvaccess/nvda/issues/4896#issuecomment-1427151822, where @Qchristensen explains how this fails?

I also had been thinking that what you described should be effective, but it wasn't in my testing.

For example, performing this in a long text segment such as the NVDA user guide, or even different parts of this issue page, and trying to initiate a say-all from the mouse position, simply doesn't work as anyone expects. Or apparently how it does in Narrator. You will get something, but it is not likely to be whatever you were just hearing under the mouse.

I think we might need a direct "read from mouse" command, to collapse all of these steps down into a single thing that actually works.

[Adriani90]

@XLTechie yes I also read that comment, but just adding a feature of nvda+left click or nvda+right click to move the focus / carret or the navigator to the mouse will not solve that problem.

1. In browsers (e.g. in the NVDA userguide), system focus can move only to interactive elements such as links. So try putting the mouse on a link and press nvda+shift+backspace twice, then nvda+a. Say all will work properly from the object under the mouse. It seems in this case the virtual cursor in browse mode follows system focus.
2. In the user guide, there is no system carret because it is not an editable area. But in an edit field, e.g. when writing a comment on github, the command nvda+shift+backspace twice works as well. NVDA reads the whole line when landing on the character under the mouse with the system carret and say all works as expected.

The only thing that might make sense is

• move the virtual cursor in browse mode to the navigator and to the review cursor: Currently tracking the virtual cursor works only in one direction, review cursor / navigator follows virtual cursor. But not vice versa.
• Move virtual cursor to the mouse: this does not currently work, because virtual cursor does not follow navigator / review cursor so there is no command for this.

A reading from mouse command will probably not work because the mouse pointer does not tell NVDA when a document / object / element starts and when it ends. So there is no definition of borders when using the mouse apart from the edges of the screen. So that's why we need the other focuses to move to the mouse.

[motobojo]

I tried "putting the mouse on a link and press nvda+shift+backspace twice, then nvda+a. Say all will work properly from the object under the mouse." This did not work as you said it would. It just started reading from the last point of focus.

When I type nvda+shift+backspace twice all it does is toggle single character reading on & off, then nvda+a announces nothing. In any case a down arrow or say all after that just commences from the last part of the document that it was reading prior to my scrolling, mouse cursor positioning and entering of these nvda commands..

Providing in more detail what I've done.

1. NVDA is off and the web page of interest (NVDA User's Guide) is open.
2. Open NVDA ensure NVDA is in the browser displaying this web page. FWIW the browser is Edge on my Win 10 system).
3. down arrow repeatedly until I'm well into the list of sections (table of contents). A link-rich portion of the web page. And NVDA is reading along as it should.
4. Scroll down using mouse scroll wheel. Get to a place that is still in this link-rich zone of the web page. Mouse over is reading the stuff under the mouse just fine. Position the mouse cursor over one of these links.
5. enter the directed key sequence (nvda+shift+backspace twice, then nvda+a)
6. Then down arrow or say all. NVDA reads from the the last position as in #3 above. It has not updated to the mouse cursor position.

Additionally (as to your 2nd point), I'm only referring to the situation when NVDA is in browse mode. When in focus mode NVDA interacts with mouse cursor positioning interactions in what I consider to be an intuitive manner (intuitive for a partially sighted person anyway).

I'm a neophyte NVDA user. I feel a little out of place commenting here amongst developers of NVDA. I'm only chiming in to add my perspective and apologize if this level of contribution is inappropriate or not very helpfu..

[Adriani90]

@motobojo which keyboard layout did you set in NVDA keyboard settings? For nvda+shift+backspace twice you must set laptopt keyboard layout. Otherwise you can use the desktop keyboard layout, and use NVDA+shift+numpadminus twice to move to the mouse, then NVDA+down arrow to trigger say all.

[motobojo]

Newbie mistake sorry. I'm using desktop. None-the-less I get the same ineffective results using NVDA+shift+numpadminus twice. I've tried that before to no avail and just tried it again with the same ineffective result.

---

From: Adriani90 @.> Sent: Thursday, May 16, 2024 10:04 AM To: nvaccess/nvda @.> Cc: motobojo @.>; Mention @.> Subject: Re: [nvaccess/nvda] Click to move focus & system carat in web pages (#4896)

@motobojohttps://github.com/motobojo which keyboard layout did you set in NVDA keyboard settings? For nvda+shift+backspace twice you must set laptopt keyboard layout. Otherwise you can use the desktop keyboard layout, and use NVDA+shift+numpadminus twice to move to the mouse, then NVDA+down arrow to trigger say all.

— Reply to this email directly, view it on GitHubhttps://github.com/nvaccess/nvda/issues/4896#issuecomment-2115636938, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AG6Q25YKV2FSOSFFSRKFPQDZCTKJZAVCNFSM4DYYA7N2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMJRGU3DGNRZGM4A. You are receiving this because you were mentioned.

[Adriani90]

@motobojo did you enabled the checkboxes in NVDA review cursor settings in order for the review cursor to follow system focus, system carret and the mouse?

[motobojo]

Yes. I have tried many combinations of those check boxes with this. But, the one I've used consistently is having those 3 enabled.

---

From: Adriani90 @.> Sent: Thursday, May 16, 2024 10:41 AM To: nvaccess/nvda @.> Cc: motobojo @.>; Mention @.> Subject: Re: [nvaccess/nvda] Click to move focus & system carat in web pages (#4896)

@motobojohttps://github.com/motobojo did you enabled the checkboxes in NVDA review cursor settings in order for the review cursor to follow system focus, system carret and the mouse?

— Reply to this email directly, view it on GitHubhttps://github.com/nvaccess/nvda/issues/4896#issuecomment-2115737656, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AG6Q257YTWQ5NP2FZG4UYZTZCTOVFAVCNFSM4DYYA7N2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TEMJRGU3TGNZWGU3A. You are receiving this because you were mentioned.Message ID: @.***>

[britechguy]

The fact is that NVDA does not and will not follow an actual mouse click as far as focus goes, and in the world of Windows being used without a screen reader, that's almost unheard of. If you click anywhere that focus can be placed, that is where the focus goes. The mouse is absolutely king when it comes to where system focus goes, be it selection, insertion point, etc.

That's what those requesting this change, and you can now include me, are asking for. It is to achieve what is the entirely and consistently expected behavior of any Windows user in response to a mouse click. It's one of the reasons most screen reader users eschew touching "the real mouse" (or mouspad) entirely because doing so can throw focus unintentionally if a click is also made.

A member of the NVDA Group, Dave Grossoehme, put it very well earlier this morning in topic, positioning system caret with mouse, on the NVDA Group:

"The sighted helper is trying to point out an item on the screen or maybe wrong spacing if in case of coding. In which case the sighted person is clicking the mouse to the point of the screen for the screen reader to have focus to land in the same point of the screen. Also why use extra time to change modes to make the screen reader cursor read the same information."

I added, in message #118471:

The use case would be almost precisely the same, as far as desired behavior, for the low vision user who's trying to throw focus elsewhere with the mouse.

I thought Thomas M's descriptions very early in this topic made it pretty clear that he wanted to be able to use the mouse to move NVDA's focus in precisely the same way it can be used to move Narrator's focus. If you are not (or at one point were not) a screen reader user and used the mouse extensively you know that it is pretty much given absolute primacy by Windows as far as changing focus of the system caret. It makes sense that this behavior carry over in those situations where anyone is using a mouse (and they're likely either fully sighted, and collaborating, or partially sighted, and attempting "big jumps with minimal effort") even when a screen reader is active. It is what any mouse user, real mouse user, expects and there is zero impact whatsoever if the real mouse is not involved.

There has got to be a better understanding that screen readers are collaboration tools as much as access tools when it comes to the workplace. And when it comes down to how real mouse clicks, made with the actual mouse, work they should work as any Windows user who routinely points and clicks would expect they should. That always means focus follows mouse click if focus can actually be thrown to the thing clicked upon. Narrator does this, and beautifully.

[jcsteh]

The fact is that NVDA does not and will not follow an actual mouse click as far as focus goes, and in the world of Windows being used without a screen reader, that's almost unheard of. If you click anywhere that focus can be placed, that is where the focus goes.

The key point is "anywhere that focus can be placed". If you click somewhere that focus can be placed, NVDA already follows that. If you click a text box, the system focus goes to the text box and NVDA will follow that. Same if you click a properly coded link or button. If you know of cases where this doesn't work, please provide a test case and steps to reproduce.

On the other hand, arbitrary text on a web page is not focusable, neither with a mouse nor anything else. That is where NVDA doesn't follow and what I believe this issue is requesting.

[britechguy]

On the other hand, arbitrary text on a web page is not focusable, neither with a mouse nor anything else. That is where NVDA doesn't follow and what I believe this issue is requesting.

About which I agree.

I would suggest a read through of the topic on the NVDA group. The territory has been covered, covered, and covered again, including a description of how Narrator handles mouse clicks and focus for reading (text on a webpage) versus what NVDA currently does. And it's behavior to mimic Narrator's that's being sought, and I understand why.

I don't know how much more explicit one could get beyond what has already been laid out.

If specific compare and contrast between NVDA and Narrator examples are needed, I can certainly try to supply same.

[jcsteh]

I don't think anything needs to be clarified further. I wanted to ensure that the information here was factual for reference, and "The fact is that NVDA does not and will not follow an actual mouse click as far as focus goes" is not correct as I described. NVDA does in fact follow clicks on things that can be focused. The issue is that it doesn't follow clicks on things that can't be focused.

[motobojo]

James is correct in the strict sense of all of the terms of interest. The real issue is with the impressions and expectation of the users. In so many ways NVDA does interact with the physical mouse as a sighted or partially sighted user might expect. That is, in a way that it does outside of the screen reader. That is great. In doing so it leaves that user with the impression that the mouse is indeed treated by NVDA as a clear provider of focus (and here please note that I am referring to focus in the most general sense). And that is where the problem lies. Even though there is no notion of "read from here" in browsing without a screen reader that user is led to believe that NVDA would interpret a (left click) over bits of the page that don't have clickable elements to describe to NVDA that the user is interested in this location as the new focus for continued reading. After all, if mouse over reading is on (as it is by default) NVDA is giving the user an unmistakable sign that NVDA is well aware of that location and correlating it well to the web page's content.

Oh, yes, and just to drive home the already belabored point ... Narrator does this updating reading focus to the mouse left click.

[Adriani90]

@motobojo

Oh, yes, and just to drive home the already belabored point ... Narrator does this updating reading focus to the mouse left click.

That's not entirely true. Lef click on an interactive element in narator activates that element. Left click on an edit field, the focus moves to the edit field if it is focusable, same as NVDA. The only thing that Narator does which NVDA is not able to do yet are following:

• The narator cursor is always synchronized with the mouse while moving the mouse (in this case Narator cursor is a combination of NVDA navigator and NVDA virtual cursor in browse mode). The syncronization between virtual cursor in browse mode and the mouse is what is missing in NVDA.
• Narator moves system focus while moving the mouse on interactive elöements. NVDA cannot do that because the system focus does not follow the NVDA navigator or the virtual cursor, and the virtual cursor does not follow the mouse.

[britechguy]

Distilled to it's essence:

Narrator: A mouse click on text moves the system caret for reading to the location clicked. Reading will commence from that point if any read command is then issued.

This behavior is desired in NVDA. That's what's being asked for.

And as far as any sighted person being able to be an active collaborator with a screen reader user, this pretty much has to happen. We who see, and use GUIs by point and click, will always try to redirect attention via point and click, and that includes in a webpage or other location where the text is not necessarily focusable in the conventional sense.

Software such as ZoomText can and does show word-by-word focus highlighting when reading text, anywhere, and I have to believe that text is loaded into a virtual buffer of some sort. There has to be a way to make a screen reader realize that if someone mouse clicks on a given word on a given screen, that the virtual cursor needs to move there whether the true Windows System caret does so or not. Some connection exists for ZoomText between the word it's reading and where that word is on the screen, including scrolling when necessary to keep what's being read visible on the screen. That sort of connection, sans the highlighting part, is essentially what's being sought. You click on text, that's where the focus goes, even if the text itself is not "focusable" in the way a control or edit box is.

[Adriani90]

@britechguy

Narrator: A mouse click on text moves the system caret for reading to the location clicked. Reading will commence from that point if any read command is then issued. This behavior is desired in NVDA. That's what's being asked for.

You seem to mix things:

• NVDA can already move the system caret when clicking the mouse while using focus mode in an edit field or when in an editor like Microsoft Word. However, NVDA does not read what is under the mouse when you click on a letter, but the say all command works as expected. @jcsteh I tested this at least in Chromium and MS Word in office 365 with focus mode. not sure if Firefox handles this differently. But in the end this is a separate issue.
• In Browse mode on web pages, which is the requested behavior here, there is no system caret. There we talk about NVDA's own virtual cursor.
• In Narator, you don't have to click to move the cursor, you just have to move the mouse and the Narator cursor moves automatically along. Performing the say all command in narator works as you describe.
• So what is actuall requested here, is to move the virtual NVDA cursor to the mouse when the mouse moves. This is what I tried to describe above.

If this is implemented, this could be an optional setting in browse mode settings called "Follow mouse cursor". When you enable it, the virtual cursor in browse mode would move along when you move the mouse which is the behavior we see in narator.

[motobojo]

If this is implemented, this could be an optional setting in browse mode settings called "Follow mouse cursor". When you enable it, the virtual cursor in browse mode would move along when you move the mouse which is the behavior we see in narator.

I'm good with that solution.