make object navigation user friendly by optional restriction for current window, locking feature and quick navigation keys

Adriani90 commented 7 years ago

Steps to reproduce:

Go in any application (browser ms office, Winamp etc.)
In Laptop layout press NVDA + Shift and start navigating by objects

Expected behavior: NVDA should allow me to restrict the object navigation for the active window or at least for the applications which are not minimized. Additionally, certain Actions should be more intuitive in this Navigation mode (i.e. adding quick Navigation keys for objects with custom context menus, easy mouse Routing and clicking key strokes, locking Feature like browse mode etc.).

Actual behavior: When you are in child object zone and press up arrow, you come to the next parent object zone. But when you press multiple times up arrow it brings you out of the window to the desktop and then you end up at the very left border where you have to go all way back, find your relevant parent objects and go to the relevant child objects. This is quite annoying in very complex applications like in finance. But also in other applications such as MS Excel can cause frustration if you always have to go all way back just because you have pressed one time too often the up arrow while trying to read the formula of a protected cell.

System configuration: NVDA version: all

NVDA Installed or portable: both

Windows version: all

Name and version of other software in use when reproducing the issue: all

Other questions:

Does the issue still occur after restarting your PC? yes

Have you tried any other versions of NVDA? yes, all

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/nvaccess/nvda/issues/7759 , or mute the thread https://github.com/notifications/unsubscribe-auth/AHgLkMNkl-hQZMwLTEu9rqOvGQIbCUl_ks5s2hcngaJpZM4QeHJz .

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

feerrenrut commented 7 years ago

@ruifontes I can understand that requirement. Settings have the downside that they are hard to discover, and annoying to toggle. Rather than a setting, how about the following:

When the object hierarchy reaches the limit of the application after the user presses NVDA+shift+up arrow, NVDA reports the object name again, and appends "at the limit of the application object hierarchy, press twice to continue up the hierarchy".

Brian1Gaff commented 7 years ago

Yes this has caught me out before as well. However in a way this echoes what can happen with nvda/b, its not all the time and I've no idea wat the controlling factor is for any window.

Sometimes I find myself wanting an nvda/b that simply steppes to an actionable object and stops to allow maybe an action say nvda keypad enter, or carry on to next object as well.

LeonarddeR commented 7 years ago

@feerrenrut commented on 15 nov. 2017 06:43 CET:

When the object hierarchy reaches the limit of the application after the user presses NVDA+shift+up arrow, NVDA reports the object name again, and appends "at the limit of the application object hierarchy, press twice to continue up the hierarchy".

Sounds interesting, but please limit this only to simple review mode in that case.

Adriani90 commented 7 years ago

I would add a seeting to the settings ring which says object navigation and you can either restrict it to the current window or to maximized windows or to let it unrestricted. That would let the people decide how they like it.

Robert-J-H commented 7 years ago

Hi Let me restate the problem: You're on a control and navigate up the tree until you're e.g. on the desktop object. The goal would be that navigating down (exclusively) brings you back to the control that you've started from.

Restriction won't solve this issue because:

the traversal down can you place at the left most position (child) at any level, e.g. navigating down from the application window brings you to the system menu and you would have to skip title bar and menu in order to be back at a pane or such.

The solution that comes to my mind is memoization with help of a simple stack. Imagine a deque (two-sided queue, from collections) for that purpose.

initialized to a maximum amount of entries (e.g. 20)
a single variable holds the current object, that is the parent node for all children that will be added to the stack later, let's call it scopeVar.
each time you move up, the child (ID) that you're coming from is pushed to the stack (right side) and the scopeVar updated.
when you move down, the right item is popped and the scopeVar updated again.
a deque will automatically pop items on the left side when the maximum is reached--no bookkeeping necessary there.
moving to a sibling does nothing
if you move up or down from a sibling, the following will happen:
1. the current object will be compared to the scopeVar. which gives of course False.
2. The deque will be cleared and all begins anew. This means that the scope and the children are all intact as long as you move only sideways. Getting back to the object that is stored in the scopeVar is therefore possible and moving down remembers the original traversal. However, moving up from any other object (after horizontal or horizontal+downwards movement) will reinitialize the deque or stack respectively.

Robert

didiercolle commented 7 years ago

"However, moving up from any other object (after horizontal or horizontal+downwards movement) will reinitialize the deque or stack respectively" Isn't reinitialization pretty drastic after keeping all that state? Instead of keeping such stack, why not analyzing when moving down the path to the focus object: if from the current object the focus object is also down, navigate to the child along the path to the focus object, otherwise fall back to the default behavior (navigate to the first child). In case there are use cases for which the path to the focus object is not good enough, would creating a stack of breadcrumbs make sense? Of course, the problem arises again: how to manage this stack: mainly how to clear/reinitialize the stack? For example, moving back tot he focus object (nvda+backspace) or executing the behavior of the current navigation object (nvda+enter). Perhaps navigating away from the focus object is good enough?

Robert-J-H commented 7 years ago

Keeping track of the focus ancestors is more costly than keeping this simple stack. Remember, we are only talking about child indexes and the stack could therefore be as simple as [3, 1, 0, 7] with a reference parent (scopeVar or current Navigator object) "Desktop". We are essentially not interested in getting back to the focus object but rather to the navigator object that the traversal has been started from. For instance: The focus might be on a toolbar. You now move left/right and down to e.g. an edit field (per object navigation). You might now want to go all the way up to the desktop object and back down again. The object that you want to eventually land on is the edit field and not the focus object. Note that the navigator does in general follow the focus but not the other way around. In other words, the focused object is still the same and returning to it is just a matter of e.g. tab and shift+tab.

Also, it might be desirable to announce the sibling index when moving down e.g. "3/4" or "sibling 3/4" if the child index is not equals to 1.

As I see it, four things are necessary for this scheme:

a preference setting
navigate up command (pushes "childInParent to the stack)
navigate down command (pops last item and sets navigator object to that index)
keep track of the reference parent for those actions. Negative comparisons will reset the stack on up/down.

Robert

jscholes commented 7 years ago

You're on a control and navigate up the tree until you're e.g. on the desktop object. The goal would be that navigating down (exclusively) brings you back to the control that you've started from.

I don't think this is restating the issue. Reliably retracing your steps after traversing the object tree isn't what's in scope here. Rather, @Adriani90 would like a ceiling to be imposed on object traversal to prevent accidentally moving outside of the currently-focussed application's hierarchy.

What about applications with multiple logical windows which are direct children of the desktop object? You need to navigate outside of the current application's object tree to find them.

For what it's worth I think this should probably be implemented as an add-on as the audience and potential use case seems a little bit confused. We can certainly argue that, when you accidentally head too far up the object tree and end up outside the current application it can be irritating to have to find your way back down again. On the other hand, the same could be said for applications like Firefox with complex object trees. If you move up to the title bar object in Firefox and then want to work your way back to the document, it's quite an involved effort. So some users might want to impose a limit when in Firefox which restricts them to a certain level, rather than imposing a hard limit at the application boundary. It's a behavioural change which could provide a productivity boost in certain situations but I'm not convinced it applies on a particularly wide or consistent level to make it into core.

If this was to be implemented in core I think a setting to turn it on and off would be fine. I agree with @feerrenrut that settings aren't the best thing for discoverability but realistically, how many users are going to turn this on? The intended audience would probably go looking for it, as with the simple review check box in the review cursor settings dialog.

Some clarification about who is going to use this and why would be welcome.

Adriani90 commented 7 years ago

Hey James and all,

I wish I could technically contribute to this topic in order to make my point more precisely. I am in learning process and so I hope I will be able anytime in the future.

The question is not who would use it. Because if we start thinking like this than many functions which are really usefull would not have been developed at all (e.g. the object navigation itself). The question is how much productivity we could bring up by such a restriction? In any case more than at this moment where you are brought out of your work window everytime you press up arrow to often. In my case, I work with a very complex software and I know how the tree is built. But if I press up arrow to often I land on desktop and I have to find the right window again. I have lots of them open and after I found the window I have to find the right part of the tree again which is very annoying sometimes.

I think if people will teach beginners to work productively with this feature than it improves accessibility of several applications significantly. And of course, averyone should be able to turn it on or off.

However, I have an idea which I don’t know if it is feasible but here it is:

Imagine you are in a climbing labyrinth where all ways are parallel or adiacent to each other. There is no way which crosses another one diagonally. Now imagine the left part of the labyrinth is longer than the right one.

So if you go up all the way on the right side you will end up under a ceiling. Now you have to turn left to go to the left side of the labyrinth to be able to pass that ceiling. The 90° structure of the crossings makes it easy for you to find the way up and down because you know where you will land when you go up and down.

Now try to reproduce this in a window of an application. As of now, the ways in the labyrinth are somehow diagonal to each other so that when I press up arrow to go up, the navigation in the labyrinth does not go just up but also left. When I go back down I land on another way which is not related to the way where I came from. And to complicate it a bit more, when I go up to far it jumps some walls and brings me anywhere in a corner where I have to go all the way back.

Does the comparison make sense to you somehow?

I guess it is quite complicated to make NVDA recognize the object structure and to convert it somehow to a parallelical structure. But a window restriction would at least ease the navigation.

Best Adriani

josephsl commented 7 years ago

Hi, I think an add-on would be better suited for this at the moment. After that, once it gets traction, we can ask NV Access to consider including the add-on code into NVDA. Thanks.

Adriani90 commented 7 years ago

James could you please give me an example of such an aplication where multiple logical windows are direct children of desktop object?

jscholes commented 7 years ago

James could you please give me an example of such an aplication where multiple logical windows are direct children of desktop object?

Sure. Some common examples could include a single web browser session with multiple windows open, or an email client like Mozilla Thunderbird displaying a folder in one window, a message for reading in a second window, and a message open for editing in a third.

Context dictates the extent to which these windows may or may not be connected, for example if I have an email message open in one window and I'm editing a reply to it in the other, those windows are logically related. If I have the Python documentation open in one Firefox window and a live stream of a tennis match open in another, they're not contextually connected other than by the fact that they're running within the same process.

The question is not who would use it. Because if we start thinking like this than many functions which are really usefull would not have been developed at all

I understand what you mean, and to a certain extent I agree. But my point was not to question the usefulness of this feature. Rather, I'd like the scope of its usefulness to be clarified and refined. Understanding the use cases of a piece of functionality can help you implement it properly, while avoiding a negative knock-on effect for the people who don't care about it.

As of now, the ways in the labyrinth are somehow diagonal to each other so that when I press up arrow to go up, the navigation in the labyrinth does not go just up but also left.

This makes sense. Unfortunately, in a physical structure you have a way to fill in the gaps so that the whole thing doesn't fall apart, and so that someone can navigate it logically. You can lay some bricks or whatever so that you go up the right hand side, then across to continue going up the left hand one. But in object navigation there is no such filler. An object only has one direct parent, and the level at which that parent resides won't necessarily have as many objects as the parent has children.

As an aside, if you'd like NVDA to remember the pathway that you took on your way up the object tree and then refollow it on the way back down, that's a very separate proposal to this one and should be submitted as another ticket. It sounds like a good idea but with a lot of potential edge cases e.g. what happens if one of the objects along the way no longer exists?

Robert-J-H commented 7 years ago

Note: This plug-in is obsolete See further down for a newer version EnhancedNavigation-1.0-dev.zip

Hi I've written a global plug-in As a proof of concept.

I'm pretty sure that it solves one aspect of the described issue--namely accidentally leaving the application without straight return path. See add-on help for more details.

Best regards Robert

Adriani90 commented 7 years ago

Oh my god! It works! Thank you thank you thank you! Really great job. It is so easy now to navigate the screen.. I think this is a big step towards more accessibility. We really have to stay on this.

I think in combination with screen review, browse mode and usual navigation mode NVDA is now the screen reader with the best accessibility features. We just have to make them as easy for the usual users to put in practice as possible.

Thank you for your great job Robert.

I would not close this issue yet because I think there could still be some interesting discussions on the restriction thing.

Adriani90 commented 7 years ago

I have noticed one thing: When I go up and down there is anything fine. But if I go up and then one left then back right and then down, NVDA does not bring me back to the last position where I pressed up. Is it possible to tell NVDA to remember all vertical paths I navigate? This would trigger NVDA to bring me to the position of the last child I have visited in each parent. Does that make sense?

josephsl commented 7 years ago

Hi, accessibility has nothing to do with restricting object navigation. Accessibility has to do with developer attitudes on making their apps work for people with disabilities. Thus I’d like to disagree with the above statement. Thanks.

Adriani90 commented 7 years ago

Yes but you also can improve accessibility for users through giving them better possibilities to reach objects which are not easy to access like in Outlook or other applications. In my view accessibility has to be considered from both perspectives. App developers and screen reader developers.

Adriani90 commented 7 years ago

ok, my statement may be a bit exagerated but I think developers could make their aplications more accessible if they know all navigation modes in screen readers. Every one tries to make its app accessible by saying wow, you can navigate by pressing tab through the whole application (like SAP for example). But through supporting other navigation modes an ap can become more accessible indeed.

Adriani90 commented 7 years ago

And if diferent navigation modes become very intuitive for usual users to put in practice then they will combine them in an optimal way for their individual cases.

Robert-J-H commented 7 years ago

Hi Adriani I'm glad that my concept goes in the right direction. Here is another version that does hopefully work better after moving left/right and back down. There may still be fringe cases where it falls back to the first child. It is a bit tricky to create an unique key for an object.

EnhancedNavigation-1.0.1-dev.zip

Robert

feerrenrut commented 7 years ago

Thanks for the proof of concept @Robert-J-H. This does seem to provide a solution to the initial issue, albeit a slightly more complicated one :wink:

One concern we here at NV Access have, is the risk of disorienting the user. The current solution forces the user to build a mental map of the objects. The risk of jumping straight to the last visited child (when firstChild) is called is that the user is unaware they are not actually at the first child. They may be unaware that there are more children to the left of the current child. One suggestion would be to announce the child number in this case. This is complicated. If we went down this path then we would likely want to do the same for going to the parent object too. This opens a can of worms.

Another other concern is the lifetime of this history. Both for keeping alive objects longer than necessary, and because the history could interfere long after the user has forgotten about the last call to parent to arrive at a certain object. Now users must always remember to explore siblings by going left and right.

Adriani90 commented 7 years ago

Roberts solution works quite well. I aggree that it can be hard for users to adapt to this art of navigation. However, it brings huge productivity for people who get used to it.

I have a potential solution in mind how to ease the navigation by objects: I would propose following steps:

I am wondering how screen review has been adapted to work just for the current active window. Would the technical aspects be applicable for Object navigation as well? Of course it should be an optional setting, let's say a keystroke for activating and deactivating the restriction
see issue #7749. By locking the object navigation you are now able to introduce navigation keystrokes to this navigation mode (e.g. in laptop layout ctrl + page up to go to the very first object level of the window, ctrl + page down to go to the last object level in a window, ctrl + home to go to the first child in the group, ctrl + end to go to the last child in the group and so on). Say for example we atribute the command NVDA + shift + spacebar to lock and unlock object navigation, the full keystroke for going to the first object level would be NVDA + shift + ctrl + page up or down, home or end. There wouldn't be any conflicts with other keystrokes
Add a key stroke for accessing the system menü or the part of the window where you can minimize, maximize or close the window, for sidebars, ribbons etc.
Activating object navigation means you are not moving mouse or keyboard cursor. But by pressing m in object navigation would drag the mouse cursor to the current object since the actual keystroke for that is NVDA + shift + m (laptop layout)
When object navigation mode is activated, the enter key would simulate a left mouse klick, so to activate an object you press m and then enter
Improve mouse draging by NVDA + shift + m (in some applications in object navigation the mouse cursor does not exactly follow the object)
Other navigation keys like in browse mode would make the orientation easier (e.g. c for objects which are combo boxes, e for objects which are edit fields, b for objects which are buttons and so on. One advantage would be that you can read the label of a field by pressing left after navigating to a edit field in an application which is actually not very accessible (e.g. Java applications where the label is a canva). Additionally, object navigation makes NVDA react faster in some applications (e.g. Firefox 57).

didiercolle commented 7 years ago

just speaking from a user perspective...

One concern we here at NV Access have, is the risk of disorienting the user. The current solution forces the user to build a mental map of the objects. Is your intention to suggest the user should not build a mental map of the objects? At least myself, when I start navigating (in whatever context: object navigation, file system navigation, traveling outside/with public transport, etc, etc) my top priority is building a mental map of the environment and positioning myself on this mental map. When I fail to do so or when I loose track, I consider myself disoriented. I can't understand/imagine how someone else can't be/feel disoriented without having such mental map, for sure when they are visually impaired. Of course, that is my very personal perspective....

... is that the user is unaware they are not actually at the first child. They may be unaware that there are more children to the left of the current child. I find this personally a very weak argument. If you are used to this behavior, you know you have to try to go left and right when you want to explore your siblings. Again my personal perspective: why would you want to "explore" your siblings? To build a mental map of the objects... once you have this mental map (remembering it after several times using the app, based on the context such as an alphabetically ordered list, etc), you perfectly know whether to go left or right.

Mainly in the case of long lists of siblings accidentally going up (cfr. accidentally ending up in the desktop window), finding your position back is very annoying because (again speaking for myself) you want to go fast and miss the sibling you were looking for and it takes away your brain effort from building/dealing with the mental map of the objects (as mentioned before my top priority when navigating). My favorite command is nvda+backspace to get rescued from such situation, bringing me back to somewhere where I have been already and have explored the environment, allowing to walk over the previous path again. Bringing me back to the position where I was in the list instead of throwing me at the beginning of the list, does not look too bad to me: I was there in the first place, so I can continue my navigation journey (exploring the neighborhood to build the mental map, or very conscious and targeted navigation) while now nvda decides in my place I want to be at the beginning of the list of siblings. (well, now when I want to go to the beginning of such long list, I abuse the current behavior by going up and down again.... of course, that does not work to jump to the end of the list of siblings...)

Speaking about disorientation and failing to build a mental map, I am anyway not a big fan of object navigation because (1) the object hierarchy does not very logical to me (due to how the app is implemented), and (2) changes while navigating around that hierarchy, whereas I do not expect this happening (should I open a new issue for this?), disallowing me to create a mental map of the hierarchy and as such leaving me feeling disoriented.

Just two examples of that changing hierarchy behavior:

In adobe acrobat reader dc: starting from the document, going left multiple times goes in my case as follows: tools button, home button, button, application menu bar, close button, restore button, minimize button, tools grouping, learn more button, store and share in the document cloud button, learn more button, store and share in the document cloud button, learn more button, .... being trapped in an infinite loop never reaching the left side; trying to go back right leaves me stuck at that learn more button and no objects seems to exist at the right: damn, where I am, where did I end up, why can I not back track? Not very helpful in building a mental picture.... luckily with my favorite command nvda+backspace I am thrown back on the document object. Starting from the focus/document object, going left lands on the tools button, going right lands on the toolbar (huh, I just went left then immediately right and I am not back where I started????), continuing one step right lands us on the show navigation tab strip button, going back left lands me again on the tools button, and I can keep circling in this 1-step-left-2-steps-right figure. So am I circling around the document object where I started in the first place? Not really because when I go 1 step right from the show navigation tab strip button, I end up on the scroll bar 0 (apparently somehow the right edge, although trying to go right repeats scroll bar 0 and not announcing no next), moving back left 1 step from that scroll bar I am again on the document object, and now I can circle around in a 2-step-left-3-step-right figure.... Starting from the document, going 1 step up and going back down 1 step, I have to move completely to the right (until reaching that scroll bar), before I can reach back the document object by going left... not at all very intuitive.
In Thunderbird: moving up to the top window "Inbox - Mozilla Thunderbird window" and going back down 1 step lands u on "system menu ...", going right a few steps gives "minimize button", "restore button", "close button", "Inbox - Mozilla Thunderbird busy" (notice the busy, whereas the parent contained window at the end), before getting a no next; trying to go left announces no previous; to get to that system menu and its buttons, I have to go again up and down again, and once again I can circle around in one direction 1-step-up-1-step-down-4-steps-right.

I thought GUI objects were organized in a undirected graph that is a tree. However, it seems that sometimes it is more a directed (planar??) graph in which you can circle around (which is obviously impossible in a tree graph). I find this much more confusing and disorienting than that I would have to get used to checking siblings on my left when exploring the object hierarchy, as I fail in my top priority building a mental picture of the object hierarchy.

Adriani90 commented 7 years ago

Didier, did you see the issue #7782? There is also a part of an application where object navigation gets stuck, namely in MS outlook.

As far as the lost position is concerned, did you try Robert's solution?

I would like to add that object navigation will never allow an user to build a full map of objects. Applications are to complex to remember everything. But you can reach objects much faster than pressing tab 20 times or so. And you can access objects which you could not access by pressing tab at all (e.g. MS Office). If you know the application you know approximately where you can find a certain group of functions. And you will find the relevant parents quite fast in case my proposed solution can be adapted and improved.

feerrenrut commented 7 years ago

@didiercolle

Is your intention to suggest the user should not build a mental map of the objects?

No, quite the contrary. I believe it's important that a mental map is built up. NVDA should aid this process, not hinder it.

you know you have to try to go left and right when you want to explore your siblings

The main issue I have with this, is that sometimes it's necessary to explore left and right, sometimes only right, and sometimes only left. This will make the process of exploring much slower. Rather than override the "first child" command, my preference would be to introduce a new "back" command.

Overriding the "first child" command as done in the proposed addon also hinders the users mental map building process because the sibling you land on can change each time, and there is no context as to where you are. For example while exploring a hierarchy, you go down from a parent, and across to a the 3rd sibling, push up several times, explore another part of the hierarchy, when you press down from the initial parent you are at the third sibling already. You have missed the landmarks of the first two siblings. I think there is a place for this, but it conflicts with the exploration use case. I think the use case described initially in this issue is worth supporting, but not by squashing others. Hence, I suggest it be a separate command.

As @Adriani90 has already mentioned, the descriptions of adobe and thunderbird sound a lot like the problem described in #7782, perhaps you could add a comment for each of these onto that issue. Please give clear (and as simple as possible) steps to replicate the problem as well as application version numbers.

feerrenrut commented 7 years ago

Just realised the previous two comments have referred to the wrong issue, please see issue #7782. I will edit this previous messages to no longer refer to #7789

Adriani90 commented 7 years ago

@Feerrenrut, could you please explain in more details your proposed "back" command?

In my view the overwrite of the first child is the best solution for the user to work productively with this type of navigation unless we improve it further for example by the steps 1 to 7 described above. I try to explain it: Imagine you are in windows explorer, go to the folder "projects". There are approximately 3.000 entries in it. Now you navigate to project "mantaro" which is entry 1.450. Now you enter that folder and go to "5. operative phase" and enter that folder. When you go back you will land back on "5. Operative phase" and one more back brings you on "mantaro". No one would like to land on entry number one of the folder. It is more confortable for the user to land on the folder he or she started from. Now if you click on "mantaro" again you will not land on "5. Operative Phase" which is ok because you know the structure which you have explored before. You can press 5 and land directly on "5. Operative Phase". But if we transfer Robert's solution on this, than you would land directly on "5. Operative Phase" because NVDA remembers the navigation path. Now you have the keystroke ctrl + home which brings you back on the first entry of the folder and I think this is needed for object navigation as well. Objects do not change as often as folder entries. But yes, you are right. The user should build somehow a map of objects. This would happen anyway if the user interacts with an application very often.

Best Adriani

Robert-J-H commented 7 years ago

@feerrenrut Thanks for your perspective, Reef. Comments inline. On 22/11/2017, Reef Turner notifications@github.com wrote:

Thanks for the proof of concept @Robert-J-H. This does seem to provide a solution to the initial issue, albeit a slightly more complicated one :wink: More complicated than what, restriction or navigating to the first child?

One concern we here at NV Access have, is the risk of disorienting the user. That's what we are dealing with here. It is very confusing that going up isn't the same as going down. This applies particularly to the movement out of root objects. I would as a minimum expect that NVDA remembers those, for instance the object below the desktop object or the panel below an (application/dialog) window.

The current solution forces the user to build a mental map of the objects. Try to make a mental map in e.g. Firefox. There are 20 levels up from the edit field I'm currently writing in... ;-) Fortunately, a lot of applications aren't that complicated. The risk of jumping straight to the last visited child (when firstChild) is called is that the user is unaware they are not actually at the first child. They may be unaware that there are more children to the left of the current child. One suggestion would be to announce the child number in this case. This is complicated. If we went down this path then we would likely want to do the same for going to the parent object too. This opens a can of worms. As you must admit, the navigation upwards circumvents currently this aspect gracefully by simply ignoring it... ;) Therefore, you're stuck with the same problem as you refer to in my proposed scheme, you still have to navigate left and right in order to get a mental picture. I wouldn't mind having the position spoken out, perhaps as an option in the object presentation dialog. However, the normal thing would be to announce the level on vertical navigation and the sibling on horizontal. This is in accordance with the table navigation paradigm, I think. Of course, the window hierarchy is not as simple in praxis as one would like, not a good old binary tree... In fact, we have multiple (sub-)trees on multiple pages. The challenge is to group those logically and to leverage the biased navigation down/left for those groups towards one that is expected by the user. After all, exploration shouldn't be the main priority. It is needed to familiarize oneself with a new application or to refresh the mental map. For production work, the memoized navigation is much better suited IMHO.

Another other concern is the lifetime of this history. Both for keeping alive objects longer than necessary, and because the history could interfere long after the user has forgotten about the last call to parent to arrive at a certain object. Now users must always remember to explore siblings by going left and right. Which is already the case for navigation up, as I said above. I can hardly imagine a situation where delayed interference comes into play. Object navigation begins normally at a low level (e.g. a button) and navigation up resets the history for that connection. It may be that some downwards target is stored if you move to a sibling and then down. However, I bet that this is in 95 % of cases exactly where you wanted to go to in the first place. The exact life cycle and structure of the history are implementation details and beyond the "proof of concept" phase.

I might add the index of the sibling in the next version and also assign a new shortcut to navigate down (although the "go to first child" should actually be NVDA+1. in order to reflect the down/left motion :wink: Joke aside, having both commands at the finger tips might be a good idea for proper comparison.

The unfortunate thing is that we don't have enough feed back from users on which behaviour they would expect/prefer. Until we can gather that, we're at a dead end, I'm afraid.

Regards Robert

feerrenrut commented 7 years ago

@Adriani90 The "back" command I refer to would operate in much the same was as the proof of concept by Robert. It would be assigned a new key combination, and should undo any previous navigation actions.

overwrite of the first child

I think many users would be upset by this change.

No one would like to land on entry number one of the folder

Again, I would question this. Sometimes, it's preferable to land where you last were, however, imagine you were looking for another sibling of "mantaro" and you do not remember if it is before or after it. You will have to go left 1450 times, so that you can start your search from the beginning. As you mentioned we could introduce another command to do this, however this would break the behaviour for existing users.

@Robert-J-H

More complicated than what, restriction or navigating to the first child?

More complicated than restriction / requiring the user to acknowledge they are leaving objects with the application. However, this issue seems to have now morphed. The common case of going up too many levels and exiting the application hierarchy, is also covered by providing a mechanism to "remember the child that was being reviewed when returning from its parent" (got a better name for this?). This covers other cases two, like applications with many controls at one level.

One concern we here at NV Access have, is the risk of disorienting the user.

To explain this concern further, and perhaps also why it was built this way, I'll try an analogy: You come down a staircase (down/numpad 2), you find yourself in a hall way with twenty doors (siblings) on the wall in front of you, stretching away to the right. You walk along the hallway (right/numpad 6) and read the names from three of the doors (o/numpad 5). You go back up the staircase, exiting the door at the top, and find yourself in another hallway (up/numpad 8).

You decide to return down the stair case. The current NVDA object navigation behaviour, is that the bottom of the staircase is still at the beginning of the hallway, as it was the previous time. Alternatively, with the behaviour in the prototype, the bottom of the staircase is now in front of the third door.

When exploring from top down this behaviour makes a lot of sense. As mentioned, users are often going in the other direction.

As you must admit, the navigation upwards circumvents currently this aspect gracefully by simply ignoring it... ;)

Yes, but this is not an argument for this behaviour. With the current behaviour, it is easy to return to the start of the siblings on any given level, by going up then down again.

I'm not opposed to the proposed functionality, I think there are plenty of cases this would be preferable. I just want to ensure that the use-cases served by the current behaviour are not broken. Because of this, my preference would be to introduce this as new functionality, rather than change the existing functionality. Those who are happy with how it works currently are not affected, as well as those looking for something else.

An alternative path forward would be to get further feedback, and ensure that users would accept this change.

The exact life cycle and structure of the history are implementation details and beyond the "proof of concept" phase

This is true, but the implementation details that affect the UX of the feature will need to be nailed down.

The unfortunate thing is that we don't have enough feed back from users.

Short of pushing the prototype more, a clear summary of the use-cases we have explored in the discussions on this issue, and how they will be addressed would be helpful. At this point, this issue is getting fairly long.

Adriani90 commented 7 years ago

Please comment there in case I forgot any important aspects.

Adriani90 commented 7 years ago

I would like to post here the use cases I'm using this navigation mode and some use cases that I know. Maybe this makes it more understandable why it is so important:

I don't want to make the impression that I really prefer this navigation mode over the others. It would just be a significant improvement in how we deal with aapplications. In my case and in some other cases that I know, object navigation is used for following: In Excel to see the formula of a cell whenn the cell is protected, in Browser to reach a notification when f6 is not working (z.B. Firefox), in Java applications to better understand how formfields are labeled and where NVDA gets stuck in tables and tree views and you don't come out by pressing tab or shift tab, in Windows 10 apps where you would have to press 20 times tab to reach the function you would like (sometimes tab or shift tab does not work at all), in malware bytes and other antivirus applications where you have live regions, during the inaccessible installation process of Visual studio 2015 to see which components are activated and which are not, in MS Outlook to simply deal with the calendar, and and and... Unless this mode is not improved and stabilized, you will not get your desired user stories because lots of users don't spend much time in understanding the issues with it. I hope you can understand what it would bring. In any case, no negative effects for the user.

Adriani90 commented 7 years ago

Other use cases have already been explained, for example by @didiercolle, @Brian1Gaff or others in other issues (e.g. Adobe reader, Thunderbird, Audio applications). ABBYY Finereader is also one of the use cases that I know of and I deal with as well. Because the menu is incredibly slugish, users prefer to reach their function by using object navigation.

josephsl commented 7 years ago

Hi,

Something to think about (something I think I brought up somewhere before). I do know that some of the things I wrote below may not sink in first, but I believe this is a necessary process to go through:

Is a picture the only thing you see on a visa or a passport? A typical visa and/or a passport contains not only your picture, but your name, citizenship status and what not. I think if we focus too much on the front picture, we won’t get anywhere ; we also need others to help, other information to complete this passport to accessibility. We have users, willing developers, and visionaries, but we’re missing two things: app developers who’ll understand what we’re saying and have willingness to learn a thing or two, and willingness from all parties to negotiate and do the work, not just theorize.

I do understand that this can come off as a criticism, but believe me: When I first joined NVDA community in 2012, I suggested that we make NVDA better and offered what I then thought were great suggestions. Back then, I was just few months into being the Korean translator, and didn't have all the knowledge and wisdom I possess today. In a way, I myself fell headfirst to this trap of making NVDA great without thinking about other dimensions, and I came out learning more than to theorize and never think just about myself and my own language community.

I'm proud of the fact that we have folks willing to represent a language community and their concerns. But please tell the German community the following:

Colors are missing from the painting you're trying to paint.

I have no final authority on this discussion - ultimately, it is up to NV Access and the community to decide. But as a respected community mentor, I'd like to ask you and the German community to reconsider your positions regarding superiority of object navigation over other methods.

Also, for future reference, what you are really describing isn't accessibility at all. Deep down, I can tell that it ultimately comes down to user experience and usability, not accessibility. Please remember this: usability is NOT the same thing as accessibility. Accessibility is more than usability, and usability is just a part of accessibility.

Another thing to remember, and also is a part of accessibility: individual programmers have power. Not only users have power, developers do have power, especially those who are willing to make the world a better place. In terms of accessibility of apps in general, the "real" people you need to talk to aren't screen reader developers unless users want someone to provide expert testimony on their behalf.

One more thing I'd like to sincerely point out (and plead with you and future issue reporters who do have skills): please become practical at some point, not write theories on a whiteboard.

My apologies for some strong statements on this public venue; but I believe it is time to say the above things to serve as a wake up call for some of us.

Thank you.

Adriani90 commented 7 years ago

Well, as one of the maintainers of german community, I conducted surveys in the last months where lots of users took part. I taught many people how to work productively with NVDA. It is all based on practical use cases. Nothing at all on theories. I don't have enough knowledge of all accessibility theories to discuss these points I must admit. But as long as usability and user experience is part of accessibility, then for me as an user this is accessibility as well. The best results can be achieved if more perspectives work together. And I don't say that app developers should continue to program their apps inaccessible just because screen reader developers would try to cope with it. Finally, as I said, this issue should not give the impression that object navigation should be superior to other navigation modes. Not at all. But it would make users and developers more flexible in dealing with applications.

And if improvements are technically not possible because of different aspects, than I hope to learn from you all experienced developers where the problems are. I don't want to smash down other NVDA features or to neglect other issues. But I think it is worthy to raise discussions about such topics between users and developers.

josephsl commented 7 years ago

Hi, justifiable. I’d like to say though that rather than programmers making apps inaccessible for eternity, they should at least invest in making things work for many with minimal screen reader hacks. Thanks.

derekriemer commented 7 years ago

I think some of these object navigation improvements might be welcome. However, I think a browse mode in desktop, or a sort of global scan mode might even have more impact. My goal is to make doing your job easier and more efficient, and object navigation is certainly powerful, but can be easy to make mistakes on. Therefore, I think we should continue this discussion here to improve the UI of this feature. I think a restriction mode of sorts might be very beneficial.

PratikP1 commented 7 years ago

I’d like to see NVAccess consider the idea of a global scan mode similar to that introduced by Narrator. The discussion on this thread has seemed to be overly complicated thus far. Narrator’s scan mode has some limitations in that it doesn’t recognize context. Whatever the system, it needs to recognize context and behave accordingly. For instance, Narrator’s implementation fails to recognize that the user is navigating through a list view, providing verbose information and requiring too many keystrokes to navigate such a construct. An intelligent system could allow for scan level navigation while intelligently determining whether to activate itself. In addition, such a system should also enable add-on developers to engage its capabilities to provide information to users.

josephsl commented 7 years ago

Hi, would you believe if someone wrote the code for that already? Ultimately, it is something I and NV Access will need to discuss regarding donating yet another add-on to the Core. Thanks.

Adriani90 commented 7 years ago

Global scan or desktop browse mode or how ever we call that is a very good idea in my opinion. There is a very important productivity aspect which should be included in this navigation mode to improve user experience. And this is the ability to recognize grouped sections. i'll give some simple practical examples. Open MS outlook, press alt key and then one time tab. Now you should be in the ribbon "start", group "new". Now got to object navigation and press up arrow. You will land on the group bar. Gor right to "quick steps". It is much faster then trying to land there by holding down the tab key or to press tab and here everything in the menu until you finally land on "quick steps".

Another one you can see in the Outlook calendar. There are functions which are grouped as follows: New, Actions, share, options, search and so on. Every group contains functions. This part of Outlook is not visible to the user when navigating with tab or shift tab.

The last one is in Fire fox. You have 50 tabs openand want to simply explore which are open without loosing cursor position.

Now imagine there are lots of quite complex applications out there, especially in professional domains like finance or so. But in every such applications, functions are grouped somehow to improve the logical structure for sighted users. Well, blind users would profit alot if they could use these grouping information as well in a very confortable way.

So in my opinion, a global scan mode should include the ability to navigate between object types and interact with them (like in browse mode) and to explore grouped sections (like in object navigation).

Adriani90 commented 7 years ago

Another example is following: You have 20 open windows and would like to land on the 15th. You don't actually know on which number you currently working, so you press alt + tab multiple times to land on your desired window. Then you press multiple times alt + tab again to land back where you come from. The thing is that 5 times alt + tab does not bring you every time on the expected window. Well, by exploring the grouped section of open windows you can navigate very fast to the one you want.

didiercolle commented 7 years ago

an additional command sounds like a good compromise for the moment.

On 27/11/2017 2:34, Reef Turner wrote:

@didiercolle https://github.com/didiercolle
Is your intention to suggest the user should not build a mental
map of the objects?
No, quite the contrary. I believe it's important that a mental map is built up. NVDA should aid this process, not hinder it.
you know you have to try to go left and right when you want to
explore your siblings
The main issue I have with this, is that sometimes it's necessary to explore left and right, sometimes only right, and sometimes only left. This will make the process of exploring much slower. Rather than override the "first child" command, my preference would be to introduce a new "back" command.

Overriding the "first child" command as done in the proposed addon also hinders the users mental map building process because the sibling you land on can change each time, and there is no context as to where you are. For example while exploring a hierarchy, you go down from a parent, and across to a the 3rd sibling, push up several times, explore another part of the hierarchy, when you press down from the initial parent you are at the third sibling already. You have missed the landmarks of the first two siblings. I think there is a place for this, but it conflicts with the exploration use case. I think the use case described initially in this issue is worth supporting, but not by squashing others. Hence, I suggest it be a separate command.

As @Adriani90 https://github.com/adriani90 has already mentioned, the descriptions of adobe and thunderbird sound a lot like the problem described in #7789 https://github.com/nvaccess/nvda/issues/7789, perhaps you could add a comment for each of these onto that issue. Please give clear (and as simple as possible) steps to replicate the problem as well as application version numbers.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/nvaccess/nvda/issues/7759#issuecomment-347057105, or mute the thread https://github.com/notifications/unsubscribe-auth/AGAaJsG0LufCygJo8TZz9_W0ZqwquDDEks5s6hG5gaJpZM4QeHJz.

Adriani90 commented 6 years ago

I have slightly edited the issue.

Adriani90 commented 6 years ago

the addon Objpad would contribute to improve object navigation. At least it provides a locking feature. Can this be implemented in NVDA?

derekriemer commented 6 years ago

@josephsl wrote that.

Adriani90 commented 4 years ago

The addon will probably not be maintained in the future anymore. Maybe we can start discussion on this again somehow. @Robert-J-H do you have an update to your enhanced object navigation plugin which you created? See this comment: https://github.com/nvaccess/nvda/issues/7759#issuecomment-345857287

gregjozk commented 4 years ago

This, when will be implemented, it could be a JAWS Frame Viewer on steroides. +1 for it.

rkingett commented 4 years ago

Coming here from the now closed, scan mode, thread. I still have the same issue as described within post 1. Typically, when I invoke this mode, I want to explore/examine what is open and in focus. This is where something like narrators scan mode would come in handy. Also, when I try to explain object nav to casual users, they just don't get it, even with tutorials. Narrator's scan mode could be something we should borrow from, at least, have everything proposed in the above comments a setting to toggle on and off.

Qchristensen commented 12 months ago

Just noting that this is something we do get asked from time to time, and would be useful for many users. One idea I had, now that we have the ability to move to next and previous object (on the whole screen) in a flattened view - if object navigation was limited to the current program, if you moved to the last object in the current program and pressed NVDA+Numpad 3 / NVDA+shift+], it would move to the first object in the next program, essentially letting you have the best of both worlds. I am curious how many people use object navigation to (deliberately) move out of the current program in the first place? Is it common? I've known a number of users who have done so inadvertently and got lost, but I haven't had anyone tell me how they use it deliberately.

nvaccess / nvda

make object navigation user friendly by optional restriction for current window, locking feature and quick navigation keys #7759