search

nvaccess / nvda

NVDA, the free and open source Screen Reader for Microsoft Windows
https://www.nvaccess.org/
Other
2.11k stars 637 forks source link

Improve word echo to read the last added/edited word #8065

Open LeonarddeR opened 6 years ago

LeonarddeR commented 6 years ago

Steps to reproduce:

  1. Open notepad with NVDA word echo enabled, character echo disabled
  2. Type: "test"
  3. Press space: NVDA says "test"
  4. Press backspace three times
  5. type: "xt"

Expected behavior:

NVDA says: "Text", similar to how iOS word echo works

Actual behavior:

NVDA says: "xt"

Discussion

I propose this as a fix for #7812. Instead of keeping a buffer of entered characters, would it be possible to always announce the most recent entered word (i.e. the word before the cursor) based on the text info for text info implementations that are fast enough?

LeonarddeR commented 6 years ago

I almost have this working smoothly. However...

I already knew that different editors use different rules for word boundaries. For example, if you type 1>2>3 in Notepad, this is seen as one entity when you move across it with control+left/right arrows. In Wordpad/Word, all characters in 1>2>3 are seen as separate entities. Firefox distinguishes 1>, 2>, 3.

If we let NVDA speak the last word based on textInfo, behaviour will be different across implementations. I don't have a problem with that, as applications themselves work differently, it is not NVDA itself that causes this. However, it kind of breaks if we have only one assumption to find word boundaries (e.g. the one in speech.speech.speakTypedCharacters or in NVDAObject.event_typedCharacter).

Personally, I find the uniscribe implementation a bit weird. it treats 1>2>3, focus.appModule, 1.2 all as single entities. I propose making the uniscribe based implementation a bit more conservative, treating almost all non-alphanumeric characters as word boundaries.

@michaeldcurran @feerrenrut, could you please give your opinion about this? Why is uniscribe ever implemented as a replacement for the pythonic implementation that's used when useUniscribe is set to False?

LeonarddeR commented 6 years ago

We could also disable uniscribe only for this specific case. Still, I assume uniscribe was added for a particular reason.

michaelDCurran commented 6 years ago

Uniscribe should certainly be used in standard edit controls such as Notepad as that is what it uses internally. Not using uniscribe would mean there is a missmatch between NVDA's idea of a word, and where control+left/rightArrow jumps.

As for using uniscribe for other offset-based TextInfos where no other API for calculating word offsets exists:  they probably could be changed to not use it. But I'm not really sure of any good examples here.

LeonarddeR commented 6 years ago

Uniscribe is used in virtual buffer browse mode I belief, but that's a non-editable case. A more important case though is command consoles. My cmd prompt when i start cmd is: C:\Users\leonard de Ruijter>. When I type git and press space, NVDA announces ruijter>git when relying on uniscribe.

I think disabling uniscribe within the scope of word echo for uniscribe based wordOffset textInfos doesn't have to be a problem as long as we only disable uniscribe when checking word boundaries for word echo. The non-uniscribe implementation splits on non-alphanumeric characters I belief, and that's almost identical to how the current implementation echoes words.

ehollig commented 6 years ago

I think #6215 is related to this.

LeonarddeR commented 6 years ago

I'd like to add an isCollapsedAtEndOfWord or isCollapsedAtStartOfWord property to textInfos in order to let the word echo functionality check whether it should announce a word reliably. This is quite easy to do for offset based textINfos. It is much more hard for UIA, ITextDocument or other textInfos that aren't offset based. @MichaelDCurran: do you have suggestions on how to do this? To start with UIA, is there an easy way to check whether the textInfo is collapsed at the end of a word? I assume in most cases, I could use the start and end properties of the rangeObj?

michaelDCurran commented 6 years ago

It could be a bit costly but:

LeonarddeR commented 6 years ago

The prototype I now have works quite nicely. I'm now at a point where another UX decision must be made.

As noted, several textInfos/applications have different rules in how they treat characters as word boundaries. We could make at least two decisions here.

  1. Always echo a word if NVDA treats a character as a word separator. This works in Firefox, Word and Wordpad. IN Notepad with uniscribe on and in LibreOffice (for which the textInfo also uses uniscribe), results are as follows:

    a. Type "one.". NVDA says one. b. Type "two.". NVDA says one.two. c. Type "three.". NVDA says one.two.three.

    Other examples of double echo are 1,3, 2.4, etc. Though a bit verbose, this implementation gives people information as soon as possible and also tells how word boundaries are interpreted by the editor of use.

  2. Always respect word separator rules of TextInfos. This means that in Firefox, we have the following result:

    a. Type "one.". NVDA says nothing, the dot at the end is part of the word b. Press space, NVDA says "one." c. Type "two.". NVDA says nothing d. Type "three.". NVDA says nothing e. Press space, NVDA says "two.three."

I propose the following:

michaelDCurran commented 6 years ago

SymphonyTextInfo inherits from IA2TextInfo. The only reason SymphonyTextInfo would use uniscribe is if IAccessibleText::textAtOffset (for word) failed. IA2TextInfo._getWordOffsets calls super in this case.

From a design perspective I much prefer point 2. However,  from a purely practical point of view, I can say why point 1 would be an advantage. However, this depends on performance. How are you checking  for these characters that break words, and how are you finding the actual start of the word?

Turning off uniscribe in general everywhere except for standard edit fields (E.g. Notepad)  is probably okay.

LeonarddeR commented 6 years ago

@michaelDCurran commented on 15 mrt. 2018 11:06 CET:

SymphonyTextInfo inherits from IA2TextInfo. The only reason SymphonyTextInfo would use uniscribe is if IAccessibleText::textAtOffset (for word) failed. IA2TextInfo._getWordOffsets calls super in this case.

This is a good point you're bringing up. Some logging in IA2TextInfo revealed that uniscribe is not used here, but rather it seems that IAccessibleText::TextAtOffset does return the wrong offsets (i.e. not the offsets of the word how Libre Office sees it when pressing control+left/right arrow).

How are you checking  for these characters that break words,

This is currently still handled in speech.speakTypedCharacters. Instead of just speaking the typed word, it uses a new function that gets the caret textInfo and tries to find the last word before the caret using another method on that TextInfo.

and how are you finding the actual start of the word?

By creating a copy of the TextInfo at the caret position, moving backwards and comparing endpoints as you suggested. This is done in a method in TextInfo that can be overridden on a per TextInfo basis if performance might be a problem.

Turning off uniscribe in general everywhere except for standard edit fields (E.g. Notepad)  is probably okay.

This would directly affect word to word movement in virtual buffers, but I have no objections to that.

LeonarddeR commented 6 years ago

Looks like point 2 is much more easier to implement than I thought, and also looks somewhat cleaner from a code perspective. I'm going to test this prototype some more and will file a pr next week.

The code is in the i8065 @babbagecom branch.

Qchristensen commented 4 years ago

Has anyone looked at this lately? We are still being asked about it (The issue is still present in NVDA 2020.1)

Adriani90 commented 4 years ago

Still reproducible in NVDA 2020.3.

ABuffEr commented 2 years ago

Hi, still reproducible in current alpha-26946,8d34a18e. Any chance to see it fixed after speech refactoring of 2019.3?