search

nvaccess / nvda

NVDA, the free and open source Screen Reader for Microsoft Windows
https://www.nvaccess.org/
Other
2.1k stars 636 forks source link

If UIA is enabled in Word, the "Scrolls the braille display back" and "Scrolls the braille display forward" will not work in focus mode #12855

Closed cary-rowen closed 3 years ago

cary-rowen commented 3 years ago

Test.docx

Steps to reproduce:

  1. Download the provided test.docx.
  2. Open the document.
  3. Open the UIA in Word in the advanced settings panel of NVDA.
  4. Connect your braille display to make it work.
  5. In MS Word, press NVDA + space to make sure to switch to focus mode.
  6. Read the content of the document and try to execute "Scrolls the braille display back / forward".

Actual behavior:

The content on the braille display changes quickly and then reverts to the original content. Did not finish scrolling.

Expected behavior:

Display scrolled content correctly.

System configuration

NVDA installed/portable/running from source:

installed and portable

NVDA version:

NVDA2021.1

Windows version:

Windows 10 21H1 (x64) build 19043.1237

Name and version of other software in use when reproducing the issue:

Microsoft 365 MSO (16.0.14326.20324) 64 bit

Other information about your system:

None

Other questions

Does the issue still occur after restarting your computer?

Yes

Have you tried any other versions of NVDA? If so, please report their behaviors.

NVDA2021.1, 2020.4 and Alpha-23765 Can be reproduced

If add-ons are disabled, is your problem still occurring?

Yes

Does the issue still occur after you run the COM Registration Fixing Tool in NVDA's tools menu?

Yes

feerrenrut commented 3 years ago

I can't reproduce this with the braille viewer. Which braille device are you using? Can anyone else confirm with their braille device?

cary-rowen commented 3 years ago

Hello, I used orbit Reader 20 Plus, Seaka Mini24 and metec40 to reproduce this. Of course, Metec40 is a third-party additional driver, which may not be representative, but the other two devices can also reproduce this problem.

feerrenrut commented 3 years ago

@leonardder @michaelDCurran Can you reproduce this issue?

LeonarddeR commented 3 years ago

Nope, I can not, though I'm testing with alpha currently and stuff might have changed. @cary-rowen Could you please test with NVDA 2021.2 and 2021.3 Alpha?

cary-rowen commented 3 years ago

Hello, I upgraded NVDA to alpha-23799, and then I installed the English version of word, and the issue can still be reproduced. Of course NVDA2021.2 is the same.

The following is a log snippet (where fk2 is the key used for scrolling):

IO - inputCore.InputManager.executeGesture (23:38:11.228) - winInputHook (12292): Input: kb(laptop):upArrow DEBUG - editableText.EditableText._hasCaretMoved (23:38:11.277) - MainThread (17984): Caret move detected using event. Elapsed: 10 ms IO - braille.BrailleBuffer.update (23:38:11.300) - MainThread (17984): Braille regions text: ['文档 ', ' '] IO - braille.BrailleHandler.update (23:38:11.300) - MainThread (17984): Braille window dots: - IO - braille.BrailleHandler.update (23:38:11.301) - MainThread (17984): Braille window dots: - IO - speech.speech.speak (23:38:11.318) - MainThread (17984): Speaking ['空白'] IO - braille.BrailleBuffer.update (23:38:11.347) - MainThread (17984): Braille regions text: ['文档 ', ' '] IO - braille.BrailleHandler.update (23:38:11.347) - MainThread (17984): Braille window dots: - IO - braille.BrailleHandler.update (23:38:11.348) - MainThread (17984): Braille window dots: - IO - inputCore.InputManager.executeGesture (23:38:11.674) - winInputHook (12292): Input: kb(laptop):downArrow DEBUG - editableText.EditableText._hasCaretMoved (23:38:11.732) - MainThread (17984): Caret move detected using bookmarks. Elapsed: 0 ms IO - braille.BrailleBuffer.update (23:38:11.757) - MainThread (17984): Braille regions text: ['Several Japanese politicians have announced their decisions to seek the leadership of the '] IO - braille.BrailleHandler.update (23:38:11.757) - MainThread (17984): Braille window dots: 2347 15 1236 15 1235 1 123 - 6 2457 1 1234 1 1345 15 234 15 - 1234 135 123 24 2345 24 14 24 1 1345 234 - 125 1 1236 15 - 1 1345 1345 135 136 IO - braille.BrailleHandler.update (23:38:11.759) - MainThread (17984): Braille window dots: 2347 15 1236 15 1235 1 123 - 6 2457 1 1234 1 1345 15 234 15 - 1234 135 123 24 2345 24 14 24 1 1345 234 - 125 1 1236 15 - 1 1345 1345 135 136 IO - speech.speech.speak (23:38:11.780) - MainThread (17984): Speaking ['Several Japanese politicians have announced their decisions to seek the leadership of the '] IO - braille.BrailleBuffer.update (23:38:11.817) - MainThread (17984): Braille regions text: ['Several Japanese politicians have announced their decisions to seek the leadership of the '] IO - braille.BrailleHandler.update (23:38:11.818) - MainThread (17984): Braille window dots: 2347 15 1236 15 1235 1 123 - 6 2457 1 1234 1 1345 15 234 15 - 1234 135 123 24 2345 24 14 24 1 1345 234 - 125 1 1236 15 - 1 1345 1345 135 136 IO - braille.BrailleHandler.update (23:38:11.820) - MainThread (17984): Braille window dots: 2347 15 1236 15 1235 1 123 - 6 2457 1 1234 1 1345 15 234 15 - 1234 135 123 24 2345 24 14 24 1 1345 234 - 125 1 1236 15 - 1 1345 1345 135 136 IO - braille.BrailleBuffer.update (23:38:13.012) - MainThread (17984): Braille regions text: ['Several Japanese politicians have announced their decisions to seek the leadership of the '] IO - braille.BrailleHandler.update (23:38:13.012) - MainThread (17984): Braille window dots: 2347 15 1236 15 1235 1 123 - 6 2457 1 1234 1 1345 15 234 15 - 1234 135 123 24 2345 24 14 24 1 1345 234 - 125 1 1236 15 - 1 1345 1345 135 136 IO - braille.BrailleHandler.update (23:38:13.013) - MainThread (17984): Braille window dots: 2347 15 1236 15 1235 1 123 - 6 2457 1 1234 1 1345 15 234 15 - 1234 135 123 24 2345 24 14 24 1 1345 234 - 125 1 1236 15 - 1 1345 1345 135 136 IO - braille.BrailleBuffer.update (23:38:14.060) - MainThread (17984): Braille regions text: ['Several Japanese politicians have announced their decisions to seek the leadership of the '] IO - braille.BrailleHandler.update (23:38:14.060) - MainThread (17984): Braille window dots: 2347 15 1236 15 1235 1 123 - 6 2457 1 1234 1 1345 15 234 15 - 1234 135 123 24 2345 24 14 24 1 1345 234 - 125 1 1236 15 - 1 1345 1345 135 136 IO - braille.BrailleHandler.update (23:38:14.061) - MainThread (17984): Braille window dots: 2347 15 1236 15 1235 1 123 - 6 2457 1 1234 1 1345 15 234 15 - 1234 135 123 24 2345 24 14 24 1 1345 234 - 125 1 1236 15 - 1 1345 1345 135 136 IO - braille.BrailleBuffer.update (23:38:15.191) - MainThread (17984): Braille regions text: ['Several Japanese politicians have announced their decisions to seek the leadership of the '] IO - braille.BrailleHandler.update (23:38:15.191) - MainThread (17984): Braille window dots: 2347 15 1236 15 1235 1 123 - 6 2457 1 1234 1 1345 15 234 15 - 1234 135 123 24 2345 24 14 24 1 1345 234 - 125 1 1236 15 - 1 1345 1345 135 136 IO - braille.BrailleHandler.update (23:38:15.192) - MainThread (17984): Braille window dots: 2347 15 1236 15 1235 1 123 - 6 2457 1 1234 1 1345 15 234 15 - 1234 135 123 24 2345 24 14 24 1 1345 234 - 125 1 1236 15 - 1 1345 1345 135 136 IO - braille.BrailleBuffer.update (23:38:16.314) - MainThread (17984): Braille regions text: ['Several Japanese politicians have announced their decisions to seek the leadership of the '] IO - braille.BrailleHandler.update (23:38:16.314) - MainThread (17984): Braille window dots: 2347 15 1236 15 1235 1 123 - 6 2457 1 1234 1 1345 15 234 15 - 1234 135 123 24 2345 24 14 24 1 1345 234 - 125 1 1236 15 - 1 1345 1345 135 136 IO - braille.BrailleHandler.update (23:38:16.315) - MainThread (17984): Braille window dots: 2347 15 1236 15 1235 1 123 - 6 2457 1 1234 1 1345 15 234 15 - 1234 135 123 24 2345 24 14 24 1 1345 234 - 125 1 1236 15 - 1 1345 1345 135 136 DEBUG - external:brailleDisplayDrivers.MetecBD._do_key (23:38:16.920) - MainThread (17984): Metec key 0x10ff IO - inputCore.InputManager.executeGesture (23:38:16.921) - MainThread (17984): Input: br(MetecBD):fk2 IO - braille.BrailleHandler.update (23:38:16.935) - MainThread (17984): Braille window dots: 1345 14 15 145 - 2345 125 15 24 1235 - 145 15 14 24 234 24 135 1345 234 - 2345 135 - 234 15 15 13 - 2345 125 15 - 123 15 1 145 15 1235 234 IO - braille.BrailleBuffer.update (23:38:17.423) - MainThread (17984): Braille regions text: ['Several Japanese politicians have announced their decisions to seek the leadership of the '] IO - braille.BrailleHandler.update (23:38:17.423) - MainThread (17984): Braille window dots: 2347 15 1236 15 1235 1 123 - 6 2457 1 1234 1 1345 15 234 15 - 1234 135 123 24 2345 24 14 24 1 1345 234 - 125 1 1236 15 - 1 1345 1345 135 136 IO - braille.BrailleHandler.update (23:38:17.424) - MainThread (17984): Braille window dots: 2347 15 1236 15 1235 1 123 - 6 2457 1 1234 1 1345 15 234 15 - 1234 135 123 24 2345 24 14 24 1 1345 234 - 125 1 1236 15 - 1 1345 1345 135 136 DEBUG - external:brailleDisplayDrivers.MetecBD._do_key (23:38:18.117) - MainThread (17984): Metec key 0x10ff IO - inputCore.InputManager.executeGesture (23:38:18.117) - MainThread (17984): Input: br(MetecBD):fk2 IO - braille.BrailleHandler.update (23:38:18.148) - MainThread (17984): Braille window dots: 1345 14 15 145 - 2345 125 15 24 1235 - 145 15 14 24 234 24 135 1345 234 - 2345 135 - 234 15 15 13 - 2345 125 15 - 123 15 1 145 15 1235 234 IO - braille.BrailleBuffer.update (23:38:18.567) - MainThread (17984): Braille regions text: ['Several Japanese politicians have announced their decisions to seek the leadership of the '] IO - braille.BrailleHandler.update (23:38:18.567) - MainThread (17984): Braille window dots: 2347 15 1236 15 1235 1 123 - 6 2457 1 1234 1 1345 15 234 15 - 1234 135 123 24 2345 24 14 24 1 1345 234 - 125 1 1236 15 - 1 1345 1345 135 136 IO - braille.BrailleHandler.update (23:38:18.568) - MainThread (17984): Braille window dots: 2347 15 1236 15 1235 1 123 - 6 2457 1 1234 1 1345 15 234 15 - 1234 135 123 24 2345 24 14 24 1 1345 234 - 125 1 1236 15 - 1 1345 1345 135 136 IO - braille.BrailleBuffer.update (23:38:19.677) - MainThread (17984): Braille regions text: ['Several Japanese politicians have announced their decisions to seek the leadership of the '] IO - braille.BrailleHandler.update (23:38:19.678) - MainThread (17984): Braille window dots: 2347 15 1236 15 1235 1 123 - 6 2457 1 1234 1 1345 15 234 15 - 1234 135 123 24 2345 24 14 24 1 1345 234 - 125 1 1236 15 - 1 1345 1345 135 136 IO - braille.BrailleHandler.update (23:38:19.679) - MainThread (17984): Braille window dots: 2347 15 1236 15 1235 1 123 - 6 2457 1 1234 1 1345 15 234 15 - 1234 135 123 24 2345 24 14 24 1 1345 234 - 125 1 1236 15 - 1 1345 1345 135 136 DEBUG - external:brailleDisplayDrivers.MetecBD._do_key (23:38:19.929) - MainThread (17984): Metec key 0x10ff IO - inputCore.InputManager.executeGesture (23:38:19.929) - MainThread (17984): Input: br(MetecBD):fk2 IO - braille.BrailleHandler.update (23:38:19.960) - MainThread (17984): Braille window dots: 1345 14 15 145 - 2345 125 15 24 1235 - 145 15 14 24 234 24 135 1345 234 - 2345 135 - 234 15 15 13 - 2345 125 15 - 123 15 1 145 15 1235 234 IO - braille.BrailleBuffer.update (23:38:20.804) - MainThread (17984): Braille regions text: ['Several Japanese politicians have announced their decisions to seek the leadership of the '] IO - braille.BrailleHandler.update (23:38:20.805) - MainThread (17984): Braille window dots: 2347 15 1236 15 1235 1 123 - 6 2457 1 1234 1 1345 15 234 15 - 1234 135 123 24 2345 24 14 24 1 1345 234 - 125 1 1236 15 - 1 1345 1345 135 136 IO - braille.BrailleHandler.update (23:38:20.806) - MainThread (17984): Braille window dots: 2347 15 1236 15 1235 1 123 - 6 2457 1 1234 1 1345 15 234 15 - 1234 135 123 24 2345 24 14 24 1 1345 234 - 125 1 1236 15 - 1 1345 1345 135 136 IO - braille.BrailleBuffer.update (23:38:21.930) - MainThread (17984): Braille regions text: ['Several Japanese politicians have announced their decisions to seek the leadership of the '] IO - braille.BrailleHandler.update (23:38:21.930) - MainThread (17984): Braille window dots: 2347 15 1236 15 1235 1 123 - 6 2457 1 1234 1 1345 15 234 15 - 1234 135 123 24 2345 24 14 24 1 1345 234 - 125 1 1236 15 - 1 1345 1345 135 136 IO - braille.BrailleHandler.update (23:38:21.932) - MainThread (17984): Braille window dots: 2347 15 1236 15 1235 1 123 - 6 2457 1 1234 1 1345 15 234 15 - 1234 135 123 24 2345 24 14 24 1 1345 234 - 125 1 1236 15 - 1 1345 1345 135 136 IO - inputCore.InputManager.executeGesture (23:38:22.824) - winInputHook (12292):

cary-rowen commented 3 years ago

I tested the seaka V6 braille display on other computers with office2016 installed, and this problem can also be reproduced.

LeonarddeR commented 3 years ago

Is UIA enabled when the advanced setting "Always use UI Automation to access Microsoft Word document controls when available" is disabled in alpha builds? If not, it is likely that you're on a version of Office that has limited support for UIA that's not mature enough.

cary-rowen commented 3 years ago

Microsoft 365 MSO (16.0.14326.20384) 64-bit Both of the following options are enabled： Enable selective registration for UI Automation events and property changes Always use UI Automation to access Microsoft Word document controls when available

cary-rowen commented 3 years ago

Don't forget this issue. All people around me who use braille displays can reproduce this.

michaelDCurran commented 3 years ago

I was finally able to reproduce this by installing and switching to a Chinese input method such as Taiwan - Microsoft Quick. then, when scrolling forward, the display keeps jumping back to the caret. This keeps happening for about 15 seconds after I last moved the caret with the arrow keys. After this 15 seconds, it behaves normally again. Also switching back to English US the problem disappears. I am investigating further. @cary-rowen are you using a chinese input method when reading the document?

michaelDCurran commented 3 years ago

While Chinese Microsoft Quick is runningin Microsoft Word, it seems that we get legacy locationchange / show events for the caret from MSAA. Once per second for about 28 seconds or so. In NVDA we currently route this caret event straight on toe the focus, whether it is MSAA or not. We certainly should not be doing this if the focus is UI Automation at very least. I will come up with a pr for this.

cary-rowen commented 3 years ago

当中文 Microsoft Quick 在 Microsoft Word 中运行时，似乎我们从 MSAA 获得了插入符号的遗留位置更改/显示事件。每秒一次，持续约 28 秒左右。 在 NVDA 中，我们目前直接将这个插入符号事件路由到焦点，无论它是否是 MSAA。如果重点至少是 UI 自动化，我们当然不应该这样做。 我会为此提出一个公关。

Hi, Yes, I am using the Simplified Chinese Microsoft Pinyin input method. I tested the build #12868 and it can solve this issue.