search

nvaccess / nvda

NVDA, the free and open source Screen Reader for Microsoft Windows
https://www.nvaccess.org/
Other
2.1k stars 634 forks source link

nvda is not advertising the microsoft store search field #14832

Open fernando-jose-silva opened 1 year ago

fernando-jose-silva commented 1 year ago

Steps to reproduce:

open the microsoft store and walk with tabe until you reach the search field. type a search that yields several suggestions, whatsapp should suggest several options. press enter to search. now return to the search field with the tab key or even with shift + tabe.

Actual behavior:

nvda announces 3 suggestions. it cuts the ad search editing. the strangest thing is that when pressing nvda + tabe the reader informs the search field. This even using nvda without add-ons.

Expected behavior:

nvda announces search editing in the log it reports as if the search field has been read, as well as in the speech viewer. Now thinking about it, nvda should start reading the search field and something interrupts this reading and makes it announce the amount of suggestions. I think that in this case there should be something that prevents nvda from cutting the field name to announce other information such as the number of suggestions. the interruption occurs so quickly that the user is not informed of the field name.

NVDA logs, crash dumps and other attachments:

System configuration

NVDA installed/portable/running from source:

instaled

NVDA version:

nvda.exe, NVDA alpha-28089,9fe52071

Windows version:

11 22621.1631

Name and version of other software in use when reproducing the issue:

WinStore.App.exe, Microsoft.WindowsStore 22301.1401.15.0

Other information about your system:

Other questions

Does the issue still occur after restarting your computer?

Have you tried any other versions of NVDA? If so, please report their behaviors.

no

If NVDA add-ons are disabled, is your problem still occurring?

yes

Does the issue still occur after you run the COM Registration Fixing Tool in NVDA's tools menu?

yes log loja.txt

josephsl commented 1 year ago

Hi, controller for/layout invalidated events and search suggestions handling… I think Narrator users are experiencing a similar thig (will test later). Thanks.

josephsl commented 1 year ago

Hi,

Update: I think this is unique to Microsoft Store as I can't replicate this in apps such as Windows 11 Settings. If it does turn out to be an issue with the ap, then I guess we might as well register a feedback via Feedback Hub about UIA events in Store.

Thanks.

seanbudd commented 1 year ago

Please report this to Microsoft

Adriani90 commented 5 months ago

I enabled UIA, speech, speech manager and events in the advanced logging, and this is what I get when navigating to the field with tab and shift+tab. If I understand it correctly NVDA seems to get more events to speak at the same time and the speech manager does not push the speech on the edit field property reporting to the synth. It prefers to push the reporting of number of suggestions. Not sure if NVDA can specify which event should be spoken first and which should be spoken afterwards, but the speech manager definitely seem to receive both events. I guess in this case both events are important so the speech manager should push first the event about the focused element and then the one about its related suggestions to the synth.

DEBUG - NVDAObjects.UIA.UIATextInfo._getTextWithFieldsForUIARange (18:41:11.562) - MainThread (9152):
_getTextWithFieldsForUIARange
DEBUG - NVDAObjects.UIA.UIATextInfo._getTextWithFieldsForUIARange (18:41:11.562) - MainThread (9152):
rootElement: Bearbeiten
DEBUG - NVDAObjects.UIA.UIATextInfo._getTextWithFieldsForUIARange (18:41:11.562) - MainThread (9152):
full text: Wh
DEBUG - NVDAObjects.UIA.UIATextInfo._getTextWithFieldsForUIARange (18:41:11.562) - MainThread (9152):
Fetching parents starting from enclosingElement
DEBUGWARNING - UIAHandler.func (18:41:11.562) - UIAHandler.UIAHandler.MTAThread (18324):
Not registering for textChange events from UIA element with class name 'TextBox' and automation ID 'TextBox'
DEBUG - NVDAObjects.UIA.UIATextInfo._getTextWithFieldsForUIARange (18:41:11.562) - MainThread (9152):
Hit root
DEBUG - NVDAObjects.UIA.UIATextInfo._getTextWithFieldsForUIARange (18:41:11.562) - MainThread (9152):
Done fetching parents
DEBUG - NVDAObjects.UIA.UIATextInfo._getTextWithFieldsForUIARange (18:41:11.562) - MainThread (9152):
Generating controlFields for parents
DEBUG - NVDAObjects.UIA.UIATextInfo._getTextWithFieldsForUIARange (18:41:11.562) - MainThread (9152):
Done generating controlFields for parents
DEBUG - NVDAObjects.UIA.UIATextInfo._getTextWithFieldsForUIARange (18:41:11.562) - MainThread (9152):
Yielding control starts for parents
DEBUG - NVDAObjects.UIA.UIATextInfo._getTextWithFieldsForUIARange (18:41:11.562) - MainThread (9152):
Done yielding control starts for parents
DEBUG - NVDAObjects.UIA.UIATextInfo._getTextWithFieldsForUIARange (18:41:11.562) - MainThread (9152):
Yielding balanced fields for textRange
DEBUG - NVDAObjects.UIA.UIATextInfo._getTextWithFieldsForUIARange (18:41:11.562) - MainThread (9152):
no children
DEBUG - NVDAObjects.UIA.UIATextInfo._getTextWithFieldsForUIARange (18:41:11.562) - MainThread (9152):
Yielding text
DEBUG - NVDAObjects.UIA.UIATextInfo._getTextWithFields_text (18:41:11.562) - MainThread (9152):
_getTextWithFields_text start
DEBUG - NVDAObjects.UIA.UIATextInfo._getTextWithFields_text (18:41:11.562) - MainThread (9152):
Walking by unit None, with further units of: [1, 2, 0]
DEBUG - NVDAObjects.UIA.UIATextInfo._getTextWithFields_text (18:41:11.562) - MainThread (9152):
Chunk has text. Fetching formatting
DEBUG - NVDAObjects.UIA.UIATextInfo._getTextWithFields_text (18:41:11.566) - MainThread (9152):
Yielding formatting and text
DEBUG - NVDAObjects.UIA.UIATextInfo._getTextWithFields_text (18:41:11.566) - MainThread (9152):
field: FieldCommand formatChange with {'language': 'de_DE'}, text: Wh
DEBUG - NVDAObjects.UIA.UIATextInfo._getTextWithFields_text (18:41:11.566) - MainThread (9152):
Done _getTextWithFields_text
DEBUG - NVDAObjects.UIA.UIATextInfo._getTextWithFieldsForUIARange (18:41:11.566) - MainThread (9152):
_getTextWithFieldsForUIARange end
IO - speech.speech.speak (18:41:11.566) - MainThread (9152):
Speaking ['Suchen', 'Eingabefeld', CancellableSpeech (still valid, devInfo< isCanceledCache: False, isValidCallback: True, isValidCallbackDevInfo: isLast: True, previouslyHad: True, isAncestorOfCurrentFocus: False, is foreground obj False, isMenuItemOfCurrentFocus: False >), 'Wh']
DEBUG - speech.manager.SpeechManager.speak (18:41:11.566) - MainThread (9152):
SpeechManager- speak (priority <SpeechPriority.NORMAL: 0>): ['Suchen  ', 'Eingabefeld  ', CancellableSpeech (still valid, devInfo< isCanceledCache: False, isValidCallback: True, isValidCallbackDevInfo: isLast: True, previouslyHad: True, isAncestorOfCurrentFocus: False, is foreground obj False, isMenuItemOfCurrentFocus: False >), 'Wh  ']
DEBUG - speech.manager.SpeechManager._queueSpeechSequence (18:41:11.566) - MainThread (9152):
SpeechManager- Out Seq: [['Suchen  ', 'Eingabefeld  ', CancellableSpeech (still valid, devInfo< isCanceledCache: False, isValidCallback: True, isValidCallbackDevInfo: isLast: True, previouslyHad: True, isAncestorOfCurrentFocus: False, is foreground obj False, isMenuItemOfCurrentFocus: False >), 'Wh  ', IndexCommand(1085)], [EndUtteranceCommand()]]
DEBUG - speech.manager.SpeechManager._queueSpeechSequence (18:41:11.566) - MainThread (9152):
SpeechManager- Current priority: 0, queLen: 2
DEBUG - speech.manager.SpeechManager._queueSpeechSequence (18:41:11.566) - MainThread (9152):
SpeechManager- current queue: [['Suchen  ', 'Gruppierung  ', CancellableSpeech (still valid, devInfo< isCanceledCache: False, isValidCallback: True, isValidCallbackDevInfo: isLast: False, previouslyHad: False, isAncestorOfCurrentFocus: True, is foreground obj False, isMenuItemOfCurrentFocus: False >), IndexCommand(1084)], [EndUtteranceCommand()]]
DEBUG - speech.manager.SpeechManager._getMostRecentlyCancelledUtterance (18:41:11.566) - MainThread (9152):
SpeechManager- Length of _cancelCommandsForUtteranceBeingSpokenBySynth: 1 Length of _indexesSpeaking: 1 
DEBUG - speech.manager.SpeechManager._doRemoveCancelledSpeechCommands (18:41:11.566) - MainThread (9152):
SpeechManager- Last index: None
DEBUG - speech.manager.SpeechManager.speak (18:41:11.566) - MainThread (9152):
SpeechManager- Will interrupt: False Will push: False | _indexesSpeaking: [1084] | _curPriQueue valid: True | _shouldPushWhenDoneSpeaking: False | _cancelledLastSpeechWithSynth False
DEBUG - speech.manager.SpeechManager.speak (18:41:11.566) - MainThread (9152):
SpeechManager- Not pushing speech
DEBUG - UIAHandler.UIAHandler.IUIAutomationPropertyChangedEventHandler_HandlePropertyChangedEvent (18:41:11.606) - Dummy-1 (23332):
handlePropertyChangeEvent called with property ControllerFor, value None for element Suchen Bearbeiten with automationID TextBox, className TextBox and frameworkID XAML
DEBUG - UIAHandler.UIAHandler.IUIAutomationPropertyChangedEventHandler_HandlePropertyChangedEvent (18:41:11.606) - Dummy-1 (23332):
propertyChange event is for focus. Redirecting event to focus NVDAObject <NVDAObjects.Dynamic_SearchFieldXamlEditableTextEditableTextWithAutoSelectDetectionUIA object at 0x06638390>
DEBUG - UIAHandler.UIAHandler.IUIAutomationPropertyChangedEventHandler_HandlePropertyChangedEvent (18:41:11.606) - Dummy-1 (23332):
Checking if should accept NVDA event UIA_controllerFor with window hwnd 0X70018 of class Windows.UI.Core.CoreWindow
DEBUG - UIAHandler.UIAHandler.IUIAutomationPropertyChangedEventHandler_HandlePropertyChangedEvent (18:41:11.606) - Dummy-1 (23332):
handlePropertyChangeEvent: queuing NVDA UIA_controllerFor event for NVDAObject <NVDAObjects.Dynamic_SearchFieldXamlEditableTextEditableTextWithAutoSelectDetectionUIA object at 0x06638390> 
DEBUG - UIAHandler.UIAHandler.IUIAutomationEventHandler_HandleAutomationEvent (18:41:11.606) - Dummy-1 (23332):
handleAutomationEvent called with event LayoutInvalidated for element [None] Liste with automationID SuggestionsList, className ListView and frameworkID XAML
DEBUG - UIAHandler.UIAHandler.isNativeUIAElement (18:41:11.606) - Dummy-1 (23332):
checking if is native UIA  element: [None] Liste with automationID SuggestionsList, className ListView and frameworkID XAML
DEBUG - UIAHandler.UIAHandler.getNearestWindowHandle (18:41:11.606) - Dummy-1 (23332):
Locating nearest ancestor windowHandle for element [None] Liste with automationID SuggestionsList, className ListView and frameworkID XAML
DEBUG - NVDAObjects.UIA.UIA.__init__ (18:41:11.666) - MainThread (9152):
No windowHandle for UIA NvDAObject. Searching UIA element ancestry for nearest windowHandle
DEBUG - UIAHandler.UIAHandler.getNearestWindowHandle (18:41:11.666) - MainThread (9152):
Locating nearest ancestor windowHandle for element [None] Liste with automationID SuggestionsList, className ListView and frameworkID XAML
DEBUG - UIAHandler.UIAHandler.getNearestWindowHandle (18:41:11.673) - Dummy-1 (23332):
Found ancestor element with valid windowHandle hwnd 0X70018 of class Windows.UI.Core.CoreWindow
DEBUG - UIAHandler.UIAHandler.isUIAWindow (18:41:11.673) - Dummy-1 (23332):
Found cached is UIA window True for hwnd hwnd 0X70018 of class Windows.UI.Core.CoreWindow
DEBUG - UIAHandler.UIAHandler.isNativeUIAElement (18:41:11.673) - Dummy-1 (23332):
treating element as native due to windowHandle hwnd 0X70018 of class Windows.UI.Core.CoreWindow. 
DEBUG - UIAHandler.UIAHandler.getNearestWindowHandle (18:41:11.673) - Dummy-1 (23332):
Got previously cached nearest windowHandle of hwnd 0X70018 of class Windows.UI.Core.CoreWindow for element [None] Liste with automationID SuggestionsList, className ListView and frameworkID XAML
DEBUG - UIAHandler.UIAHandler.IUIAutomationEventHandler_HandleAutomationEvent (18:41:11.673) - Dummy-1 (23332):
Checking if should accept NVDA event UIA_layoutInvalidated with window hwnd 0X70018 of class Windows.UI.Core.CoreWindow
DEBUG - UIAHandler.UIAHandler.IUIAutomationEventHandler_HandleAutomationEvent (18:41:11.676) - Dummy-1 (23332):
handleAutomationEvent: created object <NVDAObjects.UIA.SuggestionsList object at 0x06608D50> 
DEBUG - UIAHandler.UIAHandler.IUIAutomationEventHandler_HandleAutomationEvent (18:41:11.676) - Dummy-1 (23332):
handleAutomationEvent: queuing NVDA event UIA_layoutInvalidated for NVDAObject <NVDAObjects.UIA.SuggestionsList object at 0x06608D50> 
DEBUG - UIAHandler.UIAHandler.getNearestWindowHandle (18:41:11.676) - MainThread (9152):
Found ancestor element with valid windowHandle hwnd 0X70018 of class Windows.UI.Core.CoreWindow
DEBUG - speech.manager.SpeechManager.cancel (18:41:11.676) - MainThread (9152):
SpeechManager- Cancel
IO - speech.speech.speak (18:41:11.676) - MainThread (9152):
Speaking ['8 Vorschläge']
DEBUG - speech.manager.SpeechManager._onSynthDoneSpeaking (18:41:11.676) - synthDrivers._espeak.BgThread (24092):
SpeechManager- synthDoneSpeaking synth:SynthDriver('espeak')
DEBUG - speech.manager.SpeechManager.speak (18:41:11.676) - MainThread (9152):
SpeechManager- speak (priority <SpeechPriority.NORMAL: 0>): ['8 Vorschläge  ']
DEBUG - speech.manager.SpeechManager._queueSpeechSequence (18:41:11.676) - MainThread (9152):
SpeechManager- Out Seq: [['8 Vorschläge  ', IndexCommand(1086)], [EndUtteranceCommand()]]
DEBUG - speech.manager.SpeechManager._queueSpeechSequence (18:41:11.676) - MainThread (9152):
SpeechManager- Current priority: 0, queLen: 0
DEBUG - speech.manager.SpeechManager._getMostRecentlyCancelledUtterance (18:41:11.676) - MainThread (9152):
SpeechManager- Length of _cancelCommandsForUtteranceBeingSpokenBySynth: 0 Length of _indexesSpeaking: 0 
DEBUG - speech.manager.SpeechManager._doRemoveCancelledSpeechCommands (18:41:11.676) - MainThread (9152):
SpeechManager- Last index: None
DEBUG - speech.manager.SpeechManager.speak (18:41:11.676) - MainThread (9152):
SpeechManager- Will interrupt: False Will push: True | _indexesSpeaking: [] | _curPriQueue valid: False | _shouldPushWhenDoneSpeaking: False | _cancelledLastSpeechWithSynth False
DEBUG - speech.manager.SpeechManager.speak (18:41:11.676) - MainThread (9152):
SpeechManager- Pushing next speech
DEBUG - speech.manager.SpeechManager._pushNextSpeech (18:41:11.676) - MainThread (9152):
SpeechManager- pushNextSpeech - doneSpeaking: True
DEBUG - speech.manager.SpeechManager._pushNextSpeech (18:41:11.676) - MainThread (9152):
SpeechManager- Synth Gets: ['8 Vorschläge  ', IndexCommand(1086)]
DEBUG - speech.manager.SpeechManager._handleDoneSpeaking (18:41:11.689) - MainThread (9152):
SpeechManager- Synth done speaking, should push: False
DEBUG - speech.manager.SpeechManager._onSynthIndexReached (18:41:11.929) - synthDrivers._espeak.BgThread (24092):
SpeechManager- synthReachedIndex: 1086, synth: SynthDriver('espeak')
DEBUG - speech.manager.SpeechManager._onSynthDoneSpeaking (18:41:11.929) - synthDrivers._espeak.BgThread (24092):
SpeechManager- synthDoneSpeaking synth:SynthDriver('espeak')
DEBUG - speech.manager.SpeechManager._handleIndex (18:41:11.945) - MainThread (9152):
SpeechManager- Handle index: 1086
DEBUG - speech.manager.SpeechManager._removeCompletedFromQueue (18:41:11.945) - MainThread (9152):
SpeechManager- Removing: ['8 Vorschläge  ', IndexCommand(1086)]
DEBUG - speech.manager.SpeechManager._getMostRecentlyCancelledUtterance (18:41:11.945) - MainThread (9152):
SpeechManager- Length of _cancelCommandsForUtteranceBeingSpokenBySynth: 0 Length of _indexesSpeaking: 0 
DEBUG - speech.manager.SpeechManager._doRemoveCancelledSpeechCommands (18:41:11.945) - MainThread (9152):
SpeechManager- Last index: None
DEBUG - speech.manager.SpeechManager._pushNextSpeech (18:41:11.945) - MainThread (9152):
SpeechManager- pushNextSpeech - doneSpeaking: False
DEBUG - speech.manager.SpeechManager._pushNextSpeech (18:41:11.945) - MainThread (9152):
SpeechManager- No more speech
DEBUG - speech.manager.SpeechManager._handleDoneSpeaking (18:41:11.945) - MainThread (9152):
SpeechManager- Synth done speaking, should push: False
DEBUG - speech.manager.SpeechManager.cancel (18:41:13.486) - MainThread (9152):
SpeechManager- Cancel

I saw this issue in other places as well, e.g. in some menu bars or in Microsoft Edge whe navigating with f6 to the address bar.

Adriani90 commented 5 months ago

This is the developer info for the edit field

INFO - globalCommands.script_navigatorObject_devInfo (19:03:54.633) - MainThread (9152):
Developer info for navigator object:
name: 'Suchen'
role: EDITABLETEXT
processID: 22532
roleText: None
states: FOCUSABLE, FOCUSED
isFocusable: True
hasFocus: True
Python object: <NVDAObjects.Dynamic_SearchFieldXamlEditableTextEditableTextWithAutoSelectDetectionUIA object at 0x05936490>
Python class mro: (<class 'NVDAObjects.Dynamic_SearchFieldXamlEditableTextEditableTextWithAutoSelectDetectionUIA'>, <class 'NVDAObjects.UIA.SearchField'>, <class 'NVDAObjects.UIA.XamlEditableText'>, <class 'NVDAObjects.UIA.InaccurateTextChangeEventEmittingEditableText'>, <class 'NVDAObjects.behaviors.EditableTextWithAutoSelectDetection'>, <class 'NVDAObjects.behaviors.EditableText'>, <class 'NVDAObjects.behaviors.EditableTextWithSuggestions'>, <class 'NVDAObjects.behaviors.InputFieldWithSuggestions'>, <class 'NVDAObjects.behaviors.EditableTextBase'>, <class 'editableText.EditableText'>, <class 'NVDAObjects.UIA.UIA'>, <class 'NVDAObjects.window.Window'>, <class 'NVDAObjects.NVDAObject'>, <class 'documentBase.TextContainerObject'>, <class 'baseObject.ScriptableObject'>, <class 'baseObject.AutoPropertyObject'>, <class 'garbageHandler.TrackedObject'>, <class 'object'>)
description: ''
location: RectLTWH(left=797, top=162, width=252, height=49)
value: 'Wh'
TextInfo: <class 'NVDAObjects.UIA.UIATextInfo'>
appModule: AppModule(appModuleHandler, appName='winstore.app', processID=22532)
appModule.productName: 'Microsoft.WindowsStore'
appModule.productVersion: '22403.1401.5.0'
appModule.helperLocalBindingHandle: c_long(75644552)
windowHandle: 458776
windowClassName: 'Windows.UI.Core.CoreWindow'
windowControlID: 0
windowStyle: 1409286144
extendedWindowStyle: 2621440
windowThreadID: 13904
windowText: 'Microsoft Store'
displayText: ''
UIAElement: <POINTER(IUIAutomationElement) ptr=0x6b88e10 at 5752e40>
UIA automationID: TextBox
UIA frameworkID: XAML
UIA runtimeID: (42, 458776, 4, 8)
UIA providerDescription: [pid:22532,providerId:0x0 Main(parent link):Unidentified Provider (unmanaged:Windows.UI.Xaml.dll)]
UIA className: TextBox
UIA patterns available: TextPattern, ScrollItemPattern, TextEditPattern, LegacyIAccessiblePattern, ValuePattern