nvaccess / nvda

NVDA, the free and open source Screen Reader for Microsoft Windows
https://www.nvaccess.org/
Other
2.11k stars 637 forks source link

appModules/modern keyboard (emoji panel and friends): 2020 modernization #11454

Closed josephsl closed 4 years ago

josephsl commented 4 years ago

Hi,

Technically this is sort of a broad issue.

Background:

NVDA introduced support for emoji panel and friends (collectively known as modern keyboard) in 2017. In its early days, NVDA supported emoji panel. Since then, more features were added, including hardware keyboard suggestions, clipboard history, dictation and small changes.

Although the code works well at the moment, it poses issues. The most prominent one being timing issue and resulting COM error when trying to fetch Automation Id property for an element when modern keyboard screen isn't fully shown (see #11445 for an overview). It also had a rather poorly designed way to detect appearance of emoji panel simply because COM error was seen when NVDA tries to instantiate a UIA object when state change and certain element selected event code paths are executed, especially when element selected event fires before emoji panel is fully ready. Therefore, a 2020 modernization is proposed.

Is your feature request related to a problem? Please describe.

Issues stemming from 2017 design and recent nVDA changes surface in July 2020, including inefficient emoji panel screen detection, COM error and such. Also, due to #10377, some people cannot use emoji panel and clipboard history effectively due to traversal issues.

Describe the solution you'd like

A reimagined modern keyboard app module is in the works. Changes include:

Describe alternatives you've considered

Leave the app module as is. Modernization will show up via Windows 10 App Essentials add-on.

Additional context

See #11445 for detailed justifications.

Thanks.

josephsl commented 4 years ago

Hi,

Separate issue/PR pairs will be used to address these concerns.

Thanks.