Feature: Support for accessibility

[m-abs]

Is your feature request related to a problem? Please describe. Hi,

I work at Nota, the Danish Library and Expertise Center for people with print disabilities.

Our users have various disabilities that we need to support, some are dyslexic, some have visual impairment and other have cognitive impairment.

It is essential for our use case that we can make the app accessible, unfortunately this in one of the things NativeScript doesn’t do very well.

This is the reason we maintain this plugin: @nota/nativescript-accessibility-ext .

I had a chat with @sebawita, we agreed I should make this issue, so we at least can have a talk to see, if get built-in support for accessibility in NativeScript.

Describe the solution you'd like The solution should make it easier to make NativeScript apps accessible for people with disabilities. It should smooth out the differences between the platforms, while still provide full access to those features only found on one platform.

The biggest challenge is the differences in the accessibility support of two platforms.

To give an example of a feature which is on both platforms and one that are only on one: Setting a label on a view, on iOS we can set the string value to accessibilityLabel and on Android we set the View's ContentDescription. This label is what VoiceOver(iOS)/TalkBack(android) will read.

iOS also have the property accessibilityIdentifier, there is no equivalent on Android. This value is meant to be a single unique value that can be used to find the UIView during automated testing.

Side note: NativeScript binds the property automationText to both accessibilityLabel and accessibilityIdentifier on iOS, which is incorrect usage see #3150

This feature meant as start of discussion, so I won't into all the accessibility features here.

Is this something you're interested in supporting upstream? And if so, what are your thoughts on how best to proceed?

Describe alternatives you've considered

As mentioned we have a plugin @nota/nativescript-accessibility-ext that implements some of the features. The dream scenario is to deprecate our plugin and get a cleaner API than our more-or-less direct mapping to native-properties.

Additional context

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/71023594-feature-support-for-accessibility?utm_campaign=plugin&utm_content=tracker%2F12908224&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F12908224&utm_medium=issues&utm_source=github).

[tsonevn]

ping @manoldonev

[manoldonev]

@m-abs Thanks for reaching out!

Ideally as both iOS and Android provide APIs for making apps accessible to people with disabilities, NativeScript should be able to leverage these capabilities as well (I guess that is true for any UI framework / library).

It also makes sense that ultimately this should be implemented as part of the core framework and not as a separate plugin. However, providing proper accessibility support is no small task and currently we don't have this on the roadmap for the foreseeable future (assuming that we are supposed to drive this).

The roadmap of course is not set in stone and I will bring this into consideration on our future planning meetings with the NativeScript PMs but at the moment we can neither commit, nor spend the time to deep-dive into the subject and do a proper research (also detailed review of your plugin).

If we were to implement this as part of NativeScript the ideal process would look like:

• Spend time for building in-depth accessibility domain knowledge
• Spend time for proper research of accessibility support in the mobile world (Android, iOS, and React Native)
• Prepare a detailed design spec document for this feature (architecture & API proposal, platform limitations & differences, challenges, automated testing, feature documentation, etc.)
• Implement the feature
• Write the automated tests
• Write the documentation
• [Ongoing] maintenance

To get the discussion going can you elaborate on the following questions:

• Why did you choose to implement this as a 3rd party plugin versus contributing to the NativeScript framework directly in the first place? I can see that your plugin is BSD3 licensed while NativeScript uses Apache 2.0 -- is that a problem for you? (if some part of your plugin is reused)
• Do you have specific problems with your plugin that prevent you from implementing specific accessibility scenarios?
• What kind of maintenance effort does your plugin require on iOS/Android release basis?

[m-abs]

Hi @manoldonev

Thanks for your detailed reply.

Spend time for building in-depth accessibility domain knowledge

Of cause and we will of cause share our knowledge as a part of the process.

Spend time for proper research of accessibility support in the mobile world (Android, iOS, and React Native)

As a starting point I'll link to the guidelines provided for Android and iOS.

React Native provide some accessibility support.

....

To get the discussion going can you elaborate on the following questions:

Why did you choose to implement this as a 3rd party plugin versus contributing to the NativeScript framework directly in the first place? I can see that your plugin is BSD3 licensed while NativeScript uses Apache 2.0 -- is that a problem for you? (if some part of your plugin is reused)

Back when we started the plugin, we were all too aware that the APIs between the two platforms were very different and couldn't easily find a common API for it.

The license isn't a problem, it can be changed to Apache 2.0 if needed.

But I think most of the code should be rewritten anyway.

Do you have specific problems with your plugin that prevent you from implementing specific accessibility scenarios?

iOS have a few accessibility events we'd like to use, but they require subclassing the native views so we can't just extend the NativeScript view-classes.

See: https://developer.apple.com/documentation/uikit/accessibility/uiaccessibilityaction

The most interesting are accessibilityIncrement(), accessibilityDecrement(), accessibilityScroll(), accessibilityPerformEscape() and accessibilityPerformMagicTap().

accessibilityIncrement()/accessibilityDecrement() would make our sliders much more accessible, in that we would better control the effect of increasing and decreasing values.

accessibilityPerformMagicTap() can be implemented globally in the AppDelegate but we'd like to have it as an event on the Page element.

accessibilityPerformEscape() gives us some problems, See: #6123

What kind of maintenance effort does your plugin require on iOS/Android release basis?

So far very little. We have had a few breakages on new Android SDK release and they usually take a few hours to fix.

The biggest problem is we don't catch these problems early, which could/should be solved by adding the missing automated tests to detect them.

[manoldonev]

@m-abs Thank you for elaborating on this! We will gladly accept your offer for knowledge sharing session on accessibility but I think it makes more sense to do it as soon as we have a clearer idea on how to execute this task within the NativeScript roadmap.

As a starting point we will definitely check the provided guidelines above.

[m-abs]

Is there something, we can do to help you getting a clearer idea on how to do that?

We could/should draft some guidelines for our plugin, this might help too.

NSHipster have very good summery on accessibility for iOS, I should've linked to this earlier. https://nshipster.com/uiaccessibility/

[manoldonev]

@m-abs we had a discussion with our PMs onto the immediate roadmap and realistically we cannot start working on this before H2 (after July).

If you are open to collaboration with the implementation of this feature in the NativeScript framework I can suggest you to start drafting a design document in the next month or so (based on your accumulated domain knowledge, plugin development, and specific scenarios that you want to cover). At this point any spikes / prototypes for technical challenges should be planned and executed and we can agree on an implementation path and execute it.

Btw by design document I mean for example this one for hot module replacement feature: here(tracking) and here (design doc)

[m-abs]

Hi @manoldonev

We've started writing a design doc. https://docs.google.com/document/d/1gybrT1IQh8Fb5r0v_tZTMS06yP6xYmr9pDQIBRbokZc/edit?usp=sharing

It is still WIP, but we'll get there.

[aashaya]

Hi @m-abs,

I'm also facing many ada issues with nativescript app like - side drawer isn't focused after double tap open, images not selected on swipe etc etc. I'm going to start with nota and see how it goes. But would be ideal if Nativescript framework supported ada.

[m-abs]

Hi @aashaya,

Sorry about the late reply, I've been very busy on other tasks.

We'd love to have upstream support for this, so we wouldn't have to maintain our plugin anymore.

When I have a moment, I'll update the design doc with what we've learn since when.

Our plugin don't have support for the nativescript-ui-* plugins, simply because we don't use them ourselves. But we'd be happy to accept PR adding support for them.

If you want to chat more about this, I'm on slack under the same handle :)

[aashaya]

Thank you @m-abs. I got sidetracked but will let you know as I make progress on my app.