[Accessibility - Android] <ion-input> is not acknowledged by Android Talkback

[bryplano]

I'm submitting a...

[x] bug report (ionic-angular 3.9.8) [ ] feature request

Current behavior:

No highlighting or audible notice is provided by Android Talkback when selecting an <ion-input> field. Additionally, the field cannot be edited.

Expected behavior:

Any <ion-input> field should have the following behavior from an accessibility standpoint (to mimic a regular HTML input field):

1. Highlighted (green box)
2. Described in some manner by Android Talkback
3. The field should also be editable (the keyboard should be displayed and the user allowed to input text, just like a normal HTML input)

Steps to reproduce:

This was reproduced on a Google Pixel 2 (physical device) with Android 9.0

Prerequisite: Enable Talkback on the Android device: https://support.google.com/accessibility/android/answer/6007100?hl=en

1. Install the sample application from the Github repo and use ionic cordova run android to get the app on your connected Android device
2. Attempt to tap to the right of any of the three text labels with "Ion Input"
3. Notice that you cannot tap on any of the areas where you would normally be able to type into; you also don't receive any visual or audio feedback
4. Compare to the basic input fields at the bottom (under "HTML inputs")

Related code:

https://github.com/bryplano/AndroidTalkbackTest

Other information:

May be related to a very old v2 issue: https://github.com/ionic-team/ionic-v3/issues/69

ionic info

   Ionic CLI          : 5.2.3 
   Ionic Framework    : ionic-angular 3.9.8
   @ionic/app-scripts : 3.2.4

Cordova:

   Cordova CLI       : 8.1.2 (cordova-lib@8.1.1)
   Cordova Platforms : android 8.0.0
   Cordova Plugins   : cordova-plugin-ionic-keyboard 2.1.3, cordova-plugin-ionic-webview 4.1.1, (and 4 other plugins)

Utility:

   cordova-res : 0.6.0 
   native-run  : 0.2.8 

System:

   Android SDK Tools : 26.1.1 
   ios-deploy        : 1.9.4
   ios-sim           : 8.0.1
   NodeJS            : v10.16.0 
   npm               : 6.10.1
   OS                : macOS Mojave

[nascarjake]

I am also unable to tap ion-inputs with an android 9 device. My issue is identical to that described here: https://github.com/apache/cordova-android/issues/804

The guys over at cordova-android said it might not be them. As described in that issue ticket, if i tap on a button or an image or anything besides an input, I can then click on the input and the keyboard opens. Doing this while inspecting the page I do see the ion-item and the ion-input flash purple as if their class tag changed, however upon very careful inspection nothing is really changing in any of these elements.

Inspecting the input box shows that the "first tapped element" according to z-index is the input-cover which has all the css pointer events and touch events you'd expect. After tapping on another element first, tapping on this input-cover focuses the ion-input as expected.

Its very strange and only happens on android 9. Android 8 and ios devices have no issues at all.

[StefanRein]

Hey,

I checked here: https://github.com/apache/cordova-android/issues/804

That it works by commenting exactly out this line of code: https://github.com/ionic-team/ionic-v3/blob/master/src/components/input/input.ts#L614

I will open a pull request.

[StefanRein]

My temporary fix is right now: Create a directive and override the current method:

import { Directive, Self } from '@angular/core';
import { TextInput } from 'ionic-angular';
import { hasPointerMoved, pointerCoord } from 'ionic-angular/util/dom';

@Directive({ selector: 'ion-input,ion-textarea' })
export class TextInputDirective {
    constructor(@Self() textInput: TextInput) {
        textInput._pointerEnd = TextInputDirective.__pointerEnd.bind(textInput);
    }

    private static __pointerEnd = function(this: TextInput, ev: UIEvent) {
        if ((this._isTouch && ev.type === 'mouseup') || !(this as any)._app.isEnabled()) {
            // the app is actively doing something right now
            // don't try to scroll in the input
            ev.preventDefault();
            ev.stopPropagation();
        } else if (this._coord) {
            // get where the touchend/mouseup ended
            const endCoord = pointerCoord(ev);

            // focus this input if the pointer hasn't moved XX pixels
            // and the input doesn't already have focus
            if (!hasPointerMoved(8, this._coord, endCoord) && !this.isFocus()) {
                // ev.preventDefault(); @see https://github.com/ionic-team/ionic-v3/issues/1049
                ev.stopPropagation();

                // begin the input focus process
                this._jsSetFocus();
            }
        }

        this._coord = null;
    };
}

[StevenH86]

Have this issue also.... As my app always opens to a login screen or a dashboard with no inputs, my temp fix is to simply apply display: none to .input-cover for my login screen. Doesn't seem to have negative impacts, so I am going to roll with that until it's fixed properly.

[liamdebeasi]

As a workaround, you should be able to set role="textbox" on the ion-input.

[StevenH86]

As a workaround, you should be able to set role="textbox" on the ion-input.

This didn't work for me.

[liamdebeasi]

@StevenH86 What device/Android version are you testing this on?

[StefanRein]

@liamdebeasi Removing my workaround and changing it with your suggestion does not work.

Install Android Studio with a virtual device, e.g. Pixel XL API 28. The targeting API needs to be 28. I had this issue with the Android 9 from Samsung and Android One. How did you test this? Which Device? OS? API? On the Huawei with API 27 it is working with and without this fix.

[StevenH86]

@StevenH86 What device/Android version are you testing this on?

@liamdebeasi

Cordova Android 8.0.0 API 28

Devices tested on: Samsung s9, Pixel 3, Samsung s5, xiaomi mi9.... Also emulators with android 4.4 to 9.

Only solution that worked was the one I mentioned above.

[StefanRein]

@StevenH86 Mine did not work? Did you try? To remove the preventDefault()?

[StevenH86]

@StevenH86 Mine did not work? Did you try? To remove the preventDefault()?

You have to put this in your scss

.input-cover { display: none !important; }

but it might only work if it's also wrapped in .md { }. Haven't tried with out. But that has worked for me and tested on multiple devices.

[liamdebeasi]

A few things I'm noticing:

1. When running the example app in the original post, TalkBack does not seem to announce both ion-inputs and native inputs when tapping.
2. When browsing the web in Chrome, tapping a native input does cause TalkBack to announce it.
3. When pressing the screen and then dragging your finger over a native input in both Chrome and in the example app, TalkBack announces the input.
4. When pressing the screen and then dragging your finger over ion-input elements with role="textbox", TalkBack announces the ion-input. (TalkBack does not announce any other ion-inputs)

I am testing this on a Moto G4 running Android 7.0. Is anyone else able to reproduce these behaviors?

[StefanRein]

@liamdebeasi

I just realized the following: I did not know what is actually Android Talkback and this issue was linked somewhere else, so I did not care what this is. Actually this is some screenreader for accessibility.

What I actually solved was following problem:

App start > Login Screen > First tap is on ion-input > Problem: Keyboard does not appear. But this happens only on Android 9 / API 28.

If you tap somewhere else first (on an image for example), the Keyboard will appear on the next tap on the ion input. And it was possible to double tap the ion-input (what is not expected by users.. to open the keyboard) Again: This happened only if you initially opened the application and tried with your FIRST tap to tap on an ion-input field.

To sum it up: I think the keyboard issue I am trying to solve (and thought this issue is about) has nothing to do with a screenreader. I am sorry.

@StevenH86 @liamdebeasi

You actually just disabled the whole functionality of this cover with setting it to display: none.

https://github.com/ionic-team/ionic-v3/blob/20dfbdd3037fa86dcda2a5ce07a2476ae344a9ee/src/components/input/input.ts#L134

What is actually causing the issue is the touchend event which calls the _pointerEnd method, which has the ev.preventDefault(); call. This is preventing the keyboard showing up on the initially tap on Android 9.

This line here: https://github.com/ionic-team/ionic-v3/blob/20dfbdd3037fa86dcda2a5ce07a2476ae344a9ee/src/components/input/input.ts#L614

[StevenH86]

@StefanRein

I am aware of that solution, but I prefer not to make modifications to the ionic code as it can result in unsuspected issues.

The resolution I used has been rolled out and tested with over 10,000 users with 0 reported issues. So, for me removing the input-cover is the solution until ionic releases an official fix.

[StefanRein]

@StevenH86 Thanks! Good to know.

Yes, when I checked your solution I verified that on iOS it did still work with autoscrolling the input element into the view etc. And I tested this because actually you were removing much more code in your solution (actually the whole functionality of the input-cover) with removing (display: none => no event listeners will fire) the input-cover class.

'<div class="input-cover" *ngIf="_useAssist" ' +
    '(touchstart)="_pointerStart($event)" ' +
    '(touchend)="_pointerEnd($event)" ' +
    '(mousedown)="_pointerStart($event)" ' +
    '(mouseup)="_pointerEnd($event)"></div>',

https://github.com/ionic-team/ionic-v3/blob/20dfbdd3037fa86dcda2a5ce07a2476ae344a9ee/src/components/input/input.ts#L134

I suspect the preventDefault() to prevent giving the webview the initial active state or something like that.

But again thanks for the information that nothing weird happened right now, which leads me to the question: What does the input-cover actually do?

Because it is still scrolling into the view and the hiding caret while scrolling in iOS (known bug since 2011 or so in webkit) is also working fine.

[StevenH86]

@StefanRein

I honestly do not know what it actually does. I haven't had a lot of free time to look into it further, I just needed a solution that worked.

As the only screen in my app that it is currently possible to reach on an initial load that has an input field is the login screen, I don't mind rolling with a basic/hacky fix until a permanent solution is made by the ionic team. I only hide the input cover on the login screen, as after any other touch action is done within the app, inputs work correctly, even with the cover.

I did however notice if you tap an input on initial load, even though nothing appears to happen, if you minimize your app and then bring it back up the keyboard will appear and you are actually in the input field. Strange issue.

[utpaul]

@StefanRein if you share a simple project by using your directive, i think it will be helpful

[alex-steinberg]

@utpaul I have updated my project which illustrates the bug (from #1056) with the fix: https://github.com/alex-steinberg/ion-input-v3-issue

[bryplano]

@StevenH86 @StefanRein, are you still discussing the Android Talkback issue that is defined in the reproduction steps in this issue, or are you discussing a separate issue (and if so, that should be its own Github issue)? I honestly can't follow the convo anymore...

[bryplano]

@liamdebeasi - so your suggestion only partially fixes the issue. The box can be highlighted and is properly announced, but it is still not editable via a double-tap on the screen.

I have updated my sample repo with your suggestion. See https://github.com/bryplano/AndroidTalkbackTest/blob/master/src/pages/home/home.html#L15

[StefanRein]

@bryplano Yes sorry, we actually talked about the issue from Alex, which sounded very similar to me: https://github.com/ionic-team/ionic-v3/issues/1056 I realized later that your issue is a separated one and not a duplicate to the one mentioned above.

Is this talkback visual, too or only sound? (I asked our android users and no one could show me this talkback thing)

[bryplano]

I mentioned this in the initial post:

you also don't receive any visual or audio feedback

So normally the input is surrounded (this is on my test device) with a green box and there is an audio cue that says "edit box" ... "double tap to insert text". You can then tap twice on the screen to input text (which places a cursor and brings up the keyboard) That does not happen with an ion-input.

When I tried @liamdebeasi's solution with role="textbox", then the input is properly highlighted and the audio cue occur, but you can't edit the text by double-tapping.

Note: This bug does not occur in an Ionic 4 application. Best LTS solution would be to upgrade to Ionic 4.

[bryplano]

@StevenH86, to your point earlier about:

You have to put this in your scss

.input-cover { display: none !important; }

but it might only work if it's also wrapped in .md { }. Haven't tried with out. But that has worked for me and tested on multiple devices.

I just tried that out in a sample application:

.md .input-cover {
    display: none;
}

And that appears to make Talkback work in all situations except if the role is being set:

  <ion-item padding>
    <ion-label>Textbox Type Ion Input</ion-label>
    <ion-input type="textbox"></ion-input>
  </ion-item>

  <ion-item padding>
    <ion-label>Text Type Ion Input</ion-label>
    <ion-input type="text"></ion-input>
  </ion-item>

  <!-- This one will be selected & announced, but double-tap to input does not work -->
  <ion-item padding>
    <ion-label>Role Textbox Ion Input</ion-label>
    <ion-input role="textbox"></ion-input>
  </ion-item>

  <ion-item padding>
    <ion-label>Basic Ion Input</ion-label>
    <ion-input></ion-input>
  </ion-item>

Realistically I think that's a fair workaround for folks who run into this :D

[asimon91]

Have this issue also.... As my app always opens to a login screen or a dashboard with no inputs, my temp fix is to simply apply display: none to .input-cover for my login screen. Doesn't seem to have negative impacts, so I am going to roll with that until it's fixed properly.

I love you. Spent 3 hours on this issue. Thank you!

[tattivitorino]

@StefanRein your directive fixed my problem!!! thank you so much!! @alex-steinberg i based my fix on your project on git! thanks a lot!!!

[richardshergold]

@bryplano thanks - we had this problem in our Ionic 3 app (for Android 9 users only) and this fix seems to have sorted it. Is there a fix due in Ionic 3 for this?

[DevHaldar]

My temporary fix is right now: Create a directive and override the current method:

import { Directive, Self } from '@angular/core';
import { TextInput } from 'ionic-angular';
import { hasPointerMoved, pointerCoord } from 'ionic-angular/util/dom';

@Directive({ selector: 'ion-input,ion-textarea' })
export class TextInputDirective {
    constructor(@Self() textInput: TextInput) {
        textInput._pointerEnd = TextInputDirective.__pointerEnd.bind(textInput);
    }

    private static __pointerEnd = function(this: TextInput, ev: UIEvent) {
        if ((this._isTouch && ev.type === 'mouseup') || !(this as any)._app.isEnabled()) {
            // the app is actively doing something right now
            // don't try to scroll in the input
            ev.preventDefault();
            ev.stopPropagation();
        } else if (this._coord) {
            // get where the touchend/mouseup ended
            const endCoord = pointerCoord(ev);

            // focus this input if the pointer hasn't moved XX pixels
            // and the input doesn't already have focus
            if (!hasPointerMoved(8, this._coord, endCoord) && !this.isFocus()) {
                // ev.preventDefault(); @see https://github.com/ionic-team/ionic-v3/issues/1049
                ev.stopPropagation();

                // begin the input focus process
                this._jsSetFocus();
            }
        }

        this._coord = null;
    };
}

Thanks it fixed the problem! +1

[DevHaldar]

just wanted to ask one more thing that dont know why one of my app with similar config didnt need this fix but the other one needed this fix. Im a bit confused

[danishashfaq]

@StefanRein I did tried your solution in an ionic 3 app it is working fine. But same solution didn't worked in an ionic 2 app I have. I am facing same issue in an ionic 2 app and created same directive with some adjustment with focus methods;

 import { Directive, Self } from '@angular/core';
import { TextInput } from "ionic-angular";
import { hasPointerMoved, pointerCoord } from 'ionic-angular/util/dom';

/**
 * Directive courtesy https://github.com/StefanRein
 */

@Directive({
    selector: 'ion-input' // Attribute selector
})
export class IonInputFixDirective {

    constructor( @Self() textInput: TextInput) {
        textInput.pointerEnd = IonInputFixDirective.__pointerEnd.bind(textInput);
    }

    private static __pointerEnd = function (this: TextInput, ev: UIEvent) {
        // input cover touchend/mouseup
        if ((this._isTouch && ev.type === 'mouseup') || !(this as any)._app.isEnabled()) {
            // the app is actively doing something right now
            // don't try to scroll in the input
            ev.preventDefault();
            ev.stopPropagation();
        }
        else if (this._coord) {
            // get where the touchend/mouseup ended
            var endCoord = pointerCoord(ev);
            // focus this input if the pointer hasn't moved XX pixels
            // and the input doesn't already have focus
            if (!hasPointerMoved(8, this._coord, endCoord) && !this.hasFocus()) {
                //ev.preventDefault(); @see https://github.com/ionic-team/ionic-v3/issues/1049
                ev.stopPropagation();
                // begin the input focus process
                this.initFocus();
            }
        }
        this._coord = null;
    };

}

Can anybody help me?