Not able to Touch Button with Text or By ID in android4.4

[AbhinavMohit]

I am not able to touch the Button in my application in a dialog Box which appears after clicking on one button. I have tried with both 'findviewwithText' and 'findviewbyID'. Not able to understand why it is not working. Interesting part is there is label on Dialog Box which is detected fine but not buttons.

Please assist ASAP. Thanks in Advance

[dtmilano]

Please provide the required details to be able to reproduce the issue (e.g., dialog code, dump, etc.)

[AbhinavMohit]

We are using custom android dialog box like Linearlayout,textview in XML and In our java code we are using Android Dialog builder class. We are unable to share our Java code due to security reasons in our company. In Python this is our Method::::: def clickText(text, outfile): try: vc.dump() onTextClick = vc.findViewWithTextOrRaise(text) x = onTextClick.getX() y = onTextClick.getY() w = onTextClick.getWidth() h = onTextClick.getHeight() print x,y,w,h device.touch(x+w/2, y+h/2, "DOWN_AND_UP") if onTextClick is not None: onTextClick.touch() print 'View touched' return True else: outfile.write('view not found \n') print 'View not found' return False except Exception, e: print 'Exception On Hold' outfile.write('Exception On Hold: \n')

[dtmilano]

I don't know if this solves your problem (probably not), but I would do:

def clickText(text, outfile):
try:
    vc.dump()
    vc.findViewWithTextOrRaise(text).touch()
except Exception, e:
    print 'Exception On Hold'
    outfile.write('Exception On Hold: \n')

I understand you cannot share the app code, but at least try to isolate the problem and create some sample code so I can reproduce it and eventually fix it.

On Mon, Feb 10, 2014 at 2:01 AM, AbhinavMohit notifications@github.comwrote:

We are using custom android dialog box like Linearlayout,textview in XML and In our java code we are using Android Dialog builder class. We are unable to share our Java code due to security reasons in our company. In Python this is our Method::::: def clickText(text, outfile): try: vc.dump() onTextClick = vc.findViewWithTextOrRaise(text) x = onTextClick.getX() y = onTextClick.getY() w = onTextClick.getWidth() h = onTextClick.getHeight() print x,y,w,h device.touch(x+w/2, y+h/2, "DOWN_AND_UP") if onTextClick is not None: onTextClick.touch() print 'View touched' return True else: outfile.write('view not found \n') print 'View not found' return False except Exception, e: print 'Exception On Hold' outfile.write('Exception On Hold: \n')

Reply to this email directly or view it on GitHubhttps://github.com/dtmilano/AndroidViewClient/issues/76#issuecomment-34604997 .

Have you read my blog ? http://dtmilano.blogspot.com android junit tests ui linux cult thin clients

[mwasilew]

I ran into similar issue using 'touch()'. I'm running vellamo (https://play.google.com/store/apps/details?id=com.quicinc.vellamo&hl=en_GB). After installing it asks for accepting EULA. Calling 'touch' on the 'Accept' button doesn't work.

[mwasilew]

bisecting shows that last good version of the code is 7fc8b7f8c618ec4a2cca67f46d43b40f4cf02ed5 which corresponds to version 7.6.0

[dtmilano]

Couldn't reproduce the problem with latest version (8.17.1). I was using culebra to generate the test cases and I recommend you to do so. culebra was able to touch the Button in all the cases I could think of.

and this is one of the generated scripts:

#! /usr/bin/env python
# -*- coding: utf-8 -*-
'''
Copyright (C) 2013-2014  Diego Torres Milano
Created on 2014-11-25 by Culebra v8.17.1
                      __    __    __    __
                     /  \  /  \  /  \  /  \ 
____________________/  __\/  __\/  __\/  __\_____________________________
___________________/  /__/  /__/  /__/  /________________________________
                   | / \   / \   / \   / \   \___
                   |/   \_/   \_/   \_/   \    o \ 
                                           \_____/--<
@author: Diego Torres Milano
@author: Jennifer E. Swofford (ascii art snake)
'''

import re
import sys
import os

import unittest

from com.dtmilano.android.viewclient import ViewClient, CulebraTestCase

class CulebraTests(CulebraTestCase):

    @classmethod
    def setUpClass(cls):
        cls.kwargs1 = {'ignoreversioncheck': False, 'verbose': False, 'ignoresecuredevice': False}
        cls.kwargs2 = {'startviewserver': True, 'forceviewserveruse': False, 'autodump': False, 'ignoreuiautomatorkilled': True}
        cls.options = {'start-activity': None, 'multi-device': False, 'unit-test-class': True, 'save-screenshot': None, 'use-dictionary': False, 'dictionary-keys-from': 'id', 'scale': 0.5, 'find-views-with-content-description': False, 'window': -1, 'orientation-locked': None, 'save-view-screenshots': None, 'find-views-by-id': True, 'do-not-verify-initial-screen-dump': True, 'use-regexps': False, 'auto-regexps': None, 'use-jar': False, 'verbose-comments': False, 'gui': True, 'find-views-with-text': True, 'prepend-to-sys-path': False, 'output': '/tmp/vellamo-eula-3.py', 'unit-test-method': None, 'interactive': False}
        cls.sleep = 5

    def setUp(self):
        super(CulebraTests, self).setUp()

    def tearDown(self):
        super(CulebraTests, self).tearDown()

    def preconditions(self):
        if not super(CulebraTests, self).preconditions():
            return False
        return True

    def testSomething(self):
        if not self.preconditions():
            self.fail('Preconditions failed')

        self.vc.dump(window=-1)
        self.vc.findViewWithTextOrRaise(u'Accept').touch()
        self.vc.sleep(CulebraTests.sleep)
        self.vc.dump(window=-1)

if __name__ == '__main__':
    CulebraTests.main()

The problem could be the different in case between the text property ('Accept') and the UI ('ACCEPT'). If you can provide more details, like versions, devices, script, etc. used perhaps I could tell you something different.

[mwasilew]

I'm using older version of vellamo (1.0.6) on a dev board with Linaro Android build. I'll try to collect all components and links and describe the problem with more detail. I was hoping it would be reproducible with any vellamo but apparently it isn't :(

[dtmilano]

The output of dump -d could also be useful to help determine the problem.

On Wed, Nov 26, 2014 at 10:20 AM, mwasilew notifications@github.com wrote:

I'm using older version of vellamo (1.0.6) on a dev board with Linaro Android build. I'll try to collect all components and links and describe the problem with more detail. I was hoping it would be reproducible with any vellamo but apparently it isn't :(

— Reply to this email directly or view it on GitHub https://github.com/dtmilano/AndroidViewClient/issues/76#issuecomment-64660558 .

Have you read my blog ? http://dtmilano.blogspot.com android junit tests ui linux cult thin clients

[mwasilew]

OK, I think I found the problem - it nails down to coordinates. I went through the touch() call step by step. The coordinates of the button to touch are x=224, y=1187 (center point). Variable orientation is set to 2. After calling __transformPointByOrientation the new coordinates are x=1187 and y=1056. If I call

adb shell input tap 1187 1056

nothing happens. On the other hand when I call

adb shell input tap 224 1187

the proper button is tapped. So I suspect AVC is a bit confused by screen rotation.

[mwasilew]

One more bit of information. Here is how the dumpsys describes the display:

  mDefaultViewport=DisplayViewport{valid=true, displayId=0, orientation=3, logicalFrame=Rect(0, 0 - 720, 1280), physicalFrame=Rect(0, 0 - 720, 1280), deviceWidth=720, deviceHeight=1280}
  mExternalTouchViewport=DisplayViewport{valid=false, displayId=0, orientation=0, logicalFrame=Rect(0, 0 - 0, 0), physicalFrame=Rect(0, 0 - 0, 0), deviceWidth=0, deviceHeight=0}

So the orientation "2" doesn't appear anywhere. I'm wondering where does it come from. Is there a way to tell AVC about the screen orientation?

[dtmilano]

Are you rotating the device during the test run?

To verify the orientation used by AVC:

print ViewClient.connectToDeviceOrExit()[0].display

Show the script you are running so your problem can be understood and reproduced.

Furthermore, culebra features an option to consider the orientation should remain locked during the test run:

  -O, --orientation-locked         orientation locked in generated test

[mwasilew]

No, I'm not rotating the screen. The board doesn't have accelerometer to detect such changes anyway.

I think I found a bug (it might be different than original bug report). The touch() call looks as follows (https://github.com/dtmilano/AndroidViewClient/blob/master/src/com/dtmilano/android/viewclient.py#L872):

self.device.touch(x, y, type)

The function definition looks as follows (https://github.com/dtmilano/AndroidViewClient/blob/master/src/com/dtmilano/android/adb/adbclient.py#L588):

def touch(self, x, y, orientation=-1, eventType=DOWN_AND_UP):

By very unfortunate coincidence DOWN_AND_UP is defined as 2 (https://github.com/dtmilano/AndroidViewClient/blob/master/src/com/dtmilano/android/adb/adbclient.py#L56)

So what happens is call to touch(224 , 1187, 2). The screen orientation is still 3, so the tap goes into wrong place. I changed the call in viewclient.py to

self.device.touch(x, y, eventType=type)

And this solves the problem for me. So far I didn't go through all other touch() occurences to prepare a patch. I'll try to do that today. I guess changing DOWN_AND_UP (and other constants) in adbclient.py to some other values (like 102, so they don't overlap with other constants) would help tracking this bug.