How to locate and click the element in popupwindow.

[QingXiangBaiLian]

I can't locate the elements in popupwindow via findViewWithText or findViewWithTextOrRaise method, and there are many elements need to be click, thus I can't measure all the locations manually.

[dtmilano]

Can you please provide more context for your problem. How does the layout look like? Sceeenshot, intentions, etc.

[QingXiangBaiLian]

@dtmilano Very thanks for your quick reply！ As our app is still under developing, for business secrecy is not convenient to display here。So I give an equivalent description using a Demo as follows. The problem is I want to detect all the elements in the PopupWindow automatically and click some of them，like the button ‘A’ , 'B' and 'C' in the bellow picture As these buttons were in the PopupWindow, I can't get them via getting the page_source using appium，even couldn't be detect by uiautomatorviewer，but I can find them in the hierarchyviewer. This question have boring me for a week, I have used Culebra GUI, however the generated code using self.device.touchDip() method to click the button's coordinates directly instead of detect the button first. And now, I have made a big forward. At the beginning I using vc.dump(-1) to dump all the window then using vc.findViewWithText(u'A')to find the Button A, it return None. But if I only dump the PopupWindow (Given the ID, such as 123, then vc.dump(123) ), it turns out that I can find the Button A. And I can using vc.list() to get all the window IDs and names, then find the PopupWindow's ID. but I still struggle to list all the elements' Text after dump the PopupWindow, I know vc.traverse() can print the Text, but I want to return the Text, then using that in vc.findViewWithText().

[dtmilano]

By default, AndroidViewClient/culebra relies on the Views included in the tree produced by uiautomator dump (backend) and as you have verified already, it doesn't contain Views from other windows.

AndroidViewClient/culebra has the ability to use other backends. You should have been using ViewServer as you were dumping the content of other windows. It might be possible to find a solution but ViewServer is old and does not run in secure mode (see wiki).

There's still another backend, the newest one, called uiautomatorhelper or CulebraTester. Which I'm describing here via an example similar to your case.

CulebraTester details can be found at culebra.dtmilano.com.

Once you have the 2 APKs installed on your device (see Getting started), you start it and open the page with your desktop browser.

In this case, I'm using a demo app that shows a PopupWindow. You can verify that the DISMISS button appears in the hierarchy, and when you click it, the generated test includes the line to select and click it.

The complete generated test is

#! /usr/bin/env python
# -*- coding: utf-8 -*-
'''
Copyright (C) 2013-2018  Diego Torres Milano
Created on 2018-04-14 by CulebraTester 
                      __    __    __    __
                     /  \  /  \  /  \  /  \ 
____________________/  __\/  __\/  __\/  __\_____________________________
___________________/  /__/  /__/  /__/  /________________________________
                   | / \   / \   / \   / \   \___
                   |/   \_/   \_/   \_/   \    o \ 
                                           \_____/--<
@author: Diego Torres Milano
@author: Jennifer E. Swofford (ascii art snake)
'''

import re
import sys
import os

import unittest
try:
    sys.path.insert(0, os.path.join(os.environ['ANDROID_VIEW_CLIENT_HOME'], 'src'))
except:
    pass

import pkg_resources
pkg_resources.require('androidviewclient>=12.4.0')
from com.dtmilano.android.viewclient import ViewClient, CulebraTestCase
from com.dtmilano.android.uiautomator.uiautomatorhelper import UiAutomatorHelper, UiScrollable, UiObject, UiObject2

TAG = 'CULEBRA'

class CulebraTests(CulebraTestCase):

    @classmethod
    def setUpClass(cls):
        cls.kwargs1 = {'ignoreversioncheck': False, 'verbose': False, 'ignoresecuredevice': False}
        cls.kwargs2 = {'forceviewserveruse': False, 'useuiautomatorhelper': True, 'ignoreuiautomatorkilled': True, 'autodump': False, 'startviewserver': True, 'compresseddump': True}
        cls.options = {'start-activity': None, 'concertina': False, 'device-art': None, 'use-jar': False, 'multi-device': False, 'unit-test-class': True, 'save-screenshot': None, 'use-dictionary': False, 'glare': False, 'dictionary-keys-from': 'id', 'scale': 1, 'find-views-with-content-description': True, 'window': -1, 'orientation-locked': None, 'save-view-screenshots': None, 'find-views-by-id': True, 'log-actions': False, 'use-regexps': False, 'null-back-end': False, 'auto-regexps': None, 'do-not-verify-screen-dump': True, 'verbose-comments': False, 'gui': False, 'find-views-with-text': True, 'prepend-to-sys-path': False, 'install-apk': None, 'drop-shadow': False, 'output': None, 'unit-test-method': None, 'interactive': False}
        cls.sleep = 5

    def setUp(self):
        super(CulebraTests, self).setUp()

    def tearDown(self):
        super(CulebraTests, self).tearDown()

    def preconditions(self):
        if not super(CulebraTests, self).preconditions():
            return False
        return True

    def testSomething(self):
        if not self.preconditions():
            self.fail('Preconditions failed')

        _s = CulebraTests.sleep
        _v = CulebraTests.verbose

        self.vc.uiAutomatorHelper.findObject(bySelector='res@com.dtmilano.android.demoapplication:id/dismiss,clazz@android.widget.Button,text@DISMISS,package@com.dtmilano.android.demoapplication').clickAndWait(eventCondition='until:newWindow', timeout=_s*1000)

if __name__ == '__main__':
    CulebraTests.main()

which when run and the popup is showing it clicks on the button and dismisses it.

If you try to generate the same test with culebra you'll obtain

    def testSomething(self):
        if not self.preconditions():
            self.fail('Preconditions failed')

        _s = CulebraTests.sleep
        _v = CulebraTests.verbose

        self.vc.dump(window=-1)
        self.device.touchDip(214.1, 636.19, 0)

as culebra cannot discover the button in the popup window due to the uiautomator dump limitation and has to produce a touch by coordinates.

[QingXiangBaiLian]

Very thanks for your answer, that's solved my question perfectly!