search

appium / appium-uiautomator2-server

Appium UiAutomator/UiObject2-based server for Android UI automation. This module is used by appium-uiautomator2-driver component
Apache License 2.0
328 stars 232 forks source link

Regression - can no longer click on EditText in v4.12.3 #377

Open sshock opened 4 years ago

sshock commented 4 years ago

Something happened in v4.12.3 (likely in 6cf1e05) that has made it so I can no longer click on some EditText elements for month, day, and year in an app I am automating.

I am still able to find the elements, but when I try to click on them nothing happens. Normally when clicking on them the element becomes focused and the text gets highlighted as seen in the screenshot below.

I'm not sure why I can no longer click on these because they are EditText elements and they do have clickable true attribute.

The screenshot and scene tree below are after clicking on day "01" using uiautomator2 v4.12.2. With v4.12.3 I can't click on it.

date_picker_screenshot

<?xml version='1.0' encoding='UTF-8' standalone='yes' ?>
<hierarchy index="0" class="hierarchy" rotation="3" width="720" height="1232">
  <android.widget.FrameLayout index="0" package="com.test" class="android.widget.FrameLayout" text="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" long-clickable="false" password="false" scrollable="false" selected="false" bounds="[18,335][702,618]" displayed="true">
    <android.widget.FrameLayout index="0" package="com.test" class="android.widget.FrameLayout" text="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" long-clickable="false" password="false" scrollable="false" selected="false" bounds="[18,335][702,618]" displayed="true">
      <android.widget.FrameLayout index="0" package="com.test" class="android.widget.FrameLayout" text="" resource-id="android:id/content" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" long-clickable="false" password="false" scrollable="false" selected="false" bounds="[18,335][702,618]" displayed="true">
        <android.widget.LinearLayout index="0" package="com.test" class="android.widget.LinearLayout" text="" resource-id="android:id/parentPanel" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" long-clickable="false" password="false" scrollable="false" selected="false" bounds="[26,335][694,618]" displayed="true">
          <android.widget.FrameLayout index="0" package="com.test" class="android.widget.FrameLayout" text="" resource-id="android:id/customPanel" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" long-clickable="false" password="false" scrollable="false" selected="false" bounds="[26,335][694,555]" displayed="true">
            <android.widget.FrameLayout index="0" package="com.test" class="android.widget.FrameLayout" text="" resource-id="android:id/custom" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" long-clickable="false" password="false" scrollable="false" selected="false" bounds="[34,343][686,555]" displayed="true">
              <android.widget.DatePicker index="0" package="com.test" class="android.widget.DatePicker" text="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" long-clickable="false" password="false" scrollable="false" selected="false" bounds="[34,343][686,555]" displayed="true">
                <android.widget.LinearLayout index="0" package="com.test" class="android.widget.LinearLayout" text="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" long-clickable="false" password="false" scrollable="false" selected="false" bounds="[236,343][484,555]" displayed="true">
                  <android.widget.LinearLayout index="0" package="com.test" class="android.widget.LinearLayout" text="" resource-id="android:id/pickers" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" long-clickable="false" password="false" scrollable="false" selected="false" bounds="[236,343][484,555]" displayed="true">
                    <android.widget.NumberPicker index="0" package="com.test" class="android.widget.NumberPicker" text="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" long-clickable="false" password="false" scrollable="true" selected="false" bounds="[244,359][308,539]" displayed="true">
                      <android.widget.Button index="0" package="com.test" class="android.widget.Button" text="Dec" checkable="false" checked="false" clickable="true" enabled="true" focusable="false" focused="false" long-clickable="true" password="false" scrollable="false" selected="false" bounds="[244,359][308,425]" displayed="true" />
                      <android.widget.EditText index="1" package="com.test" class="android.widget.EditText" text="Jan" resource-id="android:id/numberpicker_input" checkable="false" checked="false" clickable="true" enabled="true" focusable="true" focused="false" long-clickable="true" password="false" scrollable="false" selected="false" bounds="[244,425][308,473]" displayed="true" />
                      <android.widget.Button index="2" package="com.test" class="android.widget.Button" text="Feb" checkable="false" checked="false" clickable="true" enabled="true" focusable="false" focused="false" long-clickable="true" password="false" scrollable="false" selected="false" bounds="[244,473][308,539]" displayed="true" />
                    </android.widget.NumberPicker>
                    <android.widget.NumberPicker index="1" package="com.test" class="android.widget.NumberPicker" text="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" long-clickable="false" password="false" scrollable="true" selected="false" bounds="[324,359][388,539]" displayed="true">
                      <android.widget.Button index="0" package="com.test" class="android.widget.Button" text="31" checkable="false" checked="false" clickable="true" enabled="true" focusable="false" focused="false" long-clickable="true" password="false" scrollable="false" selected="false" bounds="[324,359][388,425]" displayed="true" />
                      <android.widget.EditText index="1" package="com.test" class="android.widget.EditText" text="01" resource-id="android:id/numberpicker_input" checkable="false" checked="false" clickable="true" enabled="true" focusable="true" focused="true" long-clickable="true" password="false" scrollable="false" selection-start="0" selection-end="2" selected="false" bounds="[324,425][388,473]" displayed="true" />
                      <android.widget.Button index="2" package="com.test" class="android.widget.Button" text="02" checkable="false" checked="false" clickable="true" enabled="true" focusable="false" focused="false" long-clickable="true" password="false" scrollable="false" selected="false" bounds="[324,473][388,539]" displayed="true" />
                    </android.widget.NumberPicker>
                    <android.widget.NumberPicker index="2" package="com.test" class="android.widget.NumberPicker" text="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" long-clickable="false" password="false" scrollable="true" selected="false" bounds="[404,359][468,539]" displayed="true">
                      <android.widget.Button index="0" package="com.test" class="android.widget.Button" text="1969" checkable="false" checked="false" clickable="true" enabled="true" focusable="false" focused="false" long-clickable="true" password="false" scrollable="false" selected="false" bounds="[404,359][468,425]" displayed="true" />
                      <android.widget.EditText index="1" package="com.test" class="android.widget.EditText" text="1970" resource-id="android:id/numberpicker_input" checkable="false" checked="false" clickable="true" enabled="true" focusable="true" focused="false" long-clickable="true" password="false" scrollable="false" selected="false" bounds="[404,425][468,473]" displayed="true" />
                      <android.widget.Button index="2" package="com.test" class="android.widget.Button" text="1971" checkable="false" checked="false" clickable="true" enabled="true" focusable="false" focused="false" long-clickable="true" password="false" scrollable="false" selected="false" bounds="[404,473][468,539]" displayed="true" />
                    </android.widget.NumberPicker>
                  </android.widget.LinearLayout>
                </android.widget.LinearLayout>
              </android.widget.DatePicker>
            </android.widget.FrameLayout>
          </android.widget.FrameLayout>
          <android.widget.LinearLayout index="1" package="com.test" class="android.widget.LinearLayout" text="" resource-id="android:id/buttonPanel" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" long-clickable="false" password="false" scrollable="false" selected="false" bounds="[26,555][694,618]" displayed="true">
            <android.widget.LinearLayout index="0" package="com.test" class="android.widget.LinearLayout" text="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false" long-clickable="false" password="false" scrollable="false" selected="false" bounds="[34,556][686,610]" displayed="true">
              <android.widget.Button index="0" package="com.test" class="android.widget.Button" text="Cancel" resource-id="android:id/button2" checkable="false" checked="false" clickable="true" enabled="true" focusable="true" focused="false" long-clickable="false" password="false" scrollable="false" selected="false" bounds="[34,556][359,610]" displayed="true" />
              <android.widget.Button index="1" package="com.test" class="android.widget.Button" text="OK" resource-id="android:id/button1" checkable="false" checked="false" clickable="true" enabled="true" focusable="true" focused="false" long-clickable="false" password="false" scrollable="false" selected="false" bounds="[360,556][686,610]" displayed="true" />
            </android.widget.LinearLayout>
          </android.widget.LinearLayout>
        </android.widget.LinearLayout>
      </android.widget.FrameLayout>
    </android.widget.FrameLayout>
  </android.widget.FrameLayout>
</hierarchy>
mykola-mokhnach commented 4 years ago

Since https://github.com/appium/appium-uiautomator2-server/commit/6cf1e05c7578c2118c7d7f45ed354df993090f82 the code now calculating element bounds for clicking with respect to simpleBoundsCalculation setting value. So you could either try to set this setting to false while clicking or use touch actions to calculate the click coordinates in the client code.

Also, it makes sense to enable touches assistance in dev settings, so then you could see where click actually happens.

sshock commented 4 years ago

Great, thanks I will try your suggestions!

I'm curious though, do you find it surprising that an EditText inside a NumberPicker would not have the correct bounds with simpleBoundsCalculation true?

Also, can you explain the reasoning for changes in 6cf1e05? Does it clean up the code? Or is it supposed to improve performance? If it's for performance it might be one step forward one step back if it means we have to stop using simpleBoundsCalculation. Well, I guess it's not that bad since we can turn it on and off as needed and probably still have it be on most of the time...

mykola-mokhnach commented 4 years ago

I assume this particular control has a scrollable parent, which affects its actual bounds. And click is performed exactly in the middle of this bounds rect.

Does it clean up the code?

It fixes a potential issue where bounds shown in XML page source are not in sync with actual element bounds used for making operations like click on it. It also makes sense from performance perspective.

sshock commented 4 years ago

@mykola-mokhnach unfortunately, setting simpleBoundsCalculation to false didn't help.

I made these two screen recordings to first, illustrate the problem, and second, show how it was working before:

v4.12.3 broken: https://drive.google.com/file/d/1CuRoOHvu6J_DTD9eqyMw_SKser9eMZbL/view

v4.12.2 working: https://drive.google.com/file/d/1uDGE3Coll0U3bvbito2k3_U43RaVdlG6/view

mykola-mokhnach commented 4 years ago

Thanks for the detailed videos. It looks like the problem is not with coordinates, but rather clicking logic itself. Let me try some other approach

mykola-mokhnach commented 4 years ago

Could you please try if https://github.com/appium/appium-uiautomator2-server/pull/378 fixes the clicking?

sshock commented 4 years ago

Could you please try if #378 fixes the clicking?

Sure thing. I should get time to test it this afternoon...

sshock commented 4 years ago

Could you please try if #378 fixes the clicking?

@mykola-mokhnach it works!

mykola-mokhnach commented 4 years ago

Cool, I've published the patch