dtmilano / AndroidViewClient

Android ViewServer and ADB client
Apache License 2.0
1.63k stars 347 forks source link

Hangs when trying dump a ui which rendered by game engine #141

Open brantxiong opened 9 years ago

brantxiong commented 9 years ago

As I know, We can not get ui elements or structure on screen which rendered by game engine such as cocos2d-x. In this case, The culebra wil hangs showing "Please wait...", the output script is self.vc.dump(window=-1).

Would you help me solved that problem? Since I want culebra to work under this situation. I would like to Touch by PX on such UI.

dtmilano commented 9 years ago
  1. AndroidViewClient/culebra can use different back-ends. The default back-end for API >= 19 is UiAutomator. It may not find an idle state, so you may want to try other alternatives.
  2. To verify the condition run adb shell uiautomator dump
  3. Strictly speaking, if you are going to touch by PX, there's no need to download the View tree, but I think culebra does not consider this edge case. It might be possible to add it.
brantxiong commented 9 years ago
  1. Since I am using Android 4.3(API =18), the back-ends maybe "Force ViewServer use" branch. And I tried culebra on Android 4.4.4(API=19), The culebra could show sreen renderd by game engine successfully which using the "Use UiAutomator" back-end. And I could touch by PX through culebra.
  2. The adb shell uiautomator dump output is as following on screen rendered by game engine on Android 4.3 devices:
<?xml version='1.0' encoding='UTF-8' standalone='yes' ?>
<hierarchy rotation="0">
    <node index="0" text="" resource-id="" class="android.widget.FrameLayout" package="com.test.test" content-desc=""
          checkable="false" checked="false" clickable="false" enabled="true" focusable="false" focused="false"
          scrollable="false" long-clickable="false" password="false" selected="false" bounds="[0,0][720,1184]">
        <node index="0" text="" resource-id="" class="android.widget.LinearLayout" package="com.test.test"
              content-desc="" checkable="false" checked="false" clickable="false" enabled="true" focusable="false"
              focused="false" scrollable="false" long-clickable="false" password="false" selected="false"
              bounds="[0,0][720,1184]">
            <node index="0" text="" resource-id="android:id/content" class="android.widget.FrameLayout"
                  package="com.test.test" content-desc="" checkable="false" checked="false" clickable="false"
                  enabled="true" focusable="false" focused="false" scrollable="false" long-clickable="false"
                  password="false" selected="false" bounds="[0,0][720,1184]">
                <node index="0" text="" resource-id="" class="android.widget.FrameLayout" package="com.test.test"
                      content-desc="" checkable="false" checked="false" clickable="false" enabled="true"
                      focusable="false" focused="false" scrollable="false" long-clickable="false" password="false"
                      selected="false" bounds="[0,0][720,1184]">
                    <node index="0" text="" resource-id="" class="android.view.View" package="com.test.test"
                          content-desc="" checkable="false" checked="false" clickable="false" enabled="true"
                          focusable="true" focused="true" scrollable="false" long-clickable="false" password="false"
                          selected="false" bounds="[0,0][720,1184]"/>
                    <node NAF="true" index="1" text="" resource-id="" class="android.widget.EditText"
                          package="com.test.test" content-desc="" checkable="false" checked="false" clickable="true"
                          enabled="true" focusable="true" focused="false" scrollable="false" long-clickable="true"
                          password="false" selected="false" bounds="[0,0][720,96]"/>
                </node>
            </node>
        </node>
    </node>
</hierarchy>
  1. Does culebra has the option that force NOT to download the View tree? Maybe under this way, The culebra could works fine with game engine rendered screen on Android device (API < 19).
dtmilano commented 9 years ago

You may find useful the latest version (v10.5.1) which introduces null backend support (culebra --null-back-end). Let me know if it solves your problem.