横屏下的XY轴错误

moomiji commented 1 year ago

想试着给自己写的一个简单脚本从adb input换个模拟库，横屏下XY轴貌似有问题。麻烦大佬看看是哪里出的问题？

from adbutils import adb

class Connector(MNTDevice):
    def start(self):
        logger.debug(adb.connect(self.device_id))
        self.device = adb.device(self.device_id)
        self.device.screenshot()
        return super().start()

    def print_window_size(self):
        logger.debug("screen size: {}, max_x: {}, max_y: {}".format(
            self.device.window_size(),
            self.connection.max_x,
            self.connection.max_y))

>>> device_id = "192.168.1.110:33445"                        
>>> d = Connector(device_id)
2023-01-12 00:29:23.496 | DEBUG    | Helper.ADBConnector:start:9 - already connected to 192.168.1.110:33445
2023-01-12 00:29:27.926 | INFO     | pyminitouch.utils:is_device_connected:56 - device ONEPLUS A5000 online
2023-01-12 00:29:27.926 | INFO     | pyminitouch.connection:__init__:87 - searching a usable port ...
2023-01-12 00:29:29.941 | INFO     | pyminitouch.connection:__init__:89 - device 192.168.1.110:33445 bind to port 20069
2023-01-12 00:29:30.089 | INFO     | pyminitouch.connection:get_abi:35 - device 192.168.1.110:33445 is arm64-v8a
2023-01-12 00:29:30.231 | INFO     | pyminitouch.connection:__init__:27 - minitouch already existed in 192.168.1.110:33445
2023-01-12 00:29:30.236 | DEBUG    | pyminitouch.connection:_forward_port:128 - forward command: adb -s 192.168.1.110:33445 forward tcp:20069 localabstract:minitouch
2023-01-12 00:29:30.262 | DEBUG    | pyminitouch.connection:_forward_port:130 - output: b''
2023-01-12 00:29:30.265 | INFO     | pyminitouch.connection:_start_mnt:141 - start minitouch: adb -s 192.168.1.110:33445 shell /data/local/tmp/minitouch
2023-01-12 00:29:31.375 | INFO     | pyminitouch.connection:__init__:183 - minitouch running on port: 20069, pid: 23422
2023-01-12 00:29:31.390 | INFO     | pyminitouch.connection:__init__:186 - max_contact: 10; max_x: 1079; max_y: 1919; max_pressure: 0

>>> d.print_window_size()
2023-01-12 00:29:33.148 | DEBUG    | Helper.ADBConnector:print_window_size:15 - screen size: WindowSize(width=1920, height=1080), max_x: 1079, max_y: 1919

其它信息

内核名及版本：x-base-v2-20221007-1a-test adb shell getprop:

[ro.kernel.version]: [4.4]

[ro.bootimage.build.date.utc]: [1668276871]
[ro.bootimage.build.fingerprint]: [OnePlus/OnePlus5/OnePlus5:12/SQ3A.220705.004/1816:userdebug/release-keys]
[ro.bootimage.build.id]: [SQ3A.220705.004]
[ro.bootimage.build.version.release]: [12]
[ro.bootimage.build.version.sdk]: [32]

[org.pixelexperience.build_date]: [20221112-1816]
[org.pixelexperience.build_date_utc]: [1668276960]
[org.pixelexperience.build_type]: [OFFICIAL]
[org.pixelexperience.device]: [cheeseburger]
[org.pixelexperience.version]: [twelve_plus]
[org.pixelexperience.version.display]: [PixelExperience_Plus_cheeseburger-12.1-20221112-1816-OFFICIAL]

williamfzc commented 1 year ago

可以参考下之前有个同学《基于pytorch框架用resnet101加GPT搭建AI玩王者荣耀》的源码；他的驱动就是pyminitouch，也是横屏，似乎没收到其他该类反馈；

moomiji commented 1 year ago

了解了，传递给minitouch的是触摸坐标而不是屏幕坐标，需要坐标映射。要不要考虑把坐标映射也封装进去，类似STFService.apk那样？

moomiji commented 1 year ago

Use of screen coordinates requires coordinate mapping when device is rotated, like this case.

    def coordinate_mapping(self, x: int, y: int) -> tuple:
        """
        Mapping from screen coordinates to touch coordinates
        x: x-coordinate of screen
        y: y-coordinate of screen
        return: touch coordinates for minitouch
        """
        max_x = int(self.connection.max_x)
        max_y = int(self.connection.max_y)
        w = self.device.window_size().width
        h = self.device.window_size().height

        if self.need_coordinate_mapping:
            r = self.device.rotation()
        else:
            return (x, y)

        if 0 == r:
            x = int(x/w*max_x)
            y = int(y/h*max_y)
            return (x, y)
        elif 1 == r:
            x = int(x/w*max_y)
            y = int(y/h*max_x)
            return (max_x - y, x)
        elif 2 == r:
            x = int(x/w*max_x)
            y = int(y/h*max_y)
            return (max_x - x, max_y - y)
        elif 3 == r:
            x = int(x/w*max_y)
            y = int(y/h*max_x)
            return (y, max_y - x)

williamfzc commented 1 year ago

了解了，传递给minitouch的是触摸坐标而不是屏幕坐标，需要坐标映射。要不要考虑把坐标映射也封装进去，类似STFService.apk那样？

可以的，不过暂时我这边没什么这种场景，觉得有必要的话欢迎PR哈：）

williamfzc / pyminitouch

横屏下的XY轴错误 #19

其它信息