gaozhao1989 / pyjab

Python implementation for Java application UI automation with Java Access Bridge
GNU General Public License v2.0
49 stars 20 forks source link

我好像发现一个很大的BUG,希望能紧急修复 #41

Closed xinchenll closed 2 years ago

xinchenll commented 2 years ago

今天我用这个方法: element.click(simulate=True) 发现没有反映,于是我看了下你的源码,在上面加了打印,如下: if simulate: self.win32_utils._set_window_foreground(hwnd=self.hwnd.value) x = self.bounds.get("x") y = self.bounds.get("y") print(self._acc_info) print(x) print(y) width = self.bounds.get("width") height = self.bounds.get("height") print(width) print(height) if width == 0 or height == 0: raise ValueError("element width or height is 0") position_x = round(x + width / 2) position_y = round(y + height / 2) self.win32_utils._click_mouse(x=position_x, y=position_y)

最后的输出是这样: <pyjab.accessibleinfo.AccessibleContextInfo object at 0x0000000004B99640> -1 -1 -1 -1

按我的理解 self._acc_info这里应该是记录关控件的属性信息,然而它貌似是个对像,再往下去我就不好查了,反正最终获得的坐标不对,难怪没有反应,我觉得这个问题非常严重,希望快些修复。

xinchenll commented 2 years ago

我用的是1.1.4最新版的

xinchenll commented 2 years ago

@gaozhao1989 尽快

xinchenll commented 2 years ago

@gaozhao1989 我的QQ: 86543973 走向何方,方便的话请加一个

jsa34 commented 2 years ago

@xinchenll I believe this may be related to the AccessibleContextInfo being cached at object creation for a JABElement. I fixed this in #39 to dynamically get the AccessibleContextInfo on calling the properties that are using contextinfo.

Also, this is an open source project, and so, as kind as @gaozhao1989 is to dedicate his time to developing this, if it's urgent then I would suggest digging in the source and submitting a PR to resolve the issue (if the fix above doesn't solve it!)

jsa34 commented 2 years ago

Try replacing: self._acc_info

With: self._get_accessible_context_info()

gaozhao1989 commented 2 years ago

Hi @xinchenll

感谢您的反馈。我以重新在 1.1.4 版本中测试这个问题,但遗憾的是并不能复现您的场景。另外,我有注意到您留的信息中有打印控件的 bounds 信息,且值全部都为 -1。通常情况下,当控件处于“不可见”的状态时就会以此状态返回全部为 -1 值的 bounds 信息。自然以此状态,模拟点击必然是无法产生预期的动作。 如果您所控制的控件支持 jab 点击 api,我建议您去除对于模拟点击的控制再尝试,感谢

Hi @jsa34 Thanks for your quick support on this issue. The component from previous information does not have correct attribute value for "bounds"(all -1). Need more information about this component.

xinchenll commented 2 years ago

@jsa34 非常感谢你的回复,按你的要求在原码中加入: print(self._acc_info) print(self._get_accessible_context_info()) a=self._get_accessible_context_info() print(x) print(a.x)

打印为: <pyjab.accessibleinfo.AccessibleContextInfo object at 0x0000000004A02640> <pyjab.accessibleinfo.AccessibleContextInfo object at 0x00000000049BAD40> -1 -1

还是不行,但是今天我有新发现在操作主窗体的按钮点击是可以的,上面不行的是在子窗体中

我水平有限,没看懂你上面在 #39 中提到修复方案,在源码中应该如何修改呢

jsa34 commented 2 years ago

As per the comment I left on #39, I thought it may be because the element was cached in a hidden state and then became "visible" and had bound attributes that weren't -1 later, but the current implementation in 1.1.4 would always have the original state and never update the accessible context.

Do you have an example project/Java form with this element behaviour for investigating?

xinchenll commented 2 years ago

HI,@gaozhao1989 @jsa34 按你的思路,我在代码里再加上打印,如下: if simulate: self.win32_utils._set_window_foreground(hwnd=self.hwnd.value) x = self.bounds.get("x") y = self.bounds.get("y") width = self.bounds.get("width") height = self.bounds.get("height") print(self.bounds) print(self.states_en_us) if width == 0 or height == 0: raise ValueError("element width or height is 0") position_x = round(x + width / 2) position_y = round(y + height / 2) 输出为: {'x': -1, 'y': -1, 'height': -1, 'width': -1} ['enabled', 'focusable', 'opaque']

的确如你所说并没有 visible 这个属性,但是我很好奇这与我在 AccessBridgeExplore看到的不一样,我截图如下: QQ截图20220221052605

而且我也很好奇,就是同一个应用,有些窗体的按钮是可以实现模拟点击,有些窗体又不可以,我感觉这里面还是可以研究一下,只可惜在您那重现不了。

我为什么这么重视模拟点击??因为我发现我的应用有些时候在用非模拟点击时,接下来弹出的窗口会被阻塞,而模拟点击则不会,这也是我感觉困惑的地方。

gaozhao1989 commented 2 years ago

Hi @xinchenll

理论上不应该出现 inspector 与 pyjab 显示属性不一致的情况。 请尝试在 pyjab 获取到控件 bounds 属性都为 -1 后中断/断点,刷新 inspector 查看控件属性

xinchenll commented 2 years ago

Hi @gaozhao1989 按您的方式尝试在源码打上断点,输出为-1后刷新AccessBridgeExplore, 在上面看到很正常,如同我上面截的图。

gaozhao1989 commented 2 years ago

您也可以在 pyjab 输出对应控件 bounds 属性值均为 -1 时尝试打印该控件的状态(status)观察该控件是否可见

gaozhao1989 commented 2 years ago

Hi @xinchenll,

在本地多次尝试后依旧不能复现此场景,能否给出一些公用可供重现的场景?

gaozhao1989 commented 2 years ago

Close due to not reproduce