RaiMan / SikuliX-2014

SikuliX version 1.1.2 (until February 2018)
http://sikulix.com
805 stars 235 forks source link

click() doesn't click #260

Open kenorb opened 7 years ago

kenorb commented 7 years ago

I've this Python script on macOS Sierra:

app = App("BlueStacks")
app.focus()
region = Region(App.focusedWindow())
region.click("1496165553436.png")

when run in SikulixIDE 1.1.1 against BlueStacks, the search image is highlighted (which means it's found), mouse moves on the right spot, but the click is not registered, despite in log it's successful:

[log] App.focus: [28084:BlueStacks]
[log] highlight M[970,408 93x130]@S(0) for 2.0 secs
[log] highlight M[970,408 93x130]@S(0) for 2.0 secs
[log] CLICK on L(1016,473)@S(0)[0,0 1920x1200] (522 msec)

Screenshot of the app:

screen shot 2017-05-30 at 18 14 04

click-does-not-click

RaiMan commented 7 years ago

already tried: hover(image) wait(someShortTime) click()

kenorb commented 7 years ago

So what's point of click(some_img) syntax (having image to find as argument), when you need to still hover on the image and click it.

Although I've tried suggested code, but it doesn't work as expected either.

app = App("BlueStacks")
app.focus()
region = Region(App.focusedWindow())
region.hover("search_button.png")
wait(2)
click()

After clicking Run from editor, the window gets focused correctly, the right image is blinking for a second (twice), then mouse moves on it from Run button (in IDE) into that image, nothing happens for 2 seconds, then mouse moves back to the middle of the screen. What I'm doing wrong?

Maybe I need to move the mouse first using different function, then click on it, as it seems hover doesn't move the mouse cursor.

glitchassassin commented 7 years ago

If you're using region.hover, you'll need to use region.click(). Each region's recent matches are tracked separately.

RaiMan commented 7 years ago

@glitchassassin Thanks for the comment ;-) Too fast 4 me ;-)

RaiMan commented 7 years ago

@kenorb

So what's point of click(some_img) syntax (having image to find as argument), when you need to still hover on the image and click it.

This is only to test possible timing problems with your GUI. Hover moves the mouse pointer to the location (your image in this case). This "mouse over" might be registered by your GUI. the region.click() just clicks the last match in region.

Another option instead of region.click() would be to use

click(Mouse.at())

which would simply click at the current position of the mouse pointer.

kenorb commented 7 years ago

For some reason it doesn't click, despite the right location and that the click is registered in the log:

[log] App.focus: [343:BlueStacks]
[log] highlight M[71,474 85x82]@S(0) for 2.0 secs
[log] highlight M[71,474 85x82]@S(0) for 2.0 secs
Before wait: L(113,515)@S(0)[0,0 1920x1200]
After wait: L(113,515)@S(0)[0,0 1920x1200]
[log] CLICK on L(113,515)@S(0)[0,0 1920x1200] (523 msec)
[log] CLICK on L(113,515)@S(0)[0,0 1920x1200] (524 msec)

Code:

app = App("BlueStacks")
app.focus()
region = Region(App.focusedWindow())
image = region.hover("1496310394356.png")
print "Before wait: ", Mouse.at()
wait(2)
print "After wait: ", Mouse.at()
click(Mouse.at())
wait(2)
click(Mouse.at())

Maybe it's just BlueStacks app specific issue. I'll try to find some workaround and test that with different app or on different operating system.

glitchassassin commented 7 years ago

Is BlueStacks running as root? Is Sikuli running as root?

kenorb commented 7 years ago

All apps default ran as a user. The mouse moves fine (so SikuliX has access to control it), but it doesn't click (it 'think' it did, but there is no any after effect). Manually clicking on the same region works fine.

kenorb commented 7 years ago

Tested also with Screen Sharing app by sharing screen locally (so it's different app), but it is the same issue with click(). Mouse is on the right spot, but click doesn't do anything. However it works with region.doubleClick(png) (despite I need only single click)! Maybe it's possible to longer the effect of click() (press and release time), so I can test? How?