locateOnScreen gives wrong coordinates

[ifuchs]

I am using locateOnScreen to find an icon on my Mac screen. It works (returns a result) except the coordinates are 2x what they should be. Is this related to the fact that I have 2 monitors?

[damies13]

I think it's more likely because you have a retina display on your mac, have a look at #589, there is a workaround provided there.

[ifuchs]

Thanks! Dividing the coords by 2 seems to be the fix. I guess it is related to the full resolution of the display as opposed to what it is set to show.

On Feb 25, 2022, at 8:41 PM, Dave @.***> wrote:

I think it's more likely because you have a retina display on your mac, have a look at #589 https://github.com/asweigart/pyautogui/issues/589, there is a workaround provided there.

— Reply to this email directly, view it on GitHub https://github.com/asweigart/pyautogui/issues/671#issuecomment-1051430685, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABLDPSGGOXZ4L4DXKCQDQTLU5AVT7ANCNFSM5PLMZW2A. You are receiving this because you authored the thread.

[damies13]

Yes it is,

Actually if you need to deal with windows machines as well in your code, then the workaround in this comment is better it gives you 2.0 (200%) on a Mac with a retina display, but will also give you the scaling setting on windows that can be anything from 100% to 500%

https://github.com/asweigart/pyautogui/issues/589#issuecomment-876287189

The factor 2 is devicePixelRatio. pixelRatio = pyautogui.screenshot().size[0]/pyautogui.size().width

[RanaOsamaAsif]

@damies13 I'm struggling to have this workaround in my code. Any help would be very much appreciated.

My goal is to pick a pixel on the screen and run some code when that pixel color changes. The code will only run on macOS. My plan is to capture an initial screenshot of a specific region and then capture screenshots of the same region in a loop.

I've checked this StackOverflow question which mentions that simply doing pyautogui.pixel() will not give the correct color of the pixel under mouse pointer.

My code:

import pyautogui

while True:
     x, y = pyautogui.position()
     pixelRatio = pyautogui.screenshot().size[0]/pyautogui.size().width
     xNew = x / pixelRatio
     yNew = y / pixelRatio
     pyautogui.screenshot("./apple.png", region=(nNew, yNew, 50,50))

But the capture region is very off from where the mouse was clicked.

Development machine is MacBook Pro (Retina, 15-inch, Mid 2015)

[damies13]

Hi @RanaOsamaAsif,

I'll do my best to explain (i'm in hospital at the moment so i'm a little restricted)

The issue with screen scaling is that the cursor position clicked was using post scaled co-ordinates but locateOnScreen was giving pre scaled (raw) co-ordinates, so when we detected the location on the screen we had to manually scale the co-ordinates to click in the right place.

Looking at your code it looks like you are trying to do the opposite, you are trying to take a screen shot of the area under the mouse cursor, so you might need to actually scale the other way.

import pyautogui

while True:
     x, y = pyautogui.position()
     pixelRatio = pyautogui.screenshot().size[0]/pyautogui.size().width
     xNew = x * pixelRatio
     yNew = y * pixelRatio
     pyautogui.screenshot("./apple.png", region=(nNew, yNew, 50,50))

     x, y = pyautogui.position()

This gets the cursor co-ordinates (post scaled)

     pixelRatio = pyautogui.screenshot().size[0]/pyautogui.size().width

This gives you the pixelRatio (2.0 aka 200%) for a retina display

     xNew = x * pixelRatio
     yNew = y * pixelRatio

the original solution converted the raw values to scaled values so was dividing the raw by the ratio, in your case you have the scaled values and want to convert them to raw, so need to multiply the scaled values by the ratio to get raw values

I hope that makes sense and clarifies things, hopefully my foggy brain got it right and it works for you 🤞🏻

Dave.

[RanaOsamaAsif]

This worked perfectly.

Thank you so much for writing a comprehensive reply!