[QUESTION/BUG] Wrong point found with specific needle on haystack

[henriquelino]

Hello! I've been stuck with this for the last couple of days, I have a PDF with some fields I need to click, the fields are light blue, so I search these using pyautogui.locateCenterOnScreen, but as I tracked down to pyscreeze._locateAll_opencv, I will use this on demo code

What happens is:

• When using cv.matchTemplate, it finds the correct position: Box(left=30, top=243, width=8, height=9)
• When using pyscreeze it finds many boxes at top of image but none is correct

I'm glad to help if needed, but atm I don't know what to do anymore besides using matchTemplate for this exact set of needle/haystack, this might be a bug, so I guess I should open this issue

---

Some info:

• OS: Windows 10 Pro Version 21H2
• Python: 3.9.13
• PyAutogui: 0.9.53
• pyscreeze: 0.1.28

  ---

  The code:

import pyscreeze import cv2

def template_match(haystack, needle): """from: https://stackoverflow.com/questions/7853628/how-do-i-find-an-image-contained-within-an-image"""

method = cv2.TM_SQDIFF_NORMED

haystack = cv2.imread(haystack)
needle = cv2.imread(needle)

h, w = needle.shape[:2]

needle_w, needle_h = needle.shape[:2]

result = cv2.matchTemplate(needle, haystack, method)

minVal, maxVal, minLoc, maxLoc = cv2.minMaxLoc(result)
x, y = minLoc

return pyscreeze.Box(x, y, needle_w, needle_h)

if name == 'main':

haystack = r".\haystack.png"
needle = r".\needle.png"

correct_box = template_match(haystack, needle)

boxes: list[pyscreeze.Box] = list(pyscreeze._locateAll_opencv(needle, haystack))
for box in boxes:
    print(f"Pyscreeze matched {box}")

print(f"{correct_box = }")

___
# The images:
- Needle:
![needle](https://user-images.githubusercontent.com/57471416/198023405-f1d21e77-68bf-4ade-a077-58e532f292c1.png)

- haystack:
![haystack](https://user-images.githubusercontent.com/57471416/198023360-f8da95fe-cc4f-46ee-9785-e499ae743b27.png)