canonical / mir-ci

Mir CI helpers
1 stars 1 forks source link

Fix checking against frame_count in Screencopy.match #119

Closed hbatagelo closed 4 months ago

hbatagelo commented 5 months ago

self.frame_count accessed in match() could contain a frame number that is out of sync with the grabbed screenshot. This fix makes sure that the frame count checked in match() is the actual frame count when grab_screenshot() is called.

AlanGriffiths commented 5 months ago

I was under the impression that the async code is only run while in the await statement

My understanding is the same - in Python it is more like co-routines, than threading.

hbatagelo commented 5 months ago

I think this issue may be related to how the event loop schedules the callbacks. What I've observed is that ScreencopyTracker._frame_ready may be called just after grab_screenshot returns. If that happens, _frame_ready increases frame_count, but the returned image still refers to the previous frame.

When I log the values of frame_count and last_checked_frame_count in the while loop of Screencopy.match, sometimes I encounter the following scenario that motivates this PR:

# 1st iteration
screenshot = await asyncio.wait_for(self.grab_screenshot(), timeout) # In grab_screenshot, self.frame_count is some number, e.g., 10
# screenshot refers to the frame with self.frame_count==10
# But self_frame_count is now 11 because _frame_ready was scheduled to run after grab_screenshot was being waited for
if last_checked_frame_count != self.frame_count: # True because this is the first iteration and last_checked_frame_count==0
    last_checked_frame_count = self.frame_count # OK, last_checked_frame_count is now 11
    # find_template_in_image is called. Let's assume the template doesn't match with this screenshot

# 2nd iteration
screenshot = await asyncio.wait_for(self.grab_screenshot(), timeout) # In grab_screenshot, self.frame_count is 11
# screenshot refers to the frame with self.frame_count==11
# This time, self_frame_count is still 11 because there were no new frames
if last_checked_frame_count != self.frame_count: # False
    last_checked_frame_count = self.frame_count # Won't be reached, but screenshot may contain the matching template
    # ...

# If no additional frames are created, the loop will time out, even if screenshot contains the matching template