At first, we extracted a relatively small patch from the
screenshots which corresponds to the Region of Interest (ROI)
of the user's interaction by taking the mouse click point as a
reference point. Here, ROI refers to a region of the screenshot
where the user executes a mouse click such as a button, icon,
text field, etc. This processing step is very crucial because
information regarding the user interaction depends on a
relatively small part of the taken screenshots. Therefore using
the ROI reduces noise and computational effort. Later we
categorized these ROI images, based on the user's Activity,
using two different neural network architectures.
At first, we extracted a relatively small patch from the screenshots which corresponds to the Region of Interest (ROI) of the user's interaction by taking the mouse click point as a reference point. Here, ROI refers to a region of the screenshot where the user executes a mouse click such as a button, icon, text field, etc. This processing step is very crucial because information regarding the user interaction depends on a relatively small part of the taken screenshots. Therefore using the ROI reduces noise and computational effort. Later we categorized these ROI images, based on the user's Activity, using two different neural network architectures.
from https://ieeexplore.ieee.org/abstract/document/10242692