moses-palmer / pynput

Sends virtual input commands
GNU Lesser General Public License v3.0
1.77k stars 245 forks source link

Using pynput to record or playback events for GUI application testing #471

Closed smontanaro closed 2 years ago

smontanaro commented 2 years ago

It seems to me that pynput could be the basis for a cross-platforum GUI testing tool. (Does something already exist built atop this package?) I think a couple things might be necessary though. One, capturing enter, exit, focus in, focus out, and the like would be good. Two, being able to restrict recording and playback to the application being tested would be a big plus.

Feasible, already done, or not?

Finally, a nice-to-have would be a (short? comprehensive? fun?) list of open source applications using pynput, maybe on the wiki...

moses-palmer commented 2 years ago

Thank you for your suggestions!

Nothing like this built upon this library exists as far as I know. I have however used GUI testing libraries in the past, and they tend to integrate with the platform accessibility frameworks instead, to avoid having to manage pointers, windows and focus management. A few also use a combination of screenshotting and sub-image comparison to locate buttons and other components.

Anything built on pynput would probably be of the latter kind, so in addition to integrating with the platform toolkits to manage focus, screen recording would also have to be implemented.

This is quite a lot of work, and out of scope for this library. It could be part of a larger testing toolkit however.

As for your last suggestion, I agree that a list of open source applications using this library would be very nice to have. I do not really know which ones are however, as most feedback I receive are bug reports and feature requests.

I hope this comment has given you a few ideas on how to proceed. I will close this issue as I do not think there is more to do in this library, If you think otherwise, please reopen.