search

RaiMan / SikuliX-2014

SikuliX version 1.1.2 (until February 2018)
http://sikulix.com
806 stars 234 forks source link

Sikuli can not test itself #221

Closed UnitedMarsupials-zz closed 8 years ago

UnitedMarsupials-zz commented 8 years ago

I'm porting SikuliX to CentOS-7 (for my employer) and FreeBSD (for my own pleasure) and got it to build and run after some serious modifications (patches available upon request).

However, I can not reliably test the results because SikuliX does not have a self-test suit. What little there is under TestScripts/ is hardly comprehensive and is mostly broken anyway.

Wouldn't it be great, if a GUI application designed to test GUI applications were able to test itself? And it can -- it is just that no such tests have been created or, at least, none that are bundled with the application's source code...

In my opinion, this deficiency is hampering the development efforts -- unless the developers actually do have such a test-suit, but simply aren't sharing it... How do you fix a bug or add a feature without being able to automatically confirm, that the change has not accidentally broken something else?

I'd say, creating such a suit should have a very high priority...

RaiMan commented 8 years ago

Currently a regression test suite is under development, which besides helping testing after bug fixing in version 1.1.x will also be the base for making sure, that version 2 will be backwards compatible.

Version 2 will have basic function tests available from the beginning.

UnitedMarsupials-zz commented 8 years ago

Is the current test suit already usable -- even if it does not offer full coverage? (If such full coverage is even possible in principle, that is.)

Even if it provides merely a "smoke test" at this time, it is much better than nothing at all. How would I use it?

RaiMan commented 8 years ago

https://github.com/RaiMan/sikuli-regression-tests

I added a ReadMe.