Testing/integration tests

akmal-threepointsix commented 1 year ago

Main changes:

Added ImGUI Test Engine (CMakeLists.txt, emulator/lib/ImGUI_TestEngine)
Updated ImGUI, because older version was incompatible with ImGUI Test Engine (emulator/lib/ImGUI)
Added UI tests for Alarm and Timer (emulator/src/UiTests/OswAppAlarm.cpp, emulator/src/UiTests/OswAppTimer.cpp)
Updated README.md with new testing chapter
I didn't change any production code, because testing must not interfere with business logic. The only exception is that I added friend class in include/apps/main/switcher.h to access private fields of OswAppSwitcher.

Finally, I know you want to see how the tests look like. Here is the demo in max speed:

https://user-images.githubusercontent.com/30224608/219116887-a7c91519-42e1-4795-b345-2ec11aa44a2a.mp4

And here is the normal speed:

https://user-images.githubusercontent.com/30224608/219117057-7f0c12be-1816-41ab-a263-5da0e1b07e02.mp4

Closes #340

akmal-threepointsix commented 1 year ago

This PR should be merged after https://github.com/Open-Smartwatch/open-smartwatch-os/pull/339

Also, it should be linked with the issue https://github.com/Open-Smartwatch/open-smartwatch-os/issues/340

simonmicro commented 1 year ago

I love this idea! I really look forward to integrate those - would it be possible to somehow also run them in the pipelines? Problem is, that there is no display to render the window into (I think), so they would need to run headless :sweat_smile:

akmal-threepointsix commented 1 year ago

Yes, I was also thinking about adding these tests to the github pipeline, but running headless emulator did not work 😞. Testing engine has to visually see buttons, because then it generally moves cursor to these buttons and clicks them and do other stuff.

So for now we can try to add to pipelines the current version with window emulator, maybe it will work... If not, then I will also look if it is possible to do it without windows (very unlikely).

akmal-threepointsix commented 1 year ago

BTW, I forgot to mention, that there is an automatic mode for UI tests. If you execute the command ./emulator.run --ui_auto it will automatically show the emulator window, run all UI tests and then return success or failure.

I documented this command in README.md

akmal-threepointsix commented 1 year ago

I have one question regarding adding new libraries. Do I have to add ImGUI_TestEngine to the platformio.ini? Since this library is only used in the emulator, I have not added it as lib dependency in the platformio.ini. Is it correct?

Here is what I actually did with ImGUI_TestEngine:

I added ImGUI_TestEngine library as a submodule in emulator/lib.
I added it as a library to the CMakeLists.txt, so that building the project with make builds ImGUI_TestEngine as well.

akmal-threepointsix commented 1 year ago

Ok, another question. I need to fix merge conflicts, but when I am doing git merge develop on my branch testing/integrationTests, git does not update all files correctly. E.g.: OswAppAlarm still has class member with old name alarms, while in develop branch it is called notifications. And there are a lot of such files that do not get changes, or maybe my branch overwrites changes from develop.

How I can fix this?

simonmicro commented 1 year ago

I have one question regarding adding new libraries. Do I have to add ImGUI_TestEngine to the platformio.ini? Since this library is only used in the emulator, I have not added it as lib dependency in the platformio.ini. Is it correct?

Here is what I actually did with ImGUI_TestEngine:
1. I added `ImGUI_TestEngine` library as a submodule in `emulator/lib`.

2. I added it as a library to the `CMakeLists.txt`, so that building the project with `make` builds `ImGUI_TestEngine` as well.

This is completely correct - any emulator library does not go into the platformio.ini.

simonmicro commented 1 year ago

Ok, another question. I need to fix merge conflicts, but when I am doing git merge develop on my branch testing/integrationTests, git does not update all files correctly. E.g.: OswAppAlarm still has class member with old name alarms, while in develop branch it is called notifications. And there are a lot of such files that do not get changes, or maybe my branch overwrites changes from develop.

How I can fix this?

In those cases I would recommend you to rebase, as it should skip already applied commits. Otherwise, you could use git rebase -i ... to skip those bad commits or to fix some of them :)

simonmicro commented 1 year ago

Heyho @akmal-threepointsix , would it be possible to get this branch ready to merge in the next merge window starting tomorrow? Otherwise I'll go ahead and close this PR soon :tm: . Thanks!

akmal-threepointsix commented 1 year ago

Hi, sorry for silence, university is over and the work has started, brrr 😬.

I will try to merge on this weekend for the last time, the problem was this interactive rebase, which is extremely confusing and exhausting 😕. However, I spent a lot of time on these tests and it would be nice to see them in official codebase.

I will also refactor my code a little bit, because I got an idea on how to make adding new UI tests much cleaner and easier.

akmal-threepointsix commented 12 months ago

UPD: Still working...

I am doing something strictly forbidden in git community 😐, namely I created a new branch from up-to-date develop branch and now adding my integration tests in my newer branch. After that I will open a new PR and we will close this one without merging it.

Open-Smartwatch / open-smartwatch-os

Testing/integration tests #351