search

laposa / test-portal

Laravel Application integrated with Github Actions
https://laposa.ie
MIT License
1 stars 0 forks source link

Combined full test run #95

Open MartinMiksovskyLaposa opened 1 day ago

MartinMiksovskyLaposa commented 1 day ago

Consider having a full regression testing/security updates testing round. This means all of the test cases (manual ones) shall be ran.

The idea behind automated tests is to avoid manual job & save time. Simply - converting the manual job (manual test case) to the automated process (automated test case) following the exact steps as described in manual tests.

Having precisely automated manual test case = no need to run it manually as long as it passes = saved time (that could be used for ad hoc testing/different job)

In a term of Test Portal I'd like to have something like a "Complete run".

This run will follow these steps:

1) Run all automated tests available for the selected service/group of services/applications 2) After that, the manual run will trigger including only the test cases that were not a part of automated run. Explanation: we actually want to use the benefit/idea of automated testing & increment the saved time with every new automated test.

IMO having added the manual test cases for the automated tests that failed does not worth the effort. I can do a separate manual run selected only these which failed OR simply analyze the results of the automated tests following standard process: 1) if it is a bug, create an issue & reply 2) if it is a bug in the test itself, create an issue, fix the test.

I just don't want to manually check every test case if it was already ran automatically or not, it is a waste of time & cutting off the benefit of automation testing

Also I would appreciate to have an access to the manual test case from automated test result.

e.g. if 00_smoke_test fails, I'd love to have a opportunity to open a manual test case in the UI. Reason: It would save some time for us to do a manual check AND I could easily provide a steps for github issue by simply copy-pasting the test case steps into it.

norbertlaposa commented 1 day ago

Hi @MartinMiksovskyLaposa ,

I suggest the session detail screen to have 3 buttons:

If you made a service selection then it would change to:

The primary button would:

MartinMiksovskyLaposa commented 1 day ago

Sounds perfect to me, thanks @norbertlaposa

MartinMiksovskyLaposa commented 1 day ago

Have you also considered the possibility to access the manual test case from automated tests result @norbertlaposa ? Having the access just to display manual test case for failed automated tests is a great timesaver - especially in human-readable form (having the steps numbered & possibility to copy/paste these steps into an Github issue).

Idea: failed test detail could have some "show test case" button that simply loads & properly displays the test case in a pop-up/somehow (like in manual tests UI) so anyone is aware what should be done to re-test manually + the possibility to copy-paste steps to the issue.

Thanks!