ailanthustng / pe

0 stars 0 forks source link

[DG] Instructions for manual testing are too few for actual testing. #23

Open ailanthustng opened 4 years ago

ailanthustng commented 4 years ago

There are only 9 manual tests for a somewhat complicated app.

nus-pe-bot commented 4 years ago

Team's Response

The manual test instructions are non-exhaustive. If we made it exhaustive, it would be extremely verbose and our DG wouldn't fit into the maximum size limit. Instead, a better idea is to have a few representative manual test instructions so that the tester can learn the workflow. From there, the tester can follow the UG, DG and signposts within the application to continue testing. According to the severity guidelines, we have marked this issue as "Very Low" severity, because this is a minor contention for the DG that does not affect user's usage at all. As having exhaustive list of manual test instructions is not in the scope of the DG, we have marked this issue as NotInScope.

Items for the Tester to Verify

:question: Issue response

Team chose [response.NotInScope]

Reason for disagreement: Although I do understand that we are required to deviate from the provided testing instructions to find more bugs for the app, the testing instructions of a DG should provide some guidance and basis for a tester (me) to begin with testing. However, the lack of certain instructions had caused me to be unable to test the app immediately and required me to do some sort of playing around with the app.

Moreover, from the screenshot below, I believe that we were told to "cover all user-testable features", but this was not done in the DG, and hence I very much believe that it's part of the scope of DG.

Screenshot 2020-04-22 at 00.54.08.png


:question: Issue severity

Team chose [severity.VeryLow] Originally [severity.Low]

Reason for disagreement: I feel that it's not merely a cosmetic error. Furthermore it does cause some inconvenience as a tester as I had to play around with the app for a bit before I could test the commands which they left out.