search

TY1Fan / pe

0 stars 0 forks source link

DG Bug: Wrong manual testing steps #3

Open TY1Fan opened 1 week ago

TY1Fan commented 1 week ago

image.png

image.png

Steps to reproduce:

  1. Proceed to DG
  2. Proceed to the section on manual testing
  3. Observe the step on initial launch

Expected: According to the manual testing instructions, when I double click, i am suppose to see the gui.

Actual: I received an error on my end that i can't open this file. Refer to second screenshot.

Suggestion/ Why it is problematic: This instruction is misleading and there isn't other ways of testing the launching the app. Furthermore, this instruction seems to contradict the instruction on using the command java -jar ... set by the course. If I were to not use the command, I might not be able to continue testing the app since I am unable to test it. Hence, I think the severity is medium as it affects testability.

nus-pe-script commented 1 week ago

Team's Response

We believe that the user's current MacOS security and privacy settings prevents them from accessing some files.

Future iterations should provide additional guidance on how to allow access to opening the file in all operating systems.

Items for the Tester to Verify

:question: Issue response

Team chose [response.NotInScope]

Reason for disagreement: image.png

From the course website, documentation bugs can only be NotInScope if they are minor grammar mistakes that do not hinder the reader.

I do not think that the issue raised is a minor grammar error.

Furthermore, this greatly hinders the tester as the provided way to test the launching of the app is incorrect. To add on, there were no alternative methods of launching the app provided in the Manual Testing Steps and the tester had to refer to the User Guide to find out the right way to launch the app.

Hence, based on the hindrance that this documentation bug had caused, I feel that this issue should be Accepted instead of NotInScope.

## :question: Issue severity Team chose [`severity.Low`] Originally [`severity.Medium`] - [x] I disagree **Reason for disagreement:** > We believe that the user's current MacOS security and privacy settings prevents them from accessing some files. While this issue might be caused by OS settings, I feel that it is important for the dev team to verify information written in the DG and provide alternative solutions to an anticipated problem rather than let the user or reader find the solution themselves. Justification for `Medium`: 1. This bug greatly hindered my speed in testing the App as there is a misleading / wrong step documented. This warrants a `Medium` severity due to the time constrained nature of the PE. 1. Having a wrong step in Manual Testing Steps section, undermines the credibility of the dev team and their app built. This is especially the case since the first step of the manual testing section is already wrong, and would make the reader wonder if the dev team had done a thorough check on their DG before publishing or some may even wonder if the DG is outdated. 1. If I remember correctly, the course website seems to mention somewhere that the only way to launch the app should be through the terminal using the `java -jar ...` command. Hence this wrong instruction showed that the dev team were not following the requirements as specified by the course. 1. In real world context, such a mistake might be costly as professional tester hired may flag the app as unable to launch and be unwilling to continue testing the app. 1. However, I feel that the severity is not `High` as the correct step is still documented in their UG. Given the justification above, I would like to insist on my view that the severity should be `Medium` instead of `Low`.