soc-se-bot
commented
9 months ago Team's Response
We quote the reply from Prof. Damith:
"It was not meant to be comprehensive test script for testers to follow, just something to get them started. Note that dev docs normally aim for minimal yet sufficient, not comprehensive. So, some details may be omitted if you expect the omission will not mislead the tester or put them at a disadvantage. "
[https://github.com/nus-cs2103-AY2324S1/forum/issues/494]
We believe that the missing prerequisites would not mislead the tester or even put them at a disadvantage too as everything is pretty intuitive. Some extra justifications:
- The purpose of the manual testing section is to get them started, and we believe that no testers would start off by testing a product with zero entries (zero customers in our case). If someone wants to test the feature on customers, it is likely that they he/she knows that there should be existing customers in the list.
- The way we phrase our sentences in the descriptions of the expected outputs are clear enough to let the readers know that we are assuming existing customers in the list. Even in the case where the list is empty, the output error message in the product is descriptive enough to tell the tester what went wrong (we provide guidance to the tester).
Using the example you gave above, We think that it is clear enough when we say "customer at index 1" since the reader would automatically think that there should exist a customer at index 1. So even if the list is empty, we believe that the reader would have the common sense to add a customer into the list before performing the test.
As a summary, there are two main points: a.) It is clear enough even if we do not include the prerequisite. b.) We even guide the testers to do the right thing (via error messages) when something undesirable happens, in which is unlikely to happen due to the 2 justifications given above. Also, as what the prof mentioned, it is okay to omit these details in this section as long as it does not cause any trouble. We think that the missing prerequisites do not hinder the reader or by any means cause confusion to them. Hence the bug report is rejected.
For severity level, we also disagree that it is medium, it is low in our opinion, the reasons are the ones stated above. To reinforce again, these details may be omitted since removing them doesn't cause confusion to the reader and it is made clear to the reader that we are assuming customers to exist in the list.
Items for the Tester to Verify
:question: Issue response
Team chose [response.Rejected]
- [x] I disagree
Reason for disagreement: Firstly, I appreciate your detailed explanation and the reference to Prof. Damith's perspective on keeping the documentation "minimal yet sufficient." However, I'd like to respectfully offer a different viewpoint regarding the absence of prerequisites in certain test cases.
Quoting the principle of "minimal yet sufficient," I believe that not including prerequisites for some test cases might unintentionally put testers at a disadvantage. This is especially true in scenarios where the test outcomes heavily depend on the existing data within the system. For instance, the example I initially mentioned, though minor, illustrates this point. In a more complex example, such as updating the tags of a specific customer, the outcome varies based on whether the tags already exist. This complexity further underscores the need for clear prerequisites to ensure testers understand the starting conditions for each test case. And I think the same can be said for most, if not all, of the sections under Manual Testing.
Regarding your second point about the clarity of sentence phrasing in the descriptions, I agree that your team has done an excellent job in articulating the expected outcomes. However, if the prerequisite field is deemed redundant due to the clarity of descriptions, it might be more consistent to omit this field entirely. This would align with your argument that the documentation guides testers adequately through error messages and clear descriptions.
Including prerequisites for each part of the manual testing section could provide a valuable reference point, ensuring that all test cases are executed as expected and align with the indicated outcomes. This addition would help testers, especially those new to the product, to quickly grasp the necessary starting conditions for each test scenario. In conclusion, while I agree with your lowering the severity level from medium to low, I respectfully disagree that the absence of prerequisites doesn't cause any confusion. If prerequisites truly have negligible impact, then fully removing the prerequisite section might be a clearer approach. Otherwise, their inclusion can help prevent any ambiguity, particularly in scenarios where different existing data in the system can lead to varying outcomes. Thank you for considering my perspective on this matter. I believe these minor adjustments could enhance the clarity and usability of our testing documentation.
## :question: Issue severity Team chose [`severity.Low`] Originally [`severity.Medium`] - [ ] I disagree **Reason for disagreement:** [replace this with your explanation]
Prerequisites: - were mostly blank for a lot of the manual testing, even though it would help to include some
for example
a prerequisite could be that first customer exists