nus-pe-bot
commented
2 years ago Team's Response
We think the existence of a flag that is not defined in the app does not mean that the error message is wrong, since it is not realistic to check for every single possible flag and point out any issues. For users who we assume are experienced with CLI tools, it should be obvious to them that a wrong flag was used from the error message, and they should be able to rectify it themselves without too much hassle.
Items for the Tester to Verify
:question: Issue response
Team chose [response.NotInScope]
- [x] I disagree
Reason for disagreement: This is a bug from the original AB3, and our group has also accepted it when pointed out in our own product for the PE. Their argument that "it is not realistic to check for every single possible flag" is unfortunately wrong; as all you need to do is just check that the flags present, are correct flags. This is of course, finite, and very possible.
My group is confident that this is a problem that is easily fixed, and I would like to explain at a high level how it would be done here to justify my objection. In a command, when the regular expression ( [a-z0-9]+/ ) is seen, (i.e. a space, followed by at least one alphanumeric character, ending with a backslash, followed by another space), this can be identified concretely as a "prefix". All that is required is to add an additional validation check, by searching in the users input for all substrings that match this pattern, and ensure that they match the prefix-es that are expected for a given command word.
A note would have to be added in the UG to warn the user against arbitrarily using backslashes, but actually checking for this is not at all difficult.
Now, the reason we do consider this a bug, is because we ourselves ran into this issue often when using our own application (though we unfortunately did not think to rectify it, as we wrongly assumed AB3 bugs did not count and prioritised other tasks). It was very common to mis-type a prefix, or to mis-remember the correct prefix, for example here by typing nv/ for "next visit" instead of v/ for "visit".
This is a very small typo that can happen consistently in actual use, and the confusing error message we believe would inconvenience a significant number of users, especially new ones. Hence we felt it was a valid bug when we accepted it in our PE. An error message that tells the user explicitly that a prefix they have given is unexpected, would be much less confusing, compared to telling them the date is in the wrong format, given that the date shown in the example above matches the format perfectly.
:question: Issue severity
Team chose [severity.Low]
Originally [severity.Medium]
- [x] I disagree
Reason for disagreement: I have given an extensive justification above for why I believe this is a valid bug. The description of medium is: "A flaw that causes occasional inconvenience to some users but they can continue to use the product.". This type of typo was very common in our testing of our own product, and I even encountered it during the PE by erroneously assuming the prefix would be "nv/" for next visit, instead of "v/" for visit. Thus, I believe that this would qualify for occasionally inconveniencing a significant number of users, given that this type of typo is very easy to commit when typing at a fast speed.
The application appears to recognise the wrong error here. I gave the wrong label for next visit (nv/) but the correct date, and receive a warning about the format of the date, instead of a notice that I have given the wrong command.