search

jedkohjk / pe

0 stars 0 forks source link

Null values not dealt with in data file #1

Open jedkohjk opened 11 months ago

jedkohjk commented 11 months ago

Steps to reproduce

Paste the following into the data file

{
  "persons" : [ null ],
  "events" : [ ]
}

Then run the app

Expected

Based on this:

image.png

The file should be cleared and the app should start with empty data.

Actual

image.png

nus-se-bot commented 11 months ago

Team's Response

It is stated in the user guide that only advanced users should be doing this. It also warned that invalid formats will result in an empty FumbleLog. In this issue, the tester has keyed in null value into addressbook.json. In the developer guide, we have stated that we are handling null values by not accepting null values. All null values or "empty boxes" has their own NULL_INSTANCE.

It is also not reasonable to state this detail in the user guide as it is a developer detail, hence it is not stated. Additionally, users who add invalid formats into the json file will be naturally given an empty FumbleLog.

This is not a bug. Please refer to the screenshots below with the highlighted mistake.

Screenshot 2023-11-20 at 01.46.44.png

Items for the Tester to Verify

:question: Issue response

Team chose [response.Rejected]

Reason for disagreement: In the dev team's response, they wrote:

It is stated in the user guide that only advanced users should be doing this. It also warned that invalid formats will result in an empty FumbleLog.

However, when the json file provided I provided is input, the app does not even start.

The dev team seems to be refuting a documentation bug. However, I reported this as a functionality bug because the app does not fail gracefully. It does not start with an invalid json file.

This issue could realistically arise. For example, users may want to clear the persons list without clearing the events list or vice versa. Such functionality is not provided by the app (which only has a general clear command which clears both lists). Without any instructions on how to change the datafile, they may mistakenly modify the data file to look something like this, thinking that 'null' means the list is empty:

{
  "persons" : [ null ],
  "events" : [ {
  ...
  some event data
  ...
  } ]
}

The next time they use the app (which may not be immediately after they modify the data file), the app does not start. It is not obvious that the issue arose due to an invalid data file. They only expect changing the data file to clear all the app data at worst, not to cause the app to be unable to start. Hence, they cannot figure out why the app does not start.

The dev team also cites the developer guide in their response, stating that if one were to read the developer guide, it would be clear that we should not enter null values in the data file. However, I was trying to do black-box end-to-end testing for the practical exam, without referring to the developer guide. I think that we should not assume that users read the developer guide either.

In summary, I think the bug stands because:

  1. It can reasonably affect real users who have read the user guide in full
  2. The dev team did not address the bug I raised - I raised a functionality bug but they addressed a documentation bug

Prof Damith seems to agree with my assessment of this as a bug in the course forum:

Screenshot (326).png