search

akvo / akvo-flow-mobile

Akvo Flow app
GNU General Public License v3.0
18 stars 16 forks source link

Answers not matching questions #1788

Closed janagombitova closed 3 years ago

janagombitova commented 3 years ago

I am really not sure how to name this issue, but will try to explain the problem as detailed as I can. The issue came from Carmen, who working on a dataset noticed the for 2 gender option questions, where the defined options in the form are Female and Male the dataset holds also values 1, 0, 2. These values come from form version 6. But there are submissions from form version 6 that have the correct expected. values. There is no linkage to the wrong values and an enumerator either.

The question was Why is this data different?

What I found and assumptions that I checked:

Could this be a case of options and option codes mixing up?

Answer: No

Why:

Could it be that the values were overwritten via Data cleaning?

Answer: No

Why: @tangrammer checked the events related to the form and found that data cleaning happened on 2021-01-29 (16:43:58.656) CET. He got the data files exported and imported to this survey form. There is a data export made on 2021-01-20 at 15:26. This one holds submissions to version 5 and version 6 but no data for the gender questions. The next export file is from 2021-01-29 at 11:34 and this sheet has the wrong values for the gender question.

So I asked @muloem if he could get me the json file sent from the Flow app for the submission with instance ID: 127350111 as this one has the wrong values in both columns in the excel file from the 2021-01-29 at 11:34. Looking at the values in the json file sent from the app, the submissions are made to form version 6 and answers to both questions show exactly the data in the sheet - wrong values.

Screenshot 2021-03-24 at 14 05 06

Screenshot 2021-03-24 at 14 05 52

This means that the Flow app sent wrong data to an option question, numbers, despite the fact that the from version should only allow the user to select either Male or Female.

Is it possible something went wrong in the app?

Answer: ?

The idea is that maybe the enumerator had a draft submission made to version 5, received version 6 and continued with the draft version. This means that the submission is made the version 6 of the form, but maybe the questions did not show well.

In an update on this issue, I will try to prove this assumption on uat1.

janagombitova commented 3 years ago

Is it possible something went wrong in the app?

Answer : No

I could not reproduce the error. The steps I took:

  1. Created a survey on uat1 > _ _Jana > Issue 1788 > Issue 1788
  2. Published it to form version 3
  3. Assigned the form to my device
  4. Submitted two data points and left one in darft/saved mode
  5. Made changes to the survey from and published version 4
  6. Closed and opened the app to trigger the download on the new version
  7. Opened a new submission to check the form version, it was version 4
  8. Reopened my draft submission, saw that the questions changed matching version 4, but in the header it stil lsaid version 3.
  9. Finished the submission also adding data to the new questions
  10. Submitted
  11. Downloaded the Data cleaning export and saw:
    • Data was correct - I could not reproduce the error
    • The submissions stated version 3 even thought it was actually version 4

Screenshot_2021-03-24-19-51-51

Screenshot 2021-03-24 at 19 54 57

Issues found:

Could this issue be caused by our unclear publishing UX?

Is it possible that the survey was published in version 6 but the user continued to make futher changes, before the publishing completed (did not go to the messages tab) and then published version 6 again? Is it possible to have 2 versions of a form of the same number?

We know the publishing of forms in Flow still has one risk:https://github.com/akvo/akvo-flow/issues/3619 Maybe that could be the reason for this issue.

? Could there be another reason behind this data issue?

valllllll2000 commented 3 years ago

Screenshot from 2021-04-05 17-20-17

I have managed to reproduce this issue, steps 1) Add 2 text questions to a form 2) Publish but don't close the form page nor reload the page 3) Make sure to download the form to the device 4) Fill the answer but do not press submit (leave the phone in the submit tab) 5) Update the answers by making them options 6) Publish 7) Submit on the device 8) In the app settings user "reload all surveys". Because the version is still the same, if you don't do this the new updated questions will not be downloaded unless you do it this way. 9) Open a new datapoint and submit again. This time you will see that the form has 2 option questions and not text 10) Export the data, you will see that for the same version there are answers for 2 different question types

What fixes can be done on the app: see suggestions in #1544 which are all related to updating the form version

The rest should be fixed in the backend, we need to find a way to make the user reload the page after publishing or disable the publish button until publishing is completed.

valllllll2000 commented 3 years ago

closing as there is nothing that can be done more on the app, some changes were released to fix the version inconsistencies