Clarify how to identify not asked vs not answered when no response is provided

[WValenti]

The following was updated to take into consideration whether or not a response that is NOT within the expected range for a question is able to be corrected in conjunction with the submitting investigators.

For a given interview and question:

1. There is a response provided in the dataset 1.a Flow of instrument and preceding response(s) indicate the question was asked 1.a.1 Response is within expected range for question - NORMAL 1.a.2 Response is NOT within expected range for question 1.a.2.a Response is replaced with one from the expected range in conjunction with submitting investigators - CORRECTED 1.a.2.b Response has not or cannot be replaced with one from the expected range - INVALID 1.b Flow of instrument and preceding response(s) indicate the question should NOT have been asked 1.b.1 Response is within expected range for question - NORMAL-BONUS 1.b.2 Response is NOT within expected range for question 1.b.2.a Response is replaced with one from the expected range in conjunction with submitting investigators - CORRECTED-BONUS 1.b.2.b Response has not or cannot be replaced with one from the expected range - NOISE
2. There is NO response provided in the dataset 2.a Flow of instrument and preceding response indicates the question was asked - MISSING 2.b Flow of instrument and preceding response indicates the question should NOT have been asked - NA/NA (both not asked and not answered)

[WValenti]

In all circumstances, the original value is stored in the RAW database kibble table. When a value is valid (or corrected) it is stored in the CLEANED database kibble table as follows:

• NORMAL - the original provided value is in "value", and 'N' in bonus_flag.
• CORRECTED - The corrected value is in "value", the original invalid value is in "replaced_raw_value", and 'N' in "bonus_flag".
• INVALID - no row is present in CLEANED kibble, the original invalid value is in RAW kibble. (Maybe a metavalue such as [INVALID] should be in CLEANED kibble for these?)
• NORMAL-BONUS - the same as NORMAL except a 'Y' in "bonus_flag".
• CORRECTED-BONUS - the same as CORRECTED except a 'Y' in "bonus_flag"
• NOISE - no row is present in CLEANED kibble, the original invalid value is in RAW kibble. (Maybe a metavalue such as [NOISE] should be in CLEANED kibble for these?)
• MISSING - currently not in kibble, but will be added as metavalue '[MISSING]' as we run Jo's process and update.
• NA/NA - not in kibble (would fill sparse tables and consume tremendous space). These are represented in downloads as metavalue '[NANA]' or '99999'.

As an aside, here are the metavalues currently in the database and their meanings:

• [UNK]: respondent indicated that they did not know the answer
• [TMTC]: respondent indicated too many to count
• [RF]: respondent refused to answer
• [PHI]: response has been removed as it permits identification of the respondent
• [NA]: interviewer indicated question was not applicable