worldbank / iefieldkit

Stata commands designed for Impact Evaluations field work. These are tools that are used during/after a survey in the field for data quality monitoring.
MIT License
38 stars 18 forks source link

[iecorrect]: Error when [valuecurrent] is missing or wildcard when ID variable is wildcard #226

Closed bbdaniels closed 1 year ago

bbdaniels commented 2 years ago

It is unclear whether the valuecurrent column is intended to allow wildcards (I think it should). In any case, the following two setups fail.

sysuse auto.dta , clear
iecorrect apply using "/users/bbdaniels/desktop/correct.xlsx" , idvar(make) noi debug

If the sheet is:

make foreign varname value valuecurrent initials notes
*   headroom 5   BD OK to go!
Test   headroom 3 3    

I would expect an error specifying no blank cells allowed in valuecurrent. Instead, an inaccurate error is returned from _fillidorvalue:

There are 1 lines in sheet numeric where neither the ID variable values or the valuecurrent column were specified. At least one of these columns should be filled for  corrections to be made correctly.
invalid syntax
r(198);

If the sheet is:

make foreign varname value valuecurrent initials notes
*   headroom 5 * BD OK to go!
Test   headroom 3 3    

I would expect no error, with all values of headroom set to 5. Instead, an error is returned from _fillcurrent:

Column valuecurrent in sheet numeric is not of type numeric. This column should contain the values of the numeric variables to be corrected.
invalid syntax
r(198);

Corresponding correction is required in the help file at:

https://github.com/worldbank/iefieldkit/blob/682d7d5d75ad5036ec5f2c9e6d95a173a31c0465/src/help_files/iecorrect.sthlp#L126-L129

luizaandrade commented 2 years ago

I like the suggestion of always filling valuecurrent. It seems wrong to me to make changes to all observations, which is why I was not allowing that. But I guess this is indeed the expected behavior.

bbdaniels commented 2 years ago

@luizaandrade, this fix still requires revisions to the help file indicating the final functionality -- I have marked this unresolved till that is done.

luizaandrade commented 1 year ago

I have no idea what I was thinking. It makes no sense to allow changes in all observations. As of now, if you don't insert anything, it says to add a wildcard if you want any value to be replaced. And if you add wildcards on both, it returns the following error:

There are X lines in sheet numberic/string where neither the ID variable values or the valuecurrent column were specified. At least one of these columns should be filled with valid information (other than a wildcard) for corrections to be made correctly.