regarding MUAC z-score classification

[ernestguevarra]

In your function documentation, you say:

#'This function is designed to flag MFAZ and crude MUAC-based values if the fall
#'  outside outside ±3 from the observed survey sample mean and a fixed range of
#'  <100mm and >200mm for MFAZ and crude MUAC respectively. This is based on the
#'  research findings by [Bilukha, O., & Kianian, B. (2023)
#'  .](https://doi.org/10.1111/mcn.13478)

I haven't read the reference but based on what I know of SMART ENA approach to flagging the other z-scores, the flagging is not the same as the WHO flags. What you describe above is how SMART flags but the way you implemented the function performs the flags based on WHO approach.

You can check the {nipnTK} package as Mark implements here the SMART approach to flagging based on the survey sample mean.

https://nutriverse.io/nipnTK/articles/flagging.html#applying-smart-flagging-criteria-to-survey-data

and the function for SMART approach to flagging:

https://github.com/nutriverse/nipnTK/blob/main/R/national.SMART.R

If you want to keep your approach in the function, I think the description should change and say above + 3 zscores and below -3 zscores are flagged rather than saying from the survey sample mean.

[ernestguevarra]

@tomaszaba, I read through the reference you used here from Bilukha and Kianian and it confirms that you are correct in that the flags are from the survey sample mean and is not the same as the WHO approach of above and below +/- 3 zscores. It says:

We explored characteristics of flags as well as changes in survey-level MUAC-for-age z-score (MUACZ) and MUAC means, SD and percentage of flags based on three flagging approaches: ±3 and ±4 MUACZ z-scores from observed MUACZ survey mean and a fixed interval 100–200 mm of MUAC

To describe characteristics of flags excluded by MUACZ, we explored two flag exclusion ranges: z-score outside of ±4 z-scores from the observed survey sample mean, as described by the World Health Organization (1995) and z-score outside of ±3 z-scores from the observed survey sample mean used in the SMART standard data quality evaluation procedure (so called ‘plausibility check’) (SMART, 2012).

I further researched on the SMART methodology specifically the SMART plausibility check and found this document:

https://smartmethodology.org/survey-planning-tools/smart-methodology/plausibility-check/

which in page 8 states:

This test calculates the proportion (%) of flagged data present in the survey data. Flags refer to outliers, extreme values that are so far from the mean that that they are unlikely to be correct measurements. The errors are always identified based on SMART exclusion cut-offs which are more stringent than other ways of flagging erroneous data; however, on average about one in 1,000 measurements excluded may be removed incorrectly (this will have a negligible effect upon the results – there is no penalty until there are 25 flagged data per 1000 measurements). The WHO exclusion cut-offs are based upon data points that are so extreme as to be biologically impossible (see Options tab in ENA software); less extreme errors are not excluded using WHO flags. It should be noted that with perfect quality of measurements the results with no flags, WHO flags and SMART flags should all be identical or almost identical. The magnitude of the difference between the three is an indication of the quality of the data collected. SMART flags (cut-offs) are based on statistical plausibility, and exclude all values outside of ± 3 Z-scores from the mean of the surveyed population. If measurements from a surveyed population form a normal distribution (bell-shaped curve) with a SD equal to 1, based on statistical principles 99.7% of observations should lie within ±3 Z-scores from the mean. The upper and lower cut-off points for SMART flags will be different in each survey since the survey mean is used as the reference point for SMART flags (surveyed children are compared to their own population when using SMART flags).

Clearly the approach to flagging using the SMART approach is different from the WHO approach as is stated in the above quote.

So, based on this, I think your description of the function is correct but your implementation of the function for this flagging approach is incorrect as you are using the WHO Growth Standards approach (post 1995) approach to the flags rather than the SMART approach to the flags which uses the survey sample mean.

Do you agree that this is the case? or am I missing something?

The best way to empirically verify this is that in your testing suite for this function, you should use one of your datasets for SMART (I would recommend adding a dataset in your package for this purpose) and then include in this dataset a column that provides the flags based on SMART ENA output. And then using this dataset, create a test for your function and check whether your function provides the same output as the output of SMART ENA.

Does this make sense?

[ernestguevarra]

If you agree that the flagging should be consistent with the recommended SMART approach, then I think you don't need a separate function for flagging MUAC-for-age z-score. You can just use the built in function in {nipnTK} for applying SMART flags to z-score data.

[ernestguevarra]

This would also mean that you should have a separate function for z-score flagging and for raw MUAC flagging (based on 100mm to 200mm range).

[tomaszaba]

I agree with this. After I saw this issue you raised today, I reworked on the function, including re-factoring it to be reusable for either zscore or crude muac based checks. This is how I implemented:

flag_outliers <- function(x, method = c("zscore", "crude")) {
  if (method == "zscore") {
    mean_zscore <- mean(x, na.rm = TRUE)
    flags <- ifelse((x < (mean_zscore - 3) | x > (mean_zscore + 3)), 1, 0)
    flags <- ifelse(is.na(x), NA, flags)
    flags
  } else if (method == "crude") {
    flags <- ifelse(x < 100 | x > 200, 1, 0)
    flags <- ifelse(is.na(x), NA, flags)
    flags
  } else{
    message(
      "This method is not applicable. Please choose between 'zscore' and 'crude'"
    )
  }
}

What do you think about this?

[ernestguevarra]

I think there is a fundamental shift in the approach to the function given how SMART does its flagging. By default, the SMART flagging approach cannot be applied to a single zscore value. It requires zscore values from a whole survey to be able to get a survey sample mean zscore. Do you agree?

If you do, then the structure of your function has to change and should by default only accept a survey data.frame for which flagging will be applied.

Compared to the raw/crude MUAC (I think we shouldn't use raw or crude to refer to MUAC - we should just refer to them as muac and the other value as zscores) where the flagging criteria are absolute values that can be applied to a single muac value or a vector of muac values.

[tomaszaba]

I think there is a fundamental shift in the approach to the function given how SMART does its flagging. By default, the SMART flagging approach cannot be applied to a single zscore value. It requires zscore values from a whole survey to be able to get a survey sample mean zscore. Do you agree?

If you do, then the structure of your function has to change and should by default only accept a survey data.frame for which flagging will be applied.

Compared to the raw/crude MUAC (I think we shouldn't use raw or crude to refer to MUAC - we should just refer to them as muac and the other value as zscores) where the flagging criteria are absolute values that can be applied to a single muac value or a vector of muac values.

Hi @ernestguevarra , I just realized that I hadn't commented on this yet.

Let's say you have an already calculated zscore, and then want to identify outliers based on SMART. SMART says outliers is equal to any value that is less than or greater than -/+3 from the survey population mean. With this, I understand that all I need to know is the sample population mean first, and that I can achieve with a numeric vector containing zscores, without necessarily being a data frame. Then I do the mean - 3 = flag or mean +3 = flag.

In the approach that I suggested earlier above, I use my function inside a chunk of code that works on a data frame and adds a flag variable to the data frame.

Do you think this solves the issue? If not, could you please integrate your approach in when revising the PR?

As for not using the word crude, I agree. I have taken note now to update in the next commit as in the recently submitted PR I have not taken this into consideration.