Fixing GESD algorithm fail on low variance

[beansrowning]

History

As seen in @TsvetaKoleva's issue (#12), the GESD algorithm implementation will fail on low-variance data.

This was particularly the case in zero-inflated count data, which was not flagged as "low-variance" or "near zero variance" by caret's nearZeroVar() or similar

What was done:

• Added small amount to denominator of z-score calculation to avoid division-by-zero

previous

z <- abs(x_new - median(x_new))/mad(x_new) # Z-scores

new

z <- abs(x_new - median(x_new)) / (mad(x_new) + .Machine$double.eps) # Z-scores

• Added unit test to check compliance on raw count, decomposed values
• Ran styler::style_file() on R script for tidying

[mdancho84]

Thanks Sean. Just merged!

[Goutamravi]

Sean,

I am using this anomalize package and still having the below error when i use gesd method. however for the same data if i use iqr i do not get the below error. Any thoughts.

Error in if (any(vals_tbl$outlier == "No")) { : missing value where TRUE/FALSE needed

[beansrowning]

@Goutamravi Do you have a sample of data to reproduce? I might not have immediate capacity to look into it, but that might help elucidate what's going on.

[Goutamravi]

@beansrowning Thank you for your response. Unfortunately i am running through loops and i cannot share the entire dataset. The values are smaller. It is everyday sales data. I have values ranging from 0-1000.

Error in if (any(vals_tbl$outlier == "No")) { : missing value where TRUE/FALSE needed 11. anomalize::gesd(x = x, alpha = alpha, max_anoms = max_anoms, verbose = TRUE) 10. anomalize.tbl_df(., remainder, alpha = 0.05, method = "gesd") 9. anomalize(., remainder, alpha = 0.05, method = "gesd") 8. function_list[k] 7. withVisible(function_list[k]) 6. freduce(value, _function_list) 5. _fseq(_lhs) 4. eval(quote(_fseq(_lhs)), env, env) 3. eval(quote(_fseq(_lhs)), env, env) 2. withVisible(eval(quote(_fseq(_lhs)), env, env)) 1. model_pre_proc %>% time_decompose(Volume_Agg, method = "stl") %>% anomalize(remainder, alpha = 0.05, method = "gesd")

[Goutamravi]

@beansrowning I found the issue. The model was executing for 0 values. I removed Dimensional filed which have all zero values and it is executing fine. Thanks 👍