Closed Je-gh closed 1 year ago
Wow @Je-gh, so sorry for the 6 month delay in this response. This 20 controls logic simply stems from logic underpinning a p-value (i.e. if re-sampled the data again and again, we'd only see this result 1 time out of 20, or 5% of the time). The less units you have, the harder it is to make that claim. I can't remember the exact adjustment, did you ever figure out the correct approach?
Dear @edunford, I'm sorry if this question is trivial, but I have struggled to find an answer elsewhere. In your R documentation, when talking about grab_significance() you mention: One needs at least 20 control case to use the conventional .05 level. With fewer cases, significance levels need to be adjusted to accommodate the low total rank. This is a bug of rank based significance metrics.
Could you please point out to a resource that explains how this adjustment is done (to e.g., 9 instead of 20 units)?
Thank you in advance!