chrisgorgo
commented
7 years ago Thank you for this beautifully put together analysis. A few comments:
- Lack of computable threshold is not necessarily a sign of "systematic error" or "overfitting". All it means is that noise and signal distributions overlap so much it is impossible to come up with a single cutoff value that would allow to distinguish between them. This could be a true depiction of the underlying data.
- Significant activation or deactivation in the ventricles is also not necessarily an error. Some models and contrasts can yield such statistical maps. See for example this map from Human Connectome Project: http://neurovault.org/images/3129/
I do, however, admit that the model depicted in Figure 1d (the three component) does look unrealistic. The negative Gamma is too close to zero. One way to improve the procedure is to implement a version of it with priors on location of Gammas that would prefer solutions with Gammas in the tails of the Gaussian.
timcoolen
Dear Dr. Gorgolewski,
I'm carrying a study to compare the clinical usefulness of 4 visually presented covert language paradigms in 46 healthy adults. The adaptive thresholding method described in your paper in Frontiers in Human Neuroscience in 2012 appeared to me as an ideal objective comparison starting point and I have used the script you kindly made available on GitHub. However, I have encountered systematic errors, owing to what I believe is an overfitting of the Gaussian and Gamma curves. I have sent you a MS Word file for illustration purposes, which for some reason I cannot upload in any format here. In 20% of cases, the algorithm favored a model 3 over a model 1 distribution despite the lack of a computable threshold (figure 1b). In 5% of cases, it did calculate a positive threshold but which resulted only in spurious (e.g. intraventricular) activations (figure 1c). In 25% of cases, it favored a model 3 distribution over a model 2 distribution, although no negative threshold was computable (figure 1d). Except in one case (figure 1e), it did not substantially change the Gaussian-Positive Gamma intercept, but it is certainly a source of inaccuracy.
Have you experienced similar issues in your datasets and do you know how I can correct for these errors?
Thanks a lot in advance for your invaluable input.
Best regards,
Tim Coolen Resident in Radiology Erasme Hospital Université Libre de Bruxelles Belgium