anbai106
commented
3 years ago Thank you for this report.
The discrepancy between the Matlab and python version is due to the standardization method, I..e, min-max or z score. In the original Matlab implementation, we used z score, which is more robust against outliers.
However, when I implemented pyHYDRA, I switched it to min-max standardization, which scales all features to similar ranges. I did this because I found that min-max method is faster for the binary classification pipeline. Please refer to this commit: @c6c7c35.
Can you please git pull locally to have the newest version, and rerun your Matlab versus pyHYDRA experiments? Note that Matlab by default with: k=-20 is to run 20-folds CV, which is really rare, because you will have much less test subjects. Anyways, HYDRA is not really doing CV there. The CV is to have data vaiarance across each fold. If you want to have a fair comparison, please also perform 20-folds CV for pyHYDRA. By default, pyHYDRA performs 20 repeated hold-out CV.
Thank you
IoannaSkampardoni
Hi, I implemented pyHYDRA to derive subgroups in a sample of cognitively impaired subjects. The input features file consists of residualized ROI volumes after performing age- and intracranial volume- adjustement. I set k_min=2, k_max=3 and cv_repetition=20. For both k=2 and k=3, I am always getting one subgroup with substantially smaller size compared to the other subgroups. In addition, the subgroups and the ARIs differ from the matlab-HYDRA subgroups and ARIs, respectively. I attach a plot showing the residualized hippocampal volume (residual=original-adjusted) for each subgroup (P1,P2,P3) vs the control group (CN) for matlab-HYDRA and pyHYDRA as well as the subgroups' sizes and ARIs. Are those differences expected? Thank you.