search

egyptdj / stagin

STAGIN: Spatio-Temporal Attention Graph Isomorphism Network
https://arxiv.org/abs/2105.13495
GNU General Public License v3.0
85 stars 17 forks source link

ABIDE Performance #8

Closed tuttyfrutyee closed 2 years ago

tuttyfrutyee commented 2 years ago

Hi great work. I have tried your model on ABIDE dataset and could not get validation accuracy greater than 60% (chance level is around 55%), whereas with SVM I can get to 70% levels. I did optimize and play around with hyperparameters but with no luck.

To be honest, I would love to hear your speculations on this situation. Do you think the multi-site nature is the source of the problem or just "autism" is not much detectable on fMRI data? Thanks...

egyptdj commented 2 years ago

Hi @tuttyfrutyee

I think there are many reasons that GNN based models (including STAGIN) are not always providing satisfactory results on clinical samples, including the ones that you mentioned. But in this comment, I will specifically answer regarding the STAGIN model. The strength of STAGIN over other static FC approaches is its ability to exploit temporal information from the fMRI scan. However, in this case, the repetition time (TR) and the scan length can additionally affect the data distribution, which can significantly disturb the model training if these temporal characteristics are not equal across acquisition sites.

I appreciate the experiments that you are trying, and agree with you that extending the GNN models to clinical samples is a very important future direction. I think there should be some breakthrough on stabilizing the GNN training of multi-site (i.e. non-i.i.d.) fMRI data before these GNN-fMRI researches can actually be extended to the clinical field. I am actually working on the topic and would love to share any other ideas from researchers who are thinking in the same direction!

tuttyfrutyee commented 2 years ago

Great answer, thank you.