snap-stanford / GEARS

GEARS is a geometric deep learning model that predicts outcomes of novel multi-gene perturbations
MIT License
189 stars 38 forks source link

Questions about reproduction of Figure2f and Figure2cd #58

Open yu3jun opened 5 months ago

yu3jun commented 5 months ago

Hi, Thank you for publishing such an excellent paper! I'm trying to reproduce the results of Figure2f and Figure2cd.

in https://github.com/yhr91/GEARS_misc/blob/main/paper/archive/fig2f.ipynb the pd.DataFrame(out) has 80 rows × 4 columns, all methods and category have different results about Top 20 DE MSE.

I wonder how they were computed, as I use the set as you advised in https://github.com/yhr91/GEARS_misc/blob/main/paper/reproduce_preprint_results.ipynb to could only get few close results of different combinations of methods and category

How could we get different results of same method and category(like Gears and 2/2 seen)? And if we use the mean No-perturb Top 20 DE MSE to compute others' Normalized MSE of Top 20 DE Genes? I would appreciate very much if you could share some of the parametes to help reproduct the results, thanks a lot!!!

image