search

MattJBritton / ExquisiteCorrpse

Tool for visualizing feature correlations and interactions in Jupyter notebooks.
GNU General Public License v3.0
3 stars 0 forks source link

Trying to replicate your FSDs in R... #3

Open olyerickson opened 1 year ago

olyerickson commented 1 year ago

I've been trying to replicate your (awesome!) feature space diagrams in R; see my hacking at: https://github.com/TheRensselaerIDEA/FeatureSpaceDiagrams

I'm especially trying to reproduce your FSD for the Epileptic Seizure dataset; my result is similar (see below) but not exact; in particular I'm not seeing some of the "loops" that you did. As a side note, your notebook seems to show that you used Pearson correlation in all cases, but I think you would have had to have used Spearman for the Epilepsy data.

Here's an example of what I've been able to reproduce in R: image

olyerickson commented 1 year ago

I see similar results using "Euclidean" distance (the previous example was "Manhattan")... image

olyerickson commented 1 year ago

This is an example using your abs(A)-1 distance metric... Still no loops! image

MattJBritton commented 1 year ago

John,

First of all, thanks so much for expressing interest in my little project!

Unfortunately, I am leaving tomorrow for a 2-week vacation on which I will not have computer access, so it may take me a while to investigate some of these issues in detail.

At first glance though, I’m not sure why the Epilepsy data would require me to use Spearman instead of Pearson. This might be my ignorance, but my understanding is that Pearson is appropriate for continuous variables, and that the main reason to use Spearman instead is if you have categorical/ordinal features. The target for the Epilepsy data is ordinal, but that is not included in the feature space diagram here (I forget why, probably for exactly this reason). All the 178 other features are integers.

Not sure that’s a cause of our discrepancies, but I’m happy to look into all the potential causes when I return at the end of December.

Happy Holidays!

Matt