Comprehensive numerical simulations

cshen6 commented 8 years ago

[x] let us compare with more benchmarks, and add dcov and hhg using existing R packages.
[x] show FH test power at best k, MGC power at best scale and best t, plus dcov and hhg power at best t.
[x] for given model, plot the power wrt increasing sample size, for example from 25 to 300 at intervals of 25. The range should be meaningful, i.e., if all powers become 1 after 100, add some noise to your model or reduce the range.
[x] Do above for all models you would like to include in paper, just one parameter setting for each model.
[x] use simulations to illustrate we are similar to FH test under their model, but can be better out of their model and in real data.
[x] Find out what ADJ1 and ADJ2 means in your old simulation. Maybe for adj1 you directly use adjacency for distance, and for adj2 you take Euclidean distance of adjacency matrix? I strongly suggest that for all the above simulations / real data, include MGC on adj2 for benchmark too. (may not for inclusion in paper, but we really want to see the performance of all benchmarks to further our understanding! )

@jovo above is what I think for the numerical section, where we shall systematically illustrate our advantage over Fosick-H test for network dependency. What do you think? Anything to add?

youjin1207 commented 8 years ago

@jove Thanks Cencheng. Actually we might need one or two real data example. With C.elegans data, both Fosdick/MNT pvalues are so low :(

jovo commented 8 years ago

seems like a good start to me!

for dMRI data, we could test independence of topology with vertex 3d location?

youjin1207 commented 8 years ago

@jovo you mean you have relational dMRI data? Due to the context I assume in a paper, undirected and unweighted network (graph) would be most preferable.

jovo commented 8 years ago

i don't know what "relational dMRI data" means. i have undirected graphs. we can unweight them if you want.

On Fri, Aug 26, 2016 at 9:37 AM, Youjin Lee notifications@github.com wrote:

@jovo https://github.com/jovo you mean you have relational dMRI data? Due to the context I assume in a paper, undirected and unweighted network (graph) would be most preferable.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/neurodata/Multiscale-Network-Test/issues/2#issuecomment-242737398, or mute the thread https://github.com/notifications/unsubscribe-auth/AACjctD8PAgdF_Ix_Y2J8DZ5VBGlZsk1ks5qjuwFgaJpZM4Jton8 .

the glass is all full: half water, half air. neurodata.io, jovo calendar https://calendar.google.com/calendar/embed?src=joshuav%40gmail.com&ctz=America/New_York

cshen6 commented 8 years ago

for theoretical purpose and simulations, yes we should use undirected / unweighted graph;

but in real data we do not have to limit us to the assumptions :-)

On Fri, Aug 26, 2016 at 9:37 AM, Youjin Lee notifications@github.com wrote:

@jovo https://github.com/jovo you mean you have relational dMRI data? Due to the context I assume in a paper, undirected and unweighted network (graph) would be most preferable.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/neurodata/Multiscale-Network-Test/issues/2#issuecomment-242737398, or mute the thread https://github.com/notifications/unsubscribe-auth/ALX0y4p9EPskchaIHbf__F2eSWAPV2ttks5qjuwFgaJpZM4Jton8 .

youjin1207 commented 8 years ago

@jovo great! Can you please send me a link to such dMRI with relevant attributes?

jovo commented 8 years ago

@gkiar can tell you the best one to start with.

On Fri, Aug 26, 2016 at 12:08 PM, Youjin Lee notifications@github.com wrote:

@jovo https://github.com/jovo great! Can you please send me a link to such dMRI with relevant attributes?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/neurodata/Multiscale-Network-Test/issues/2#issuecomment-242778256, or mute the thread https://github.com/notifications/unsubscribe-auth/AACjcslbPKpD_0t71aIrpqLdl9wQxJ5yks5qjw9jgaJpZM4Jton8 .

the glass is all full: half water, half air. neurodata.io, jovo calendar https://calendar.google.com/calendar/embed?src=joshuav%40gmail.com&ctz=America/New_York

gkiar commented 8 years ago

We're generally recommending the following voxelwise graphs: http://openconnecto.me/data/public/MR/m2g_v1_1_1/KKI2009/derivatives/bg/

Each node has an attribute called "spatial_id" which is a Morton Z-order representation of their physical coordinate in space. Depending on what language you're using, we've written a couple tools to help you turn these into (X,Y,Z) triples. These scripts can be found here (Python), and here (R).

youjin1207 commented 8 years ago

@gkiar Thanks, I will take a look at it.

jovo commented 8 years ago

thanks!

On Mon, Aug 29, 2016 at 10:03 AM, Greg Kiar notifications@github.com wrote:

We're generally recommending the following voxelwise graphs: http://openconnecto.me/data/public/MR/m2g_v1_1_1/KKI2009/derivatives/bg/

Each node has an attribute called "spatial_id" which is a Morton Z-order representation of their physical coordinate in space. Depending on what language you're using, we've written a couple tools to help you turn these into (X,Y,Z) triples. These scripts can be found here (Python https://github.com/neurodata/ndgrutedb/blob/master/MR-OCP/mrcap/zindex.pyx#L86), and here (R https://github.com/neurodata/ndmg/blob/master/analysis/MortonXYZ.R).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/neurodata/Multiscale-Network-Test/issues/2#issuecomment-243132964, or mute the thread https://github.com/notifications/unsubscribe-auth/AACjcmwB3izgzjL-SubGx_-MNBftS3v-ks5qkuangaJpZM4Jton8 .

the glass is all full: half water, half air. neurodata.io, jovo calendar https://calendar.google.com/calendar/embed?src=joshuav%40gmail.com&ctz=America/New_York

youjin1207 commented 8 years ago

@jove @gkiar Triple (x,y,z) location is way too much correlated with network topology! No matter what scale and what statistics I use we get the least p-value.

jovo commented 8 years ago

that's great!

On Tuesday, August 30, 2016, Youjin Lee notifications@github.com wrote:

@jove https://github.com/jove @gkiar https://github.com/gkiar Triple (x,y,z) location is way too much correlated with network topology! No matter what scale and what statistics I use we get the least p-value.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/neurodata/Multiscale-Network-Test/issues/2#issuecomment-243529364, or mute the thread https://github.com/notifications/unsubscribe-auth/AACjcrmLmAYlkWAg1C0LDa5IEke54472ks5qlHJ2gaJpZM4Jton8 .

the glass is all full: half water, half air. neurodata.io, jovo calendar https://calendar.google.com/calendar/embed?src=joshuav%40gmail.com&ctz=America/New_York

gkiar commented 8 years ago

Very cool!

cshen6 commented 8 years ago

Youjin,

that is great for jovo and greg, probably because so their tools did the right thing :-)

I have quite a number of real networks plus labels, that I used to try classifications on. I can send them for you to try different tests.

On Tue, Aug 30, 2016 at 4:52 PM, Greg Kiar notifications@github.com wrote:

Very cool!

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/neurodata/Multiscale-Network-Test/issues/2#issuecomment-243577047, or mute the thread https://github.com/notifications/unsubscribe-auth/ALX0y8WHc97YZPta__q0x-IpYJ57hqUsks5qlJgmgaJpZM4Jton8 .

cshen6 commented 8 years ago

Youjin,

I tried all graph data I have, by testing between: Euclidean distance of the adjacency matrix vs distance of the labels (+1 to off-diagonal distances).

MGC equals the global correlation for all of them, which yields very significant p-values. So it may be too easy again...But I didn't try FH test, so maybe it worth a try.

Attached graph data are all in matlab format, but you can use the matlab.R package to get the data, or just copy paste the adjacency and the label from Matlab.

Best, Cencheng

On Tue, Aug 30, 2016 at 5:01 PM, Cencheng Shen cshen6@jhu.edu wrote:

Youjin,

that is great for jovo and greg, probably because so their tools did the right thing :-)

I have quite a number of real networks plus labels, that I used to try classifications on. I can send them for you to try different tests.

On Tue, Aug 30, 2016 at 4:52 PM, Greg Kiar notifications@github.com wrote:

Very cool!

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/neurodata/Multiscale-Network-Test/issues/2#issuecomment-243577047, or mute the thread https://github.com/notifications/unsubscribe-auth/ALX0y8WHc97YZPta__q0x-IpYJ57hqUsks5qlJgmgaJpZM4Jton8 .

cshen6 commented 8 years ago

probably all done except last adjacency one

neurodata / Multiscale-Network-Test

Comprehensive numerical simulations #2