SAM doesn't return right DAG

FenTechSolutions / CausalDiscoveryToolbox

Package for causal inference in graphs and in the pairwise settings. Tools for graph structure recovery and dependencies are included.

https://fentechsolutions.github.io/CausalDiscoveryToolbox/html/index.html

MIT License

1.08k stars 198 forks source link

SAM doesn't return right DAG #65

Closed arthurshatver closed 4 years ago

arthurshatver commented 4 years ago

I just run example in documentation https://fentechsolutions.github.io/CausalDiscoveryToolbox/html/causality.html#sam . I got complete graph, where every node is connected with all others nodes. is it something i don't do right.

diviyank commented 4 years ago

Hello Arthur,

Actually the hyperparameters (mostly lambda1 and lambda2) have to be a little tuned depending on the size of the graph. We thought of having it automatically set depending on the size, but finally chose of having the default fixed. Could you try with : lambda1 = .1 and lambda2=1e-04?

This unittest was to show how the SAM works, I guess I should update it with the good parameters.

Best, Diviyan

arthurshatver commented 4 years ago

Hello Arthur,

Actually the hyperparameters (mostly lambda1 and lambda2) have to be a little tuned depending on the size of the graph. We thought of having it automatically set depending on the size, but finally chose of having the default fixed. Could you try with : lambda1 = .1 and lambda2=1e-04?

This unittest was to show how the SAM works, I guess I should update it with the good parameters.

Best, Diviyan

Hello Diviyan,

I did as you said, but nothing was changed Capture

diviyank commented 4 years ago

Hello again, This is strange ; i'll look into it. If you have time, could you test with the implementation here: https://github.com/Diviyan-Kalainathan/SAM ?

arthurshatver commented 4 years ago

Hello again, This is strange ; i'll look into it. If you have time, could you test with the implementation here: https://github.com/Diviyan-Kalainathan/SAM ?

I'm sorry,but again the same problem. You can close this issue. I will try other motheds. Thank you for helping.

diviyank commented 4 years ago

Oh my I've just noticed your problem !

The fact is that SAM returns a confidence score for each edge of the graph ( even if it's close to 0). You would be able to see it with the adjacency matrix (networkx.adjacency_matrix(output).todense() i think). Therefore, it plots each edge as it is present in the graph (even if the associated weight is low.)

You must first proceed to threshold the confidence level of the graph before trying to plot it. From experience, thresholding to .6 or .7 leads to obtaining the best results.

Best, Diviyan

diviyank commented 4 years ago

I will close this issue for inactivity; as I consider it as solved.

ConnLiu commented 3 years ago

Oh my I've just noticed your problem !

The fact is that SAM returns a confidence score for each edge of the graph ( even if it's close to 0). You would be able to see it with the adjacency matrix (networkx.adjacency_matrix(output).todense() i think). Therefore, it plots each edge as it is present in the graph (even if the associated weight is low.)

You must first proceed to threshold the confidence level of the graph before trying to plot it. From experience, thresholding to .6 or .7 leads to obtaining the best results.

Best, Diviyan

Hi, @Diviyan-Kalainathan Your algorithm is really awesome, and I am wondering about how you transform the output of SAM( the confidence score matrix) to a causal graph which is DAG. I have tried thresholding to .6 or .7, but it returns a bad result. As the picture shows below, there are only several edges but a loop.