bd2kccd / causal-cmd

16 stars 8 forks source link

Discrepancies between final result and edge-probabilities in bootsrapping. Which is true? #79

Closed yasu-sh closed 1 year ago

yasu-sh commented 1 year ago

Hi, developers,

I have concerned if the result is correct or not in bootstrapping. There are 3 discrepancies in the edge list. I have tried to understand the json file structure as a hint. But too difficult for me.

Could you solve this issue? This result would be the same from the version around 1.2.1(I experienced in past).

PS > java -jar causal-cmd-1.4.1-jar-with-dependencies.jar --algorithm gfci --data-type discrete --delimiter comma --score bdeu-score --json-graph --dataset asia.csv --prefix c-tetrad --percentResampleSize 100 --numberResampling 5 --test g-square-test
Bootstrappiung: Number of searches that didn't return a graph = 0

Graph Edges:

  1. B o-> D [B o-> D]:1.0000;
  2. B o-o S [B o-o S]:1.0000;
  3. E --> D [D --> E]:1.0000;
  4. E --> X [E --> X]:1.0000;
  5. L o-> E [E o-> L]:1.0000;
  6. L o-o S [L o-o S]:1.0000;
  7. T o-> E [E o-> T]:1.0000;

I compared the same dataset and setting at TETRAD-GUI ver. 7.1.2-2 image

data: asia.csv (created from bnlearn, data(asia)) causal-cmd version 1.4.1, downloaded from https://s01.oss.sonatype.org/content/repositories/releases/io/github/cmu-phil/causal-cmd/1.4.1/

java -version openjdk version "1.8.0_342" OpenJDK Runtime Environment Corretto-8.342.07.3 (build 1.8.0_342-b07) OpenJDK 64-Bit Server VM Corretto-8.342.07.3 (build 25.342-b07, mixed mode)

asia.csv log file is here: c-tetrad.txt corresponding json file here: c-tetrad_graph.json.txt

``

yasu-sh commented 1 year ago

Additional info: This is the verbose of TETRAD-GUI. This could mean the TETRAD-GUI has this bug.

Resampling Search Result (10): Graph Nodes:(Cut) Graph Edges:

  1. E --> D dd nl
  2. E --> X dd nl
  3. B o-> D
  4. L o-> E
  5. L o-o S
  6. S o-o B
  7. T o-> E Graph Attributes: BIC: -11215.269098 Graph Node Attributes: (Cut)

Final Resampling Search Result: Graph Nodes:(Cut) Graph Edges:

  1. B o-> D [B o-> D]:1.0000;
  2. B o-o S [B o-o S]:1.0000;
  3. E --> D [D --> E]:1.0000;
  4. E --> X [E --> X]:1.0000;
  5. L o-> E [E o-> L]:1.0000;
  6. L o-o S [L o-o S]:1.0000;
  7. T o-> E [E o-> T]:1.0000; Graph Node Attributes: (Cut)
yasu-sh commented 1 year ago

This issue may be closed since the tetrad-lib is root cause. The action has been started. at cmu-phli/tetrad#1409.