Large calculation time difference between r-causal and causal-cmd v1.5.0

[yasu-sh]

I am wondering why the calculation time has big difference between as of 2019 and as of 2023. Could you tell me why this big difference happens? algorithms is different? I have used profiling tool at IntelliJ IDEA at causal-cmd v.1.5.0. but the process looks normal and test consumes log-gamma calculation a lot.

r-causal based on causal-cmd ver. 1.2.0-SNAPSHOT

[1] "INFO No prior knowledge for TETRAD." [1] "r-causal used Parameters: numberResampling = 0 / samplePrior = 10 / structurePrior = 1 / faithfulnessAssumed = false / verbose = false / maxDegree = 6 / symmetricFirstStep = true / printStream = " [1] "INFO structure learning completed. user/sys/elapsed duraton: 3.03 / 0.01 / 3.33" Actual turn around time: 3.33 secs.

causal-cmd based on the @kvb2univpitt 's release

[1] "INFO Causal structure learning started. Recipe: 5 | Algorithm: fges" [1] "INFO structure learning with causal-cmd started." [1] "INFO No prior knowledge for TETRAD." [1] "javaw -Xmx3G -Xss4096k -jar causal-cmd-1.5.0-jar-with-dependencies.jar --data-type discrete --delimiter comma --score bdeu-score --json-graph --maxDegree 6 --algorithm fges --dataset dt.tetrad.csv --prefix c-tetrad --knowledge prior.txt --priorEquivalentSampleSize 10 --numberResampling 0 --parallelized --symmetricFirstStep" [1] "INFO structure learning completed. user/sys/elapsed duraton: 0.19 / 0.64 / 100.25" Actual turn around time: 100 secs.

no prior (no information at prior.txt) dataset: hailfinder (https://github.com/bd2kccd/causal-cmd/files/10555256/dt.tetrad.csv)

condition is same as below, including dataset.: https://github.com/bd2kccd/causal-cmd/issues/80#issuecomment-1411716409

The process aborted since the profiling makes much longer.

[kvb2univpitt]

@yasu-sh One of the issue has to do with default parameters. If you don't provide a specific parameter, it will use the default value set by Java (boolean type) and by Tetrad (int type and double type). Each Tetrad algorithms has its own default parameter values that is optimized for it. Use the --default flag would make all the parameters, including boolean type, to use the algorithm specific default parameter values. This may improve your run time.

You can still override the default values set by the --default flag if you specify the value for a specific parameter.

Of course, there are other factors that also affect the runtime.

[jdramsey]

@yasu-sh Also, from 2019 to this year is a lot of time--we've made a number of optimizations to the code since then, even in the FGES class. Though I'm sure Kevin is right about parameter choices and how the parameters get used.

[yasu-sh]

@jdramsey @jdramsey Thanks for your views and opinion. I tried the argument as you recommended. The duration is not so changed. I will check at simple arguments later.

[1] "INFO No prior knowledge for TETRAD." [1] "java -Xmx512m -jar causal-cmd-1.5.0-jar-with-dependencies.jar --data-type discrete --delimiter comma --score bdeu-score --json-graph --maxDegree 6 --algorithm fges --dataset dt.tetrad.csv --prefix c-tetrad --seed 42 --default --priorEquivalentSampleSize 10 --numberResampling 0 --parallelized --symmetricFirstStep"

1. INSERT trcol41 --> trcol27 [] 46651.56856485745 degree = 1 indegree = 1 cond = 1
2. INSERT trcol27 --> trcol32 [] 29804.224330255773 degree = 2 indegree = 1 cond = 1
3. INSERT trcol27 --> trcol37 [] 28490.912815049505 degree = 3 indegree = 1 cond = 1 (omit)
4. DELETE trcol13 --> trcol25 H = [] NaYX = [trcol15] degree = 6 indegree = 3 diff = [trcol15] (37.74207135744655) cond = 1
5. INSERT trcol13 --> trcol14 [] 639.8455362967507 degree = 6 indegree = 3 cond = 2
6. DELETE trcol13 --> trcol14 H = [trcol15] NaYX = [trcol15] degree = 6 indegree = 3 diff = [] (23.54686309976387) cond = 0 --- Directing trcol15 --- trcol14 to trcol14 --> trcol15 Elapsed time = 90.791 s [1] "INFO structure learning completed. user/sys/elapsed duraton: 0.11 / 0.15 / 94.17"

[jdramsey]

Thanks!

[yasu-sh]

As I mentioned yesterday, I compared 3 conditions:

1. causal-cmd v1.5.0 latest version from sonartype
2. causal-cmd v.1.2.0-SNAPSHOT Downloaded from r-causal repos.
3. causal-cmd v1.2.0-SNAPSHOT Build at my machine from git, revision 17767060f08aecf7348599294aa7f1a880f34380 (committed on Nov 2, 2019) dataset: dt.hailfinder.csv condition: v1.5.0 - default / v1.2.0 - verbose used since the '--default' arg. does not exists

basic execution at powershell

v 1.5.0 PS E:\Tetrad> java -jar "causal-cmd-1.5.0-jar-with-dependencies.jar" --data-type discrete --delimiter comma --score bdeu-score --json-graph --algorithm fges --dataset dt.hailfinder.csv --default Elapsed time = 4843.091 s

v. 1.2.0 PS E:\Tetrad> Measure-Command {java -jar "causal-cmd-1.2.0-SNAPSHOT-jar-with-dep-selfbuild.jar" --data-type discrete --delimiter comma --score bdeu-score --json-graph --algorithm fges --dataset dt.hailfinder.csv --verbose} TotalSeconds : 11.9347421

I have notices several points as below:

• the search process steps is not so different between v1.2.0 and v1.5.0.
• But causal-cmd v.1.5.0 takes much longer time , espcially at over degree = 7. (v1.5.0: 4843 secs / v1.2.0 12secs)
• r-causal's causal-cmd would have been modified by @chirayukong and verbose does not work, no intermediate information available. I needed to build the jar at similar timing of upload of r-causal's jar to compare.

How come,,,? I feel like seeing v.1.2.0 by profiling, but it might not work for getting new ideas. @kvb2univpitt @jdramsey I would appreciate if you could make comments or views?

Surely hailfinder dataset has many degrees as ground truth. it would make time-consuming. (Lowering maxDegrees is easy way, but it does not mean good result for many-degree strucure.)

Logs

1. causal-cmd-v1.5.0-default-hailfinder.txt
2. causal-cmd-v.1.2.0-rcausal-hailfinder.txt
3. causal-cmd-v1.2.0-Build-AsofDec2019-hailfinder.txt

PS E:\Tetrad> java -version openjdk version "11.0.18" 2023-01-17 OpenJDK Runtime Environment Temurin-11.0.18+10 (build 11.0.18+10) OpenJDK 64-Bit Server VM Temurin-11.0.18+10 (build 11.0.18+10, mixed mode)

[jdramsey]

Well, one comment I can make is that the R version is very out of date; the person working on that left the group some time ago, and there's been some difficulty updating that, though a number of people are interested, so I may try my hand at it myself (though I am not skilled at R progrmming). It it's just a matter of changing the causal command version and updating the calls made, perhaps I can. but it's not usign 1.5.0 but some much older version.

[jdramsey]

Much work has been done since 1.2.0 in any case; I would rely more on 1.5.0. BTW we will soon some out with a new version; enough changes have accrued I think, even in this short time.

[yasu-sh]

Thanks. Okay, I rely on ver. 1.5.0 and use different settings on the latest version, as you recommended. I just thought newer version could make faster search time.

Surely accuracy and speed has trade-off. Let me know when you notice some ideas coming up.

[jdramsey]

Hold on, for 1.5.0, could you add the "--defaults" flag?

[jdramsey]

Hold on...let me think...

On Fri, Feb 3, 2023 at 1:55 AM Yasuhiro Shimodaira @.***> wrote:

Thanks. Okay, I rely on ver. 1.5.0 and use different settings on the latest version, as you recommended. I just thought newer version could make faster search time.

Surely accuracy and speed has trade-off. Let me know when you notice some ideas coming up.

— Reply to this email directly, view it on GitHub https://github.com/bd2kccd/causal-cmd/issues/83#issuecomment-1415158338, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACLFSR4Q3PAOHSH323AKSALWVSTUHANCNFSM6AAAAAAUNPQSLU . You are receiving this because you were mentioned.Message ID: @.***>

[jdramsey]

lngamma huh? That's interesting. I did switch from our "quick and dirty" ln gamma calculation to the library lngamma calculation in the Apache library--I wonder if that could have made the difference? I suppose I could switch back...

On Fri, Feb 3, 2023 at 7:20 AM Joseph Ramsey @.***> wrote:

Hold on...let me think...

On Fri, Feb 3, 2023 at 1:55 AM Yasuhiro Shimodaira < @.***> wrote:

Thanks. Okay, I rely on ver. 1.5.0 and use different settings on the latest version, as you recommended. I just thought newer version could make faster search time.

Surely accuracy and speed has trade-off. Let me know when you notice some ideas coming up.

— Reply to this email directly, view it on GitHub https://github.com/bd2kccd/causal-cmd/issues/83#issuecomment-1415158338, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACLFSR4Q3PAOHSH323AKSALWVSTUHANCNFSM6AAAAAAUNPQSLU . You are receiving this because you were mentioned.Message ID: @.***>

[jdramsey]

Here's the lngamma I was using:

public static double lngamma(double xx) { //Returns the value ln[?(xx)] for xx > 0.

if (xx <= 0) return Double.NaN;

//Internal arithmetic will be done in double precision, a nicety

that you can omit if ?ve-?gure //accuracy is good enough. double x; double y; double tmp; double ser;

int j;
y = x = xx;
tmp = x + 5.5;
tmp -= (x + 0.5) * Math.log(tmp);
ser = 1.000000000190015;
for (j = 0; j <= 5; j++) {
    ser += ProbUtils.cof[j] / ++y;
}
return -tmp + Math.log(2.5066282746310005 * ser / x);

}

private static final double[] cof = {76.18009172947146, -86.50532032941677, 24.01409824083091, -1.231739572450155, 0.1208650973866179e-2, -0.5395239384953e-5};

Here's the loggamma I switched to in the Apache library:

public static double logGamma(double x) { double ret; if (!Double.isNaN(x) && !(x <= 0.0)) { if (x < 0.5) { return logGamma1p(x) - FastMath.log(x); }

    if (x <= 2.5) {
        return logGamma1p(x - 0.5 - 0.5);
    }

    if (x <= 8.0) {
        int n = (int)FastMath.floor(x - 1.5);
        double prod = 1.0;

        for(int i = 1; i <= n; ++i) {
            prod *= x - (double)i;
        }

        return logGamma1p(x - (double)(n + 1)) + FastMath.log(prod);
    }

    double sum = lanczos(x);
    double tmp = x + 4.7421875 + 0.5;
    ret = (x + 0.5) * FastMath.log(tmp) - tmp + HALF_LOG_2_PI +

FastMath.log(sum / x); } else { ret = Double.NaN; }

return ret;

}

public static double logGamma1p(double x) throws NumberIsTooSmallException, NumberIsTooLargeException { if (x < -0.5) { throw new NumberIsTooSmallException(x, -0.5, true); } else if (x > 1.5) { throw new NumberIsTooLargeException(x, 1.5, true); } else { return -FastMath.log1p(invGamma1pm1(x)); } }

public static double log1p(double x) { if (x == -1.0) { return Double.NEGATIVE_INFINITY; } else if (x == Double.POSITIVE_INFINITY) { return Double.POSITIVE_INFINITY; } else { double xpa; if (!(x > 1.0E-6) && !(x < -1.0E-6)) { xpa = (x 0.3333333333333333 - 0.5) x + 1.0; return xpa x; } else { xpa = 1.0 + x; double xpb = -(xpa - 1.0 - x); double[] hiPrec = new double[2]; double lores = log(xpa, hiPrec); if (Double.isInfinite(lores)) { return lores; } else { double fx1 = xpb / xpa; double epsilon = 0.5 fx1 + 1.0; return epsilon * fx1 + hiPrec[1] + hiPrec[0]; } } } }

private static double log(double x, double[] hiPrec) { if (x == 0.0) { return Double.NEGATIVE_INFINITY; } else { long bits = Double.doubleToRawLongBits(x); if (((bits & Long.MIN_VALUE) != 0L || x != x) && x != 0.0) { if (hiPrec != null) { hiPrec[0] = Double.NaN; }

        return Double.NaN;
    } else if (x == Double.POSITIVE_INFINITY) {
        if (hiPrec != null) {
            hiPrec[0] = Double.POSITIVE_INFINITY;
        }

        return Double.POSITIVE_INFINITY;
    } else {
        int exp = (int)(bits >> 52) - 1023;
        if ((bits & 9218868437227405312L) == 0L) {
            if (x == 0.0) {
                if (hiPrec != null) {
                    hiPrec[0] = Double.NEGATIVE_INFINITY;
                }

                return Double.NEGATIVE_INFINITY;
            }

            for(bits <<= 1; (bits & 4503599627370496L) == 0L; bits <<= 1) {
                --exp;
            }
        }

        double ab;
        double xa;
        if ((exp == -1 || exp == 0) && x < 1.01 && x > 0.99 &&

hiPrec == null) { double xa = x - 1.0; double xb = xa - x + 1.0; double tmp = xa * 1.073741824E9; double aa = xa + tmp - tmp; double ab = xa - aa; xa = aa; xb = ab; double[] lnCoef_last = LN_QUICK_COEF[LN_QUICK_COEF.length - 1]; ab = lnCoef_last[0]; xa = lnCoef_last[1];

            for(int i = LN_QUICK_COEF.length - 2; i >= 0; --i) {
                aa = ab * xa;
                ab = ab * xb + xa * xa + xa * xb;
                tmp = aa * 1.073741824E9;
                ab = aa + tmp - tmp;
                xa = aa - ab + ab;
                double[] lnCoef_i = LN_QUICK_COEF[i];
                aa = ab + lnCoef_i[0];
                ab = xa + lnCoef_i[1];
                tmp = aa * 1.073741824E9;
                ab = aa + tmp - tmp;
                xa = aa - ab + ab;
            }

            aa = ab * xa;
            ab = ab * xb + xa * xa + xa * xb;
            tmp = aa * 1.073741824E9;
            ab = aa + tmp - tmp;
            xa = aa - ab + ab;
            return ab + xa;
        } else {
            double[] lnm = FastMath.lnMant.LN_MANT[(int)((bits &

4499201580859392L) >> 42)]; double epsilon = (double)(bits & 4398046511103L) / (4.503599627370496E15 + (double)(bits & 4499201580859392L)); double lnza = 0.0; double lnzb = 0.0; double tmp; double aa; if (hiPrec != null) { tmp = epsilon 1.073741824E9; aa = epsilon + tmp - tmp; ab = epsilon - aa; xa = aa; double numer = (double)(bits & 4398046511103L); double denom = 4.503599627370496E15 + (double)(bits & 4499201580859392L); aa = numer - aa denom - ab * denom; double xb = ab + aa / denom; double[] lnCoef_last = LN_HI_PREC_COEF[LN_HI_PREC_COEF.length - 1]; double ya = lnCoef_last[0]; double yb = lnCoef_last[1];

                for(int i = LN_HI_PREC_COEF.length - 2; i >= 0; --i) {
                    aa = ya * xa;
                    ab = ya * xb + yb * xa + yb * xb;
                    tmp = aa * 1.073741824E9;
                    ya = aa + tmp - tmp;
                    yb = aa - ya + ab;
                    double[] lnCoef_i = LN_HI_PREC_COEF[i];
                    aa = ya + lnCoef_i[0];
                    ab = yb + lnCoef_i[1];
                    tmp = aa * 1.073741824E9;
                    ya = aa + tmp - tmp;
                    yb = aa - ya + ab;
                }

                aa = ya * xa;
                ab = ya * xb + yb * xa + yb * xb;
                lnza = aa + ab;
                lnzb = -(lnza - aa - ab);
            } else {
                lnza = -0.16624882440418567;
                lnza = lnza * epsilon + 0.19999954120254515;
                lnza = lnza * epsilon + -0.2499999997677497;
                lnza = lnza * epsilon + 0.3333333333332802;
                lnza = lnza * epsilon + -0.5;
                lnza = lnza * epsilon + 1.0;
                lnza *= epsilon;
            }

            tmp = 0.6931470632553101 * (double)exp;
            aa = 0.0;
            ab = tmp + lnm[0];
            xa = -(ab - tmp - lnm[0]);
            tmp = ab;
            aa += xa;
            ab += lnza;
            xa = -(ab - tmp - lnza);
            tmp = ab;
            aa += xa;
            ab += 1.1730463525082348E-7 * (double)exp;
            xa = -(ab - tmp - 1.1730463525082348E-7 * (double)exp);
            tmp = ab;
            aa += xa;
            ab += lnm[1];
            xa = -(ab - tmp - lnm[1]);
            tmp = ab;
            aa += xa;
            ab += lnzb;
            xa = -(ab - tmp - lnzb);
            aa += xa;
            if (hiPrec != null) {
                hiPrec[0] = ab;
                hiPrec[1] = aa;
            }

            return ab + aa;
        }
    }
}

}

You're suggesting the second method is slower than the first. That's a real possibility; and then the question is, which is more accurate? My hunch is that the library method is more accurate?

My hunch is the second should be more accurate; it's a more careful calculation.

If I get a chance, I'll compare the two.

Joe

On Fri, Feb 3, 2023 at 7:25 AM Joseph Ramsey @.***> wrote:

lngamma huh? That's interesting. I did switch from our "quick and dirty" ln gamma calculation to the library lngamma calculation in the Apache library--I wonder if that could have made the difference? I suppose I could switch back...

On Fri, Feb 3, 2023 at 7:20 AM Joseph Ramsey @.***> wrote:

Hold on...let me think...

On Fri, Feb 3, 2023 at 1:55 AM Yasuhiro Shimodaira < @.***> wrote:

Thanks. Okay, I rely on ver. 1.5.0 and use different settings on the latest version, as you recommended. I just thought newer version could make faster search time.

Surely accuracy and speed has trade-off. Let me know when you notice some ideas coming up.

— Reply to this email directly, view it on GitHub https://github.com/bd2kccd/causal-cmd/issues/83#issuecomment-1415158338, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACLFSR4Q3PAOHSH323AKSALWVSTUHANCNFSM6AAAAAAUNPQSLU . You are receiving this because you were mentioned.Message ID: @.***>

[jdramsey]

Yeah, I switched to the Apache Gamma.loggamma method in the 7.1.3-1 release--item 52 here:

https://github.com/cmu-phil/tetrad/releases

[jdramsey]

...and the BDeu score makes extensive use of this function:

   double score = 0.0;

    score += getPriorForStructure(parents.length, N);

    double cellPrior = getSamplePrior() / (c * r);
    double rowPrior = getSamplePrior() / r;

    for (int j = 0; j < r; j++) {
        score -= Gamma.logGamma(rowPrior + n_j[j]);

        for (int k = 0; k < c; k++) {
            score += Gamma.logGamma(cellPrior + n_jk[j][k]);
        }
    }

    score += r * Gamma.logGamma(rowPrior);
    score -= c * r * Gamma.logGamma(cellPrior);

[jdramsey]

Holy crap, I never did the comparison--out to 6 decimal places they'r the same!!

Log gamma values

x   ProUtils    Apache
10.0    12.801827   12.801827
20.0    39.339884   39.339884
30.0    71.257039   71.257039
40.0    106.631760  106.631760
50.0    144.565744  144.565744
60.0    184.533829  184.533829
70.0    226.190548  226.190548
80.0    269.291098  269.291098
90.0    313.652830  313.652830
100.0   359.134205  359.134205

Here's the code:

   public static void compareLogGammas() {
        NumberFormat nf = new DecimalFormat("0.000000");

        System.out.println("Log gamma values");
        System.out.println();

        System.out.println("x\tProUtils\tApache");

        for (int i = 10; i <= 100; i += 10) {
            double x = i / 1d;

            double g1 = ProbUtils.lngamma(x);
            double g2 = Gamma.logGamma(x);

            System.out.println(x + "\t" + nf.format(g1) + "\t" + nf.format(g2));
        }
    }

[jdramsey]

So which is faster?

Here's my code:

    public static void compareLogGammas2() {
        NumberFormat nf = new DecimalFormat("0.000000");

        long start = ManagementFactory.getThreadMXBean().getCurrentThreadCpuTime();

        int numIter = 10000000;
        double nano = 1000000000d;

        for (int i = 0; i < numIter; i++) {
            ProbUtils.lngamma(1000.0);
        }

        long stop = ManagementFactory.getThreadMXBean().getCurrentThreadCpuTime();

        System.out.println("Elapsed ProbUtils.lngamma = " + (stop - start) / nano + "s");

        long start2 = ManagementFactory.getThreadMXBean().getCurrentThreadCpuTime();

        for (int i = 0; i < numIter; i++) {
            Gamma.logGamma(1000.0);
        }

        long stop2 = ManagementFactory.getThreadMXBean().getCurrentThreadCpuTime();

        System.out.println("Elapsed Gamma.logGamma = " + (stop2 - start2) / nano + "s");
    }

Here's the result:

Elapsed ProbUtils.lngamma = 1.630955s
Elapsed Gamma.logGamma = 0.26709s

Gamma.logGamma is significantly faster in CPU time. Hmm... so that' s not the reason.

[jdramsey]

Hold on, there was also a change in whether FGES was parallelized or not. @yasu-sh what size machine are you using? Parallelization for FGES was turned off by default for 7.1.3-1 (I'll add that to the release notes). To turn it on, I think you need to add the flag "--parallelized". Could you try that?

    private boolean parallelized = false;

This is a new field in FGES that was added in April, 2022. We added this so that we could run FGES in the algcomparison API and compare it to other algorithms, where the multithreading was done across algorithm runs instead of inside FGES.

[jdramsey]

Added this note to the release notes for 7.1.3-1 (sorry for leaving it out--that was a major release; there are probably a few other defaults I left out as well:

Set FGES to be single-thread by default; to to multithreading, the flag "parallelized" needs to be set to true (in causal cmd, "--parallelized".

[yasu-sh]

@jdramsey Thanks a lot for telling your analysis and consideration! I read the code of R's library(dll) for combination size calclulation about a year ago. I knew the gamma function was used and the calclulation time was much faster than my colleague's code in C++. So I think the gamma function of log is a key to make efficient search.

I have checked different cofiguration at v.1.5.0 already, and I set the similar option settings including --parallelized as long as I could. (I noticed v.1.2.0 have parallelize enabled always and I also thought FGES needed to enable to make native optimized search.)

Hold on, there was also a change in whether FGES was parallelized or not. @yasu-sh what size machine are you using?

OS: Microsoft Windows 10 Pro x64 21H2 / 10.0.19044 N/A Build 19044 System Model: HP ZBook Studio G4 Processor(s): 1 Processor(s) Installed. Intel(R) Core(TM) i7-7820HQ CPU @ 2.90GHz、2904 Mhz、4 Cores, 8 logical processors [01]: Intel64 Family 6 Model 158 Stepping 9 GenuineIntel ~2904 Mhz System Locale: ja;Japanese Total Physical Memory: 32,381 MB

I would like to add some later on. See you then.

[jdramsey]

@yasu-sh Have you discovered any new information about this? Your machine should certainly be large enough to do parallel processing. When you run java, it's possible the heap size is too emall--that would slow things down a lot. You can increase the heap size by adding a flag to the Java command like -Xmx10g for 10 Gigabytes.

[yasu-sh]

@jdramsey Unfortunately, no information so far. I noticed this phenomena on this issue at -Xmx4G flag for java. So I thought the heap memory size would be enough to get the process functional....

I am afraid I need to help my colleagues at another projects for 2 weeks, therefore I have little time to check about this topic. Today, I tried at the statement as below and took so many minues(already 30min.) Not ended now.

Win10 Not fully occupied by process,,, windows specific???

java -Xmx10G -jar "causal-cmd-1.5.0-jar-with-dependencies.jar" --data-type discrete --delimiter comma --score bdeu-score --json-graph --algorithm fges --dataset dt.hailfinder.csv --parallelized

datafile(posted the same above): https://github.com/bd2kccd/causal-cmd/files/10575729/dt.hailfinder.csv

[yasu-sh]

This time (just added parallelized & expanded 10GB heap memory), out of memory error came up. Finally saturated at 10GB memory consumption and made these as below.

PS E:\Tetrad> java -Xmx10G -jar "causal-cmd-1.5.0-jar-with-dependencies.jar" --data-type discrete --delimiter comma --score bdeu-score --json-graph --algorithm fges --dataset dt.hailfinder.csv --parallelized java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError at java.base/java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1006) at edu.cmu.tetrad.search.Fges.calculateArrowsForward(Fges.java:718) at edu.cmu.tetrad.search.Fges.access$500(Fges.java:61) at edu.cmu.tetrad.search.Fges$1AdjTask.call(Fges.java:593) at edu.cmu.tetrad.search.Fges$1AdjTask.call(Fges.java:541) at java.base/java.util.concurrent.ForkJoinTask$AdaptedCallable.exec(ForkJoinTask.java:1448) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) at java.base/java.util.concurrent.ForkJoinTask.tryExternalHelp(ForkJoinTask.java:381) at java.base/java.util.concurrent.ForkJoinTask.externalAwaitDone(ForkJoinTask.java:323) at java.base/java.util.concurrent.ForkJoinTask.doJoin(ForkJoinTask.java:398) at java.base/java.util.concurrent.ForkJoinTask.quietlyJoin(ForkJoinTask.java:1078) at java.base/java.util.concurrent.ForkJoinPool.invokeAll(ForkJoinPool.java:2519) at edu.cmu.tetrad.search.Fges.reevaluateForward(Fges.java:616) at edu.cmu.tetrad.search.Fges.fes(Fges.java:522) at edu.cmu.tetrad.search.Fges.search(Fges.java:207) at edu.cmu.tetrad.algcomparison.algorithm.oracle.cpdag.Fges.search(Fges.java:79) at edu.pitt.dbmi.causal.cmd.tetrad.TetradRunner.lambda$runSearch$3(TetradRunner.java:203) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at edu.pitt.dbmi.causal.cmd.tetrad.TetradRunner.runSearch(TetradRunner.java:202) at edu.pitt.dbmi.causal.cmd.tetrad.TetradRunner.runAlgorithm(TetradRunner.java:119) at edu.pitt.dbmi.causal.cmd.CausalCmdApplication.runTetrad(CausalCmdApplication.java:149) at edu.pitt.dbmi.causal.cmd.CausalCmdApplication.main(CausalCmdApplication.java:113) Caused by: java.lang.OutOfMemoryError at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490) at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:603) ... 22 more Caused by: java.lang.OutOfMemoryError: Java heap space at edu.cmu.tetrad.search.BDeuScore.localScore(BDeuScore.java:112) at edu.cmu.tetrad.search.BDeuScore.localScoreDiff(BDeuScore.java:179) at edu.cmu.tetrad.search.Fges.scoreGraphChange(Fges.java:1078) at edu.cmu.tetrad.search.Fges.insertEval(Fges.java:776) at edu.cmu.tetrad.search.Fges.access$600(Fges.java:61) at edu.cmu.tetrad.search.Fges$1EvalTask.call(Fges.java:679) at edu.cmu.tetrad.search.Fges$1EvalTask.call(Fges.java:660) at java.base/java.util.concurrent.ForkJoinTask$AdaptedCallable.exec(ForkJoinTask.java:1448) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)

[yasu-sh]

@jdramsey even at 10GB, the max memory size might not be enough. Let me try 25GB..

[yasu-sh]

Sometime explosive memory consumption happens like saws(for woods). I hope to use conventional calcluration since the calculation is more effecient than newer table model of apache commons math lib. Could @jdramsey try the same data(hailfinder) at same options at your computer as below(memory can be changed)? I saw your past comments had Mac window, operation systems could be related to this phenomena.

java -Xmx25G -jar "causal-cmd-1.5.0-jar-with-dependencies.jar" --data-type discrete --delimiter comma --score bdeu-score --json-graph --algorithm fges --dataset dt.hailfinder.csv --parallelized

I have checked by myself at Mac machine. ~Mac(Unix based), Win & Unix have different features and I have read that windows system cannot fork the memory so many memory copies via TCP/IP in R environment whereas unix system can just copies the memory of the running enviromnet in the same computer.~

jconsole's manual page

[yasu-sh]

@jdramsey The other day I have purchased a Mac mini M2(new model) with minimum spec for my private. I have tried at the condition as below and the result / conclusion is the similar. The new scoring algorithm at parallel computing does not suit for many-degree causal network??

parallelized setting	Yes	No
duration	Stopped over 2h	1937.134 secs
Out of memory error	Occurred	Not Occurred
Verbose's progress	2x -3x slower than the other	Slow, but progress

Mac-mini Tetrad % java -version
openjdk version "11.0.18" 2023-01-17
OpenJDK Runtime Environment Temurin-11.0.18+10 (build 11.0.18+10)
OpenJDK 64-Bit Server VM Temurin-11.0.18+10 (build 11.0.18+10, mixed mode)

Tetrad % /usr/sbin/system_profiler -detailLevel mini 
Hardware:
    Hardware Overview:
      Model Name: Mac mini
      Model Identifier: Mac14,3
      Model Number: MMFJ3J/A
      Chip: Apple M2
      Total Number of Cores: 8 (4 performance and 4 efficiency)
      Memory: 8 GB
      System Firmware Version: 8419.80.7
      OS Loader Version: 8419.80.7

Memory:
      Memory: 8 GB
      Type: LPDDR5
      Manufacturer: Micron

Mac-mini Tetrad % java -Xmx5G -jar "causal-cmd-1.5.0-jar-with-dependencies.jar" --data-type discrete --delimiter comma --score bdeu-score --json-graph --algorithm fges --dataset dt.hailfinder.csv --parallelized --verbose
1. INSERT ScnRelPlFcst --> Scenario [] 46651.56856485745 degree = 1 indegree = 1 cond = 1
2. INSERT ScnRelPlFcst --> ScenRelAMIns [] 29804.224330255773 degree = 2 indegree = 0 cond = 1
3. INSERT Scenario --> ScenRel34 [] 28490.912815049505 degree = 2 indegree = 0 cond = 1
4. INSERT AreaMesoALS --> CombVerMo [] 25244.402367736307 degree = 2 indegree = 0 cond = 1
5. INSERT CapChange --> CompPlFcst [] 21347.6368234152 degree = 2 indegree = 0 cond = 1
6. INSERT PlainsFcst --> N34StarFcst [] 13651.483290013432 degree = 2 indegree = 1 cond = 1
7. INSERT Scenario --> WindFieldPln [] 10240.609710882767 degree = 3 indegree = 1 cond = 1
8. INSERT Scenario --> SfcWndShfDis [] 9896.183754378471 degree = 4 indegree = 1 cond = 1
9. INSERT MountainFcst --> R5Fcst [] 9841.66758822378 degree = 4 indegree = 1 cond = 1
10. INSERT N34StarFcst --> R5Fcst [MountainFcst] 11481.578596648274 degree = 4 indegree = 1 cond = 2
--- Directing MountainFcst --> R5Fcst
11. INSERT Scenario --> ScenRelAMCIN [] 9827.080206968974 degree = 5 indegree = 2 cond = 1
12. INSERT AMInstabMt --> InsInMt [] 9200.035720480517 degree = 5 indegree = 2 cond = 1
13. INSERT CombMoisture --> AreaMoDryAir [] 8697.303519154746 degree = 5 indegree = 2 cond = 1
14. INSERT Scenario --> Dewpoints [] 8060.360232083614 degree = 6 indegree = 2 cond = 1
15. INSERT Scenario --> WindAloft [] 7691.486439166631 degree = 7 indegree = 2 cond = 1
16. INSERT CombClouds --> CldShadeOth [] 7582.724420807997 degree = 7 indegree = 2 cond = 1
17. INSERT InsInMt --> CldShadeConv [] 6951.145041866659 degree = 7 indegree = 2 cond = 1
18. INSERT Scenario --> MvmtFeatures [] 6064.721401109418 degree = 8 indegree = 2 cond = 1
19. INSERT InsInMt --> OutflowFrMt [] 5861.459915747448 degree = 8 indegree = 2 cond = 1
20. INSERT AMCINInScen --> MorningCIN [] 5744.261326911303 degree = 8 indegree = 2 cond = 1
21. INSERT VISCloudCov --> CombClouds [] 5505.233648352496 degree = 8 indegree = 2 cond = 1
22. INSERT InsInMt --> MountainFcst [] 5471.96694397506 degree = 8 indegree = 2 cond = 1
23. INSERT InsSclInScen --> AMInsWliScen [] 5389.565668086932 degree = 8 indegree = 2 cond = 1
24. INSERT InsChange --> LoLevMoistAd [] 5314.347428221692 degree = 8 indegree = 2 cond = 1
25. INSERT Scenario --> TempDis [] 5268.570835127262 degree = 9 indegree = 2 cond = 1
26. INSERT AMCINInScen --> CapInScen [] 5161.809906832612 degree = 9 indegree = 2 cond = 1
27. INSERT CapChange --> CapInScen [AMCINInScen] 7935.477957001622 degree = 9 indegree = 2 cond = 2
--- Directing AMCINInScen --> CapInScen
28. INSERT InsChange --> InsSclInScen [AMInsWliScen] 5100.860672891173 degree = 9 indegree = 2 cond = 2
--- Directing AMInsWliScen --> InsSclInScen
29. INSERT Scenario --> RHRatio [] 5024.198470412222 degree = 10 indegree = 2 cond = 1
30. INSERT Scenario --> LowLLapse [] 4395.0530244349575 degree = 11 indegree = 2 cond = 1
^[OP^[OP11java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError
    at java.base/java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1006)
    at edu.cmu.tetrad.search.Fges.calculateArrowsForward(Fges.java:718)
    at edu.cmu.tetrad.search.Fges.access$500(Fges.java:61)
    at edu.cmu.tetrad.search.Fges$1AdjTask.call(Fges.java:593)
    at edu.cmu.tetrad.search.Fges$1AdjTask.call(Fges.java:541)
    at java.base/java.util.concurrent.ForkJoinTask$AdaptedCallable.exec(ForkJoinTask.java:1448)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Caused by: java.lang.OutOfMemoryError
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
    at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:603)
    ... 11 more
Caused by: java.lang.OutOfMemoryError: Java heap space
    at edu.cmu.tetrad.search.BDeuScore.localScore(BDeuScore.java:111)
    at edu.cmu.tetrad.search.BDeuScore.localScoreDiff(BDeuScore.java:179)
    at edu.cmu.tetrad.search.Fges.scoreGraphChange(Fges.java:1078)
    at edu.cmu.tetrad.search.Fges.insertEval(Fges.java:776)
    at edu.cmu.tetrad.search.Fges.access$600(Fges.java:61)
    at edu.cmu.tetrad.search.Fges$1EvalTask.call(Fges.java:679)
    at edu.cmu.tetrad.search.Fges$1EvalTask.call(Fges.java:660)
    ... 6 more
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError
    at java.base/java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1006)
    at edu.cmu.tetrad.search.Fges.calculateArrowsForward(Fges.java:718)
    at edu.cmu.tetrad.search.Fges.access$500(Fges.java:61)
    at edu.cmu.tetrad.search.Fges$1AdjTask.call(Fges.java:593)
    at edu.cmu.tetrad.search.Fges$1AdjTask.call(Fges.java:541)
    at java.base/java.util.concurrent.ForkJoinTask$AdaptedCallable.exec(ForkJoinTask.java:1448)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Caused by: java.lang.OutOfMemoryError
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
    at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:603)
    ... 11 more
Caused by: java.lang.OutOfMemoryError: Java heap space
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError
    at java.base/java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1006)
    at edu.cmu.tetrad.search.Fges.calculateArrowsForward(Fges.java:718)
    at edu.cmu.tetrad.search.Fges.access$500(Fges.java:61)
    at edu.cmu.tetrad.search.Fges$1AdjTask.call(Fges.java:593)
    at edu.cmu.tetrad.search.Fges$1AdjTask.call(Fges.java:541)
    at java.base/java.util.concurrent.ForkJoinTask$AdaptedCallable.exec(ForkJoinTask.java:1448)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Caused by: java.lang.OutOfMemoryError
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
    at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:603)
    ... 11 more
Caused by: java.lang.OutOfMemoryError: Java heap space
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError
    at java.base/java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1006)
    at edu.cmu.tetrad.search.Fges.calculateArrowsForward(Fges.java:718)
    at edu.cmu.tetrad.search.Fges.access$500(Fges.java:61)
    at edu.cmu.tetrad.search.Fges$1AdjTask.call(Fges.java:593)
    at edu.cmu.tetrad.search.Fges$1AdjTask.call(Fges.java:541)
    at java.base/java.util.concurrent.ForkJoinTask$AdaptedCallable.exec(ForkJoinTask.java:1448)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Caused by: java.lang.OutOfMemoryError
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
    at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:603)
    ... 11 more
Caused by: java.lang.OutOfMemoryError: Java heap space
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError
    at java.base/java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1006)
    at edu.cmu.tetrad.search.Fges.calculateArrowsForward(Fges.java:718)
    at edu.cmu.tetrad.search.Fges.access$500(Fges.java:61)
    at edu.cmu.tetrad.search.Fges$1AdjTask.call(Fges.java:593)
    at edu.cmu.tetrad.search.Fges$1AdjTask.call(Fges.java:541)
    at java.base/java.util.concurrent.ForkJoinTask$AdaptedCallable.exec(ForkJoinTask.java:1448)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
Caused by: java.lang.OutOfMemoryError
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
    at java.base/java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:603)
    ... 11 more
Caused by: java.lang.OutOfMemoryError: Java heap space
(manually stopped)

Mac-mini Tetrad % java -Xmx5G -jar "causal-cmd-1.5.0-jar-with-dependencies.jar" --data-type discrete --delimiter comma --score bdeu-score --json-graph --algorithm fges --dataset dt.hailfinder.csv --verbose  
1. INSERT ScnRelPlFcst --> Scenario [] 46651.56856485745 degree = 1 indegree = 1 cond = 1
2. INSERT ScnRelPlFcst --> ScenRelAMIns [] 29804.224330255773 degree = 2 indegree = 0 cond = 1
3. INSERT Scenario --> ScenRel34 [] 28490.912815049505 degree = 2 indegree = 0 cond = 1
4. INSERT AreaMesoALS --> CombVerMo [] 25244.402367736307 degree = 2 indegree = 0 cond = 1
5. INSERT CapChange --> CompPlFcst [] 21347.6368234152 degree = 2 indegree = 0 cond = 1
6. INSERT PlainsFcst --> N34StarFcst [] 13651.483290013432 degree = 2 indegree = 1 cond = 1
7. INSERT ScnRelPlFcst --> WindFieldPln [] 10240.609710882767 degree = 3 indegree = 1 cond = 1
8. INSERT Scenario --> SfcWndShfDis [] 9896.183754378471 degree = 3 indegree = 1 cond = 1
9. INSERT MountainFcst --> R5Fcst [] 9841.66758822378 degree = 3 indegree = 1 cond = 1
10. INSERT N34StarFcst --> R5Fcst [MountainFcst] 11481.578596648274 degree = 3 indegree = 1 cond = 2
--- Directing MountainFcst --> R5Fcst
11. INSERT Scenario --> ScenRelAMCIN [] 9827.080206968974 degree = 4 indegree = 2 cond = 1
12. INSERT AMInstabMt --> InsInMt [] 9200.035720480517 degree = 4 indegree = 2 cond = 1
13. INSERT CombMoisture --> AreaMoDryAir [] 8697.303519154746 degree = 4 indegree = 2 cond = 1
14. INSERT Scenario --> Dewpoints [] 8060.360232083614 degree = 5 indegree = 2 cond = 1
15. INSERT ScnRelPlFcst --> WindAloft [] 7691.486439166631 degree = 5 indegree = 2 cond = 1
16. INSERT CombClouds --> CldShadeOth [] 7582.724420807997 degree = 5 indegree = 2 cond = 1
17. INSERT InsInMt --> CldShadeConv [] 6951.145041866659 degree = 5 indegree = 2 cond = 1
18. INSERT ScnRelPlFcst --> MvmtFeatures [] 6064.721401109418 degree = 5 indegree = 2 cond = 1
19. INSERT InsInMt --> OutflowFrMt [] 5861.459915747448 degree = 5 indegree = 2 cond = 1
20. INSERT AMCINInScen --> MorningCIN [] 5744.261326911303 degree = 5 indegree = 2 cond = 1
21. INSERT VISCloudCov --> CombClouds [] 5505.233648352496 degree = 5 indegree = 2 cond = 1
22. INSERT InsInMt --> MountainFcst [] 5471.96694397506 degree = 5 indegree = 2 cond = 1
23. INSERT InsSclInScen --> AMInsWliScen [] 5389.565668086932 degree = 5 indegree = 2 cond = 1
24. INSERT InsChange --> LoLevMoistAd [] 5314.347428221692 degree = 5 indegree = 2 cond = 1
25. INSERT Scenario --> TempDis [] 5268.570835127262 degree = 6 indegree = 2 cond = 1
26. INSERT AMCINInScen --> CapInScen [] 5161.809906832612 degree = 6 indegree = 2 cond = 1
27. INSERT CompPlFcst --> CapInScen [AMCINInScen] 7935.477957001622 degree = 6 indegree = 2 cond = 2
--- Directing AMCINInScen --> CapInScen
28. INSERT InsChange --> InsSclInScen [AMInsWliScen] 5100.860672891173 degree = 6 indegree = 2 cond = 2
--- Directing AMInsWliScen --> InsSclInScen
29. INSERT Scenario --> RHRatio [] 5024.198470412222 degree = 7 indegree = 2 cond = 1
30. INSERT Scenario --> LowLLapse [] 4395.0530244349575 degree = 8 indegree = 2 cond = 1
31. INSERT ScnRelPlFcst --> SynForcng [] 4258.72156103166 degree = 8 indegree = 2 cond = 1
32. INSERT ScnRelPlFcst --> MeanRH [] 4160.4474252851105 degree = 8 indegree = 2 cond = 1
33. INSERT CurPropConv --> LLIW [] 4155.38932550736 degree = 8 indegree = 2 cond = 1
34. INSERT LatestCIN --> CurPropConv [LLIW] 4051.9878197565013 degree = 8 indegree = 2 cond = 2
--- Directing LLIW --> CurPropConv
35. INSERT Scenario --> MidLLapse [] 3463.0283821636185 degree = 9 indegree = 2 cond = 1
36. INSERT OutflowFrMt --> Boundaries [] 3411.097146228003 degree = 9 indegree = 2 cond = 1
37. INSERT AreaMesoALS --> AreaMoDryAir [CombMoisture] 3155.7691316592245 degree = 9 indegree = 2 cond = 2
--- Directing CombMoisture --> AreaMoDryAir
38. INSERT WndHodograph --> OutflowFrMt [InsInMt] 3152.94192621985 degree = 9 indegree = 2 cond = 2
--- Directing InsInMt --> OutflowFrMt
39. INSERT WndHodograph --> CldShadeConv [InsInMt] 3053.342200996385 degree = 9 indegree = 2 cond = 2
--- Directing InsInMt --> CldShadeConv
40. INSERT ScnRelPlFcst --> WindFieldMt [] 2789.047194566061 degree = 9 indegree = 2 cond = 1
41. INSERT CombVerMo --> N07muVerMo [] 2731.467407400931 degree = 9 indegree = 2 cond = 1
42. INSERT SubjVertMo --> CombVerMo [N07muVerMo] 2655.020437775369 degree = 9 indegree = 2 cond = 2
--- Directing N07muVerMo --> CombVerMo
43. INSERT QGVertMotion --> CombVerMo [] 4117.928027072281 degree = 9 indegree = 2 cond = 3
44. INSERT IRCloudCover --> CombClouds [VISCloudCov] 2458.094871995503 degree = 9 indegree = 2 cond = 2
--- Directing VISCloudCov --> CombClouds
45. INSERT AreaMesoALS --> CldShadeOth [] 2191.0516760104747 degree = 9 indegree = 2 cond = 2
46. INSERT CombMoisture --> SatContMoist [] 2126.1178240752342 degree = 9 indegree = 2 cond = 1
47. INSERT RaoContMoist --> CombMoisture [SatContMoist] 2532.7527230040578 degree = 9 indegree = 2 cond = 2
--- Directing SatContMoist --> CombMoisture
48. INSERT MorningBound --> Boundaries [] 1895.3523960106195 degree = 9 indegree = 2 cond = 2
49. INSERT AMDewptCalPl --> AMInsWliScen [] 1726.9878538675002 degree = 9 indegree = 2 cond = 1
50. INSERT LIfr12ZDENSd --> AMInsWliScen [AMDewptCalPl] 1422.3951010527926 degree = 9 indegree = 2 cond = 2
--- Directing AMDewptCalPl --> AMInsWliScen
51. INSERT CapChange --> InsChange [LoLevMoistAd] 1196.3283128400344 degree = 9 indegree = 2 cond = 2
--- Directing LoLevMoistAd --> InsChange
52. INSERT CurPropConv --> PlainsFcst [] 849.0071934315165 degree = 9 indegree = 2 cond = 1
53. INSERT ScenRelAMCIN --> AMCINInScen [MorningCIN] 778.5394196887337 degree = 9 indegree = 2 cond = 2
--- Directing MorningCIN --> AMCINInScen
54. INSERT Date --> Scenario [] 727.7999368397141 degree = 10 indegree = 2 cond = 1
55. INSERT AreaMoDryAir --> CldShadeOth [] 703.1535649705329 degree = 10 indegree = 2 cond = 3
56. INSERT AreaMesoALS --> CapChange [] 700.5598957238835 degree = 10 indegree = 2 cond = 1
57. INSERT PlainsFcst --> ScnRelPlFcst [] 570.1579930226726 degree = 10 indegree = 2 cond = 1
58. INSERT InsSclInScen --> PlainsFcst [] 570.0031714512916 degree = 10 indegree = 2 cond = 2
59. INSERT CldShadeOth --> CapChange [] 479.3933521717736 degree = 10 indegree = 2 cond = 2
60. INSERT AMInsWliScen --> ScnRelPlFcst [] 416.7982150651951 degree = 10 indegree = 2 cond = 2
61. INSERT WndHodograph --> Boundaries [] 354.91872928021985 degree = 10 indegree = 2 cond = 3
62. INSERT ScenRel34 --> N34StarFcst [] 121.17752072456369 degree = 10 indegree = 2 cond = 2
63. INSERT CldShadeOth --> MountainFcst [] 63.76639495397103 degree = 10 indegree = 2 cond = 1
64. INSERT CldShadeOth --> InsInMt [] 148.14735435118564 degree = 10 indegree = 2 cond = 2
65. INSERT CldShadeConv --> CapChange [] 59.48265800724039 degree = 10 indegree = 2 cond = 3
66. INSERT ScenRelAMIns --> Scenario [] 45.67621526152453 degree = 11 indegree = 2 cond = 2
67. INSERT ScnRelPlFcst --> ScenRel34 [] 43.32892665240189 degree = 11 indegree = 2 cond = 2
68. INSERT ScnRelPlFcst --> ScenRelAMCIN [] 22.7365392031544 degree = 12 indegree = 2 cond = 2
67. DELETE CldShadeOth --> MountainFcst H = [] NaYX = [InsInMt] degree = 12 indegree = 2 diff = [InsInMt] (37.74207135744655)  cond = 1
68. INSERT CldShadeOth --> AMInstabMt [] 639.8455362967507 degree = 12 indegree = 2 cond = 2
67. DELETE CldShadeOth --> AMInstabMt H = [InsInMt] NaYX = [InsInMt] degree = 12 indegree = 2 diff = [] (23.54686309976387)  cond = 0
--- Directing InsInMt --- AMInstabMt to AMInstabMt --> InsInMt
Elapsed time = 1937.134 s

[jdramsey]

@yasu-sh I will give it a shot. Sorry, we've been focused lately on getting a new release of Tetrad and causal-cmd out the door so I haven't had a lot of time to think (other than that). I don't think the new release will solve this problem. However, I've got 64 GB of RAM on my laptop so maybe I can get further than you, I don't know. Anyway, we're supposed to get the new release out by today or maybe tomorrow so maybe after that, I'll have a little bit of time to address this.

[jdramsey]

@yasu-sh Could you zip and attach this file for me--would speed things up for me:

dt.hailfinder.csv

Thanks, or give me the link to download this. You may have given this to me before but I can't find it immediately.

[jdramsey]

Never mind, I found it in the thread. I've got a few minutes right now; let me try loading it up in the interface and searching...

[jdramsey]

Actually, I would remove columns 27 and 41 from the data; they have too many categories. That might be the problem. Are they needed? Without them, fGES finishes quite quickly.

[jdramsey]

Here's the dataset without these variables (tab-delimited)--see if causal-cmd can deal with this instead.

dt.hailfinder.subset.txt

My theory is that the problem is that these 11-category variables were causing the algorithm to make cross-classification tables that were just too big.

[jdramsey]

Maybe I'll think about adding a feature to automatically flag variables like this that have too many categories to deal with in a discrete setting.

[jdramsey]

Actually, the GRaSP algorithm (default settings) deals just fine with the full dataset; it's just FGES that doesn't, though if you remove the two variables from the data I suggested it does. I have to think about that. In any case, it seems GRaSP is a better strategy for dealing with the full data. I hadn't considered that.

So either:

• Run FGES on dt.hailfinder.subset.dt, or
• Run GRaSP on dt.hailfinder.csv

You get similar results either way, just for the GRaSP analysis with the two extra variables.

[jdramsey]

By the way, the way I discovered that columns 27 and 41 had too many categories was to use the mixed data loader with an increasing number of max discrete categories. Turns out that to load the full dataset as discrete you need to set max discrete categories to 11, which loads 27 and 41 as discrete, so those two columns each have 11 categories, which is quite a large number of categories to have for a discrete variable in such a large dataset.

[yasu-sh]

@jdramsey Thank you for investigating and telling me a lot about the root cause. The result of dataset - dt.hailfinder.subset.txt is as you mentioned. I will try next - GRaSP.

PS E:\Tetrad> java -Xmx25G -jar "causal-cmd-1.5.0-jar-with-dependencies.jar" --data-type discrete --delimiter tab --score bdeu-score --json-graph --algorithm fges --dataset dt.hailfinder.subset.txt --parallelized --verbose
1. INSERT AreaMesoALS --> CombVerMo [] 25244.402367736307 degree = 1 indegree = 1 cond = 1
2. INSERT CapChange --> CompPlFcst [] 21347.6368234152 degree = 1 indegree = 1 cond = 1
3. INSERT ScenRel34 --> ScenRelAMIns [] 14739.351494166422 degree = 1 indegree = 1 cond = 1
(CUT)
79. DELETE ScenRelAMCIN --> WindAloft H = [] NaYX = [] degree = 17 indegree = 2 diff = [] (23.695249037307804)  cond = 2
80. INSERT CldShadeOth --> AMInstabMt [] 639.8455362967507 degree = 17 indegree = 2 cond = 2
79. DELETE CldShadeOth --> AMInstabMt H = [InsInMt] NaYX = [InsInMt] degree = 17 indegree = 2 diff = [] (23.54686309976387)  cond = 0
--- Directing InsInMt --- AMInstabMt to AMInstabMt --> InsInMt
Elapsed time = 5.241 s

PS E:\Tetrad> java -Xmx25G -jar "causal-cmd-1.5.0-jar-with-dependencies.jar" --data-type discrete --delimiter tab --score bdeu-score --json-graph --algorithm fges --dataset dt.hailfinder.subset.txt --default
1. INSERT AreaMesoALS --> CombVerMo [] 25244.402367736307 degree = 1 indegree = 1 cond = 1
2. INSERT CapChange --> CompPlFcst [] 21347.6368234152 degree = 1 indegree = 1 cond = 1
3. INSERT ScenRel34 --> ScenRelAMIns [] 14739.351494166422 degree = 1 indegree = 1 cond = 1
(CUT)
79. DELETE ScenRelAMCIN --> WindAloft H = [] NaYX = [] degree = 17 indegree = 2 diff = [] (23.695249037307804)  cond = 2
80. INSERT CldShadeOth --> AMInstabMt [] 639.8455362967507 degree = 17 indegree = 2 cond = 2
79. DELETE CldShadeOth --> AMInstabMt H = [InsInMt] NaYX = [InsInMt] degree = 17 indegree = 2 diff = [] (23.54686309976387)  cond = 0
--- Directing InsInMt --- AMInstabMt to AMInstabMt --> InsInMt
Elapsed time = 6.168 s

Diffrence between 1.5.0 parallelized / default(not paralleled)

same edge findings with small fraction difference at scores

causal-cmd v1.2.0 - r-causal internal

PS E:\Tetrad> Measure-Command {java -Xmx25G -jar "causal-cmd-1.2.0-SNAPSHOT-jar-with-dependencies.jar" --data-type discrete --delimiter tab --score bdeu-score --json-graph --algorithm fges --dataset dt.hailfinder.subset.txt --verbose --skip-latest }
TotalSeconds      : 6.3909959
TotalMilliseconds : 6390.9959

[yasu-sh]

This is the verbose of GRaSP with g-square-test. It has moderate speed to find good result. (It makes better results than fges, doesn't it? I hav'nt tried comparing yet..)

PS E:\Tetrad> java -Xmx25G -jar "causal-cmd-1.5.0-jar-with-dependencies.jar" --data-type discrete --delimiter comma --score bdeu-score --algorithm grasp --dataset dt.hailfinder.csv --test g-square-test --verbose
Edges: 115      |        Score Improvement: 21.358659   |        Tucks Performed: [] [PlainsFcst, ScenRel34]
Edges: 117      |        Score Improvement: 29.930053   |        Tucks Performed: [] [MorningBound, Boundaries]
Edges: 118      |        Score Improvement: 44.847998   |        Tucks Performed: [] [Dewpoints, WindAloft]
Edges: 117      |        Score Improvement: 37.145838   |        Tucks Performed: [] [SatContMoist, AreaMoDryAir]
(CUT)
Edges: 77       |        Score Improvement: 22.893827   |        Tucks Performed: [[MvmtFeatures, Scenario], [ScnRelPlFcst, MidLLapse], [Dewpoints, ScnRelPlFcst]] [InsChange, InsSclInScen]
Edges: 77       |        Score Improvement: 0.000000    |        Tucks Performed: [[LatestCIN, LLIW], [Dewpoints, ScnRelPlFcst]] [ScnRelPlFcst, Date]
Edges: 77       |        Score Improvement: 43.642965   |        Tucks Performed: [] [InsChange, CapChange]
Edges: 76       |        Score Improvement: 27.641958   |        Tucks Performed: [[LoLevMoistAd, CompPlFcst], [N07muVerMo, SubjVertMo]] [InsChange, LoLevMoistAd]
# Edges = 76 Score = -988900.8697642368 (GRaSP) Elapsed 548.648 s
Final order = [Date, ScnRelPlFcst, Dewpoints, MidLLapse, WindFieldPln, WindFieldMt, TempDis, ScenRel34, MvmtFeatures, (CUT)
N07muVerMo, R5Fcst, LIfr12ZDENSd]
Elapsed time = 551.008 s

[jdramsey]

Oh that's awesome! Yeah in general GRaSP is doing better than fGES. I can send you the paper if you like.

[jdramsey]

You know, partly because of your comparisons and partly because I just got a paper to review that uses it, I'm thinking of finally trying to update py-causal to the latest Tetrad. it's not my code, so I have to think about how to do it, but the code is not that complicated in Python, and I know where everything is located currently in Tetrad, so maybe I can do it.

By the way here's the GRaSP paper: https://arxiv.org/abs/2206.05421

[jdramsey]

Oh I see the problem people are having... it's so hard to install javabridge on a recent Mac. This issue.

https://stackoverflow.com/questions/74793639/unable-to-install-javabridge-with-pip-on-macos-monterey-m1

Maybe I'll get a Linux box at least and try again.

[jdramsey]

Hm. I think if I had a machine where I could easily install Javabridge (I think linux would be good) I could easily update the version of causal-cmd in py-causal.

[yasu-sh]

@jdramsey Congratulations on the release of tetrad-v7.2.0! Python has become more popular than other languages among ML programmers...

I chose FGES since it looked runnging very fast and perfomance was stable on time and result in tetrad. This time I understand the traditional algorithm also chages. Let me consider on the matter later on.

[yasu-sh]

@jdramsey Thanks for your inform me about your consideration! I will move onto using GRaSP since the FGES won't be working well at latest versions of Tetrad.

(--default option is used if possible, --json-graph option is not used for no additional errors)

Execution code example PS > (Measure-Command { java -Xmx25G -jar ".\causal-cmd-1.4.1-jar-with-dependencies.jar" --data-type discrete --delimiter comma --algorithm fges --score bdeu-score --dataset dt.hailfinder.csv --verbose --default | Out-Default }).TotalSeconds

data: hailfinder.csv (incl. 11 categories)

version	release	Process duration(sec)
		FGES	GRaSP
1.1.3	2019-11-02	3.4	NA(before dev.)
1.2.0-SNAPSHOT	(2019-12-26)	9.2	NA
1.2.0	2020-11-11	7.1	NA
1.2.1	2020-12-07	7.7	NA
1.2.2	2021-02-22	2813.6	NA
1.3.0	2021-05-05	2912.0	NA
1.4.0	2022-06-16	-	-(initial)
1.4.1	2022-07-11	4156.7	5.5
1.4.2	2022-12-14	-	-
1.5.0	2023-01-23	4843.1	5.7
1.6.0	2023-02-16	Next measurement	9.2
1.6.1	2023-02-22	over 12 hours	8 .0

[jdramsey]

Hi Yasuhiro,

Sorry, I got busy for a few days. I think GRaSP is a good option, but let me look into this now. (sorry, I didn't get much sleep.)

Let me go back through the messages and think.

On Wed, Mar 1, 2023 at 4:00 AM Yasuhiro Shimodaira @.***> wrote:

@jdramsey https://github.com/jdramsey Thanks for your inform me about the your consideration! I will move onto using GRaSP since the FGES won't be working well at latest versions of Tetrad. (default is used when possible) data: hailfinder.csv (incl. 11 categories) version release Process duration(sec) FGES GRaSP 1.1.3 2019-11-02 3.4 x(before dev.) 1.2.0-SNAPSHOT (2019-12-26) 9.2 x 1.2.0 2020-11-11 7.1 x 1.2.1 2020-12-07 7.7 x 1.2.2 2021-02-22 2813.6 x 1.3.0 2021-05-05 2912 x 1.4.0 2022-06-16 - -(initial) 1.4.1 2022-07-11 Next measurement 5.5 1.4.2 2022-12-14 - - 1.5.0 2023-01-23 4843.1 5.7 1.6.0 2023-02-16 Next measurement 9.2 1.6.1 2023-02-22 over 10 hours 8

— Reply to this email directly, view it on GitHub https://github.com/bd2kccd/causal-cmd/issues/83#issuecomment-1449617189, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACLFSR2QMONAAYFFXXV3VS3WZ4FZRANCNFSM6AAAAAAUNPQSLU . You are receiving this because you were mentioned.Message ID: @.***>

[jdramsey]

Oh sorry, you closed the issue! You know, we have been thinking about doing Tetrad in Python, just the last few days actually, in earnest, since we've been having so much trouble with Javabridge and py-causal on our end. In fact, we've tried a totally different way of doing it, using the Python project JPype. we made a project (just. few days old at this point) that seems to solve the problem completely--

https://github.com/cmu-phil/py-tetrad

This is still new, so there may be issues with it, but JPype works beautifully and is very easy to install on Mac (and we just tried Windows yesterday!)

Maybe try installing it according to the instructions? I could help you move your code over if you'd like.

[jdramsey]

We set up py-tetrad to use the latest Tetrad 7.2.2, but you obviously could figure out how to get it to use other versions if you just look at the code. It's really a very think wrapper on Tetrad. The only issue from your point of view is that you need to use a version of Java > 1.8 and you need to use a version of Python >= 3.7, so it may not be possible to use versions of Tetrad from a very long time ago.

[jdramsey]

What happened was that we had spent so much time fixing issues in Tetrad and finally had a version (7.2.2) that seemed to have fixed all outstanding reported bugs...so then we stopped and had a meeting and decided the next thing to do was to make it easier to deal with Tetrad from Python... so Bryan Andrews and I started looking around at recent methods for doing this and found JPype, which is just beautiful.

[yasu-sh]

@jdramsey I appreciate your proposals and I will check the instruction of py-tetrad. I thought it looked nice since tetrad itself can be wrapped.

[jdramsey]

I agree--there is really no separation between Python and java using this method. You just have to get used to writing Java "in Python" using Python syntax; JPype handles all of the details for you. It "marries" the Python and Java virtual machines to make this happen.

There are few details. You have to start the JVM from Python as before, and you can only do it once per session. Also, like I said you need to use >= 3.7 version for Python (since all underlying reference to Python 2 needs to be eliminated from the underlying code). A few other things.

[yasu-sh]

My experience on analytics is mainly on R. But some years ago I used Python more often. Tetrad software has useful GUI. I need to make not R interface but Python, it would be possible and need consideration. Anyway, thanks for your offer. Most of my colleagues are using Python. No problem I can ask them.

[jdramsey]

Thanks!

[jdramsey]

I'd like to revisit R as well at some point, and see if there aren't more up-to-date ways of connecting R to Java. The main impediment is I would have to find time to do that myself; there's no one else in our group that does R. But I will try to get to it.