Unable to specify random seed for sampling in Bootstrapping?

[yasu-sh]

Dear developers, @jdramsey, @kvb2univpitt

Several months ago I made a request on the bootstrapping in: https://github.com/bd2kccd/causal-cmd/issues/80

I am grateful for the added functionality. But recently I have noticed that even when setting a seed and executing with bootstrapping, I cannot obtain reproducible results in causal-cmd v. 1.11.

I attached two log files, both analyzing the same data with the same parameters. If there are any mistakes how to set seed or other parameters, please let me know. If not, we would be appreciated if you could check whether the code generates randomness under the seed setting.

Hopefully the latest version(causal-cmd v.1.12 or above) is available on Java 11 since our environment is limited. (if you updated the version from 8 to 17 and skipped Java 11 since there were no particular reasons for Java 17.

• Java(TM) SE Runtime Environment 18.9 (build 11.0.18+9-LTS-195)
• Windows 11 Pro (Intel)
• Memory 64 GB

tetrad_240308-111141.log tetrad_240308-111250.log

[kvb2univpitt]

@yasu-sh Thanks for reporting the bug. I'll take a look at it. This should not happen. If it's not working in causal-cmd then it will not work in tetrad, too. I'll get back to you on this.

[jdramsey]

@yasu-sh I'm just curious: What are the reasons you can't move to 17? I'm just curious. We have people who want us even to move to 21 and stay up-to-date with Java versions... it's so hard to find a good version...

[yasu-sh]

@jdramsey It's natural for you to have the quesion. We have a common data analysis system(Server side) which runs on Java 11. I cannot get colleagues to change the Java system since they have already checked on Java 11 in past. I have not recognized the Server side Java version is still 11 until recent days. For development purpose I have already added Java 17 on my PC.

My idea is as below: causal-cmd(and tetrad-lib): Java 11 or above tetrad(tetrad-gui, reader): Java 17 or above

Let me know if you notice this idea does not work.

[yasu-sh]

@kvb2univpitt @jdramsey Regarding Java 11 compatibilities, if you have some difficulties to enable some new technical ideas by using Java 17, I can make actions to find countermeasures for our side, including Java 17 installation.

I am appreciated if you focus on confirming the results be reproductive. It is important. Java versions issues are trivial, compared to that.

[kvb2univpitt]

@yasu-sh We decided to stick with Java 17 and move to Java 21 in the future. The latest version of Java provides better security and performances. We take advantages of some of the new features in each Java release for better performances and code readability. For an example, we replaced some Java Classes with Java Records for specific use. We are planning to use virtual threads instead of platform thread in the future when we switch over to Java 21 LTS

If your case, you can download and extract the zip or tar file version of OpenJDK17 to your home directory. Instead of typing java -jar ..., you can call the java command directly from your local OpenJDK17 directory /home/user/jdk-17.0.10+7/bin/java -jar ... . I think this should work.

[yasu-sh]

@kvb2univpitt Thanks for telling on the future direction. In our case, we need to use Java on the common account(Server side specific), not local account directory.

It is complicated since causal-cmd is called by R. But we are finidng a countermeasure now like you mentioned in windows system.

[yasu-sh]

@kvb2univpitt @jdramsey I have found a workaround for using tetrad 1.12 on Java 17(Corretto 17 zip-extracted on windows) at our server. Could you tell me if you could found a solution or some temporal findings on this issue: random seeds?

If you need more information, let me know.

[jdramsey]

@yasu-sh I'll coordinate with Kevin on this. It's possible I broke something, but I'm not sure how to tell quite yet.

[kvb2univpitt]

@jdramsey @yasu-sh This is definitely a Tetrad issue. I ran Tetrad version 7.6.1 with FGES using the same bootstrap parameters and got a different results each time I run it. Causal-cmd is just a wrapper around Tetrad. If Tetrad is broken, causal-cmd would be broken, too.

[yasu-sh]

@kvb2univpitt @jdramsey Thanks for your re-confirmation on Tetrad also. It seems to mean that the arguments for search algortihms with bootstrapping work properly in causal-cmd.

I might create another issue on Tetrad.

[jdramsey]

@yasu-sh Kevin and I looked at this yesterday (I think). it's a problem in 7.6.1 but not in the development branch. Hmm.

[jdramsey]

By the way I believe we're almost ready to publish a new version of Tetrad. I'll double check to make sure the bootstrapping issue works in it.

[yasu-sh]

@jdramsey Thanks a lot! I hope it was just a version-specific issue.