Infinite Recursion in RinitJVM_with_padding

[lukeaanderso]

Hi,

Thank you in advance for your time. I recently have had issues with rJava hanging when calling .jinit, very similar to what was linked here https://github.com/NixOS/nixpkgs/pull/171597 and https://github.com/s-u/rJava/issues/272. I pulled down the repository, rebuilt with debugging and started looking at RJAVA_JVM_STACK_WORKAROUND. For context, the baseline hang looks like the below with debugging enabled

> require(rJava)
Loading required package: rJava
> .jinit()
JSW workaround: (level 4)
RinitJVM(non-threaded): cannot disable guard pages
  RLIMIT_STACK (rlimsize) 8388608
  R_CStackStart 0x7ffd6199b000
  R_CStackLimit 7969177
  maxBound 0x7ffd6119b000
  oldBound 0x7ffd6119afff

{must now be killed}

and noticed that forcing RJAVA_JVM_STACK_WORKAROUND=2 works, at the expense of reducing the stack, as noted in the warning. I started debugging by adding print statements within RinitJVM_with_padding

diff --git a/src/init.c b/src/init.c
index 6805d71..5e6b23e 100644
--- a/src/init.c
+++ b/src/init.c
@@ -576,6 +576,10 @@ static SEXP RinitJVM_with_padding(SEXP par, intptr_t padding, char *last) {
     /* reduce the risk that dummy will be optimized out */
   dummy[0] = (char) (uintptr_t) &dummy;
   padding -= (last - dummy) * R_CStackDir;
+  //_dbg(rjprintf("  LA: padding %d\n",  padding));
+  //_dbg(rjprintf("  LA: last %d\n",  last));
+  //_dbg(rjprintf("  LA: dummy %d\n",  dummy));
+
   if (padding <= 0)
     return RinitJVM_real(par, 0);
   else

With just padding being printed, the code stays in an infinite loop

LA: padding 2097041
LA: padding 2097042
LA: padding 2097043
LA: padding 2097044
LA: padding 2097045
...

However when I print last or dummy the loop breaks.

...
  LA: padding 307
  LA: last 1425866431
  LA: dummy 1425866367
  LA: padding 243
  LA: last 1425866367
  LA: dummy 1425866303
  LA: padding 179
  LA: last 1425866303
  LA: dummy 1425866239
  LA: padding 115
  LA: last 1425866239
  LA: dummy 1425866175
  LA: padding 51
  LA: last 1425866175
  LA: dummy 1425866111
  LA: padding -13
  LA: last 1425866111
  LA: dummy 1425866047
RinitJVM(non-threaded): initJVM returned 0
JSW workaround (ctd): (level 3)
  newBound 0x7ffe549d8fff
  new R_CStackLimit 7957504
RcallMethod (env=27389f8):

This suggests the compiler is optimizing away these variables. I can either set the environment variable to 2, or maybe add some hack to my local install to try and help the compiler not remove the variable. Do you have any suggestions? On an older compiler I am not seeing these issues. For context here is my Makevars and gcc/g++ versions

cat ~/.R/Makevars
CC  = /bin/gcc
CXX = /bin/g++
CFLAGS   = -O2 -march=native
CXXFLAGS = -O2 -march=native

/bin/gcc --version
gcc (GCC) 11.2.1 20220127 (Red Hat 11.2.1-9)
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

g++ --version
g++ (GCC) 11.2.1 20220127 (Red Hat 11.2.1-9)
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

R --version
R version 4.2.1 (2022-06-23) -- "Funny-Looking Kid"
Copyright (C) 2022 The R Foundation for Statistical Computing
Platform: x86_64-pc-linux-gnu (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under the terms of the
GNU General Public License versions 2 or 3.
For more information about these matters see
https://www.gnu.org/licenses/.

Thank you again for your time

Luke Anderson

[s-u]

@lukeaanderso thanks. I assume this is in some old JDK < 10 since that code should only be used for legacy JVMs. I can replicate it on Debian testing with gcc 12 and JDK 8. Apparently the compiler does tail call optimization (TCO) and thus completely destroys our stack detection attempts. Please test rJava 1.0-8, by making dummy volatile I attempted to convince gcc to avoid TCO - seems to work at least for now.

[lukeaanderso]

Confirmed this works for me. Thanks for looking quickly.

[s-u]

Thanks!

[patientyesterday10]

@s-u this is great; any chance the latest version will be pushed to CRAN in the near future?

[HenrikBengtsson]

@s-u this is great; any chance the latest version will be pushed to CRAN in the near future?

I second this, please push the fixed version of rJava to CRAN, especially since this must affect anyone on Ubuntu 22.04, which comes with GCC 11 and Java 1.8 by default.

Details:

I can confirm that this affects also systems with GCC 11 and Java 1.8, e.g. https://github.com/HenrikBengtsson/CBI-software/issues/61. I now ran into the problem with Ubuntu 22.04, which comes with these version by default:

$ java -version
openjdk version "1.8.0_382"
OpenJDK Runtime Environment (build 1.8.0_382-8u382-ga-1~22.04.1-b05)
OpenJDK 64-Bit Server VM (build 25.382-b05, mixed mode)

$ gcc --version
gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Copyright (C) 2021 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Installing:

install.packages("https://rforge.net/rJava/snapshot/rJava_1.0-8.tar.gz")

works around the problem.

[HenrikBengtsson]

rJava 1.0-10 is on CRAN as of 2023-12-02. I can confirm that this updated version solves the problem reported here.

Installation of rJava now works and

> rJava::.jinit()
> 

no longer blocks. I've confirmed this with Java JDK 1.8;

$ R CMD config --all | grep -E "^(JAVA|JAR)"
JAVA = /usr/bin/java
JAVAC = /usr/bin/javac
JAVAH = /usr/bin/javah
JAR = /usr/bin/jar
JAVA_HOME = /usr/lib/jvm/java-8-openjdk-amd64/jre
JAVA_LIBS = -L/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server -ljvm
JAVA_CPPFLAGS = -I/usr/lib/jvm/java-8-openjdk-amd64/jre/../include -I/usr/lib/jvm/java-8-openjdk-amd64/jre/../include/linux

and

$ gcc --version
gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0