JRE "Out of Memory Error" when running doctests

[strogdon]

I have lots of Jave Runtime errors when testing sog 9.3.beta5. They were not present, as far as I know, with 9.3.beta4. I also have this with vanilla and it's been present there for some time. I hadn't noticed it because there was no apparent failure. In any event this is fairly new. The head of hs_err_pidxxxxx.log

#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (malloc) failed to allocate 32744 bytes for ChunkPool::allocate
# Possible reasons:
#   The system is out of physical RAM or swap space
#   The process is running with CompressedOops enabled, and the Java Heap may be blocking the growth of the native heap
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
#   JVM is running with Unscaled Compressed Oops mode in which the Java heap is
#     placed in the first 4GB address space. The Java Heap base address is the
#     maximum limit for the native heap growth. Please use -XX:HeapBaseMinAddress
#     to set the Java Heap base and to place the Java Heap above 4GB virtual address.
# This output file may be truncated or incomplete.
#
#  Out of Memory Error (allocation.cpp:272), pid=10038, tid=0x00007fac05265640
#
# JRE version: OpenJDK Runtime Environment (8.0_272-b10) (build 1.8.0_272-b10)
# Java VM: OpenJDK 64-Bit Server VM (25.272-b10 mixed mode linux-amd64 compressed oops)
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#

[strogdon]

I do not have systemd installed. So ${PORTAGE_BUILDDIR}/homedir should be something like /var/tmp/portage/sci-mathematics/sage-9999/work/sage-9999/src-python3_9/homedir? If so, I see no homedir. I still have the work folder available.

[kiwifb]

/var/tmp/portage/sci-mathematics/sage-9999/homedir actually.

[strogdon]

Ah yes, of course. However, that folder here contains no .java stuff.

# ls -al /var/tmp/portage/sci-mathematics/sage-9999/homedir
total 20
drwxr-xr-x 5 portage portage 4096 Jan 29 12:22 .
drwx------ 8 portage portage 4096 Jan 29 12:24 ..
drwxr-xr-x 3 portage portage 4096 Jan 29 11:23 .gap
drwxr-xr-x 3 portage portage 4096 Jan 29 11:22 .maxima
drwxr-xr-x 6 portage portage 4096 Jan 29 11:23 .sage

[kiwifb]

Anything in /var/lib/portage/home/?

[strogdon]

Just the hidden empty file .keep_acct-user_portage-0. I had, at one time, move it out of the way to see if there was a difference, but didn't notice anything.

[kiwifb]

Let's just try something different what is the output of eselect fontconfig list?

[strogdon]

OK. From temp/environment

declare -x SANDBOX_DEBUG="0"
declare -x SANDBOX_DENY=""
declare -x SANDBOX_PREDICT="/var/tmp/portage/sci-mathematics/sage-9999/homedir:/dev/crypto:/var/cache/man:/proc/self/coredump_filter:/dev/random:/proc/self/coredump_filter:/dev/random:/proc/self/coredump_filter:/var/cache/fontconfig:/usr/lib/python3.8:/usr/lib64/python3.8:/usr/lib/pypy2.7:/usr/lib/pypy3.6:/usr/lib/portage/pym:/usr/local:/usr/lib/python3.9:/usr/lib64/python3.9:/usr/lib/pypy2.7:/usr/lib/pypy3.6:/usr/lib/portage/pym:/usr/local"
declare -x SANDBOX_READ="/:/var/tmp/portage"
declare -x SANDBOX_VERBOSE="1"
declare -x SANDBOX_WRITE=":/dev/console:/dev/fd:/dev/full:/dev/null:/dev/ptmx:/dev/pts/:/dev/pty:/dev/shm:/dev/tts:/dev/tty:/dev/vc/:/dev/zero:/proc/self/fd:/tmp/:/usr/lib/cf:/usr/lib/conftest:/usr/lib32/cf:/usr/lib32/conftest:/usr/lib64/cf:/usr/lib64/conftest:/usr/portage/distfiles/git3-src:/usr/tmp/cf:/usr/tmp/conftest:/var/tmp/:/var/tmp/portage:/var/tmp/portage/sci-mathematics/sage-9999/homedir/.bash_history"

And a .java folder exists under /var/tmp/portage

# ls -al /var/tmp/portage/.java/fonts/
total 28
drwxr-xr-x 7 portage portage 4096 Jan 29 18:22 .
drwxr-xr-x 3 portage portage 4096 Aug 25  2017 ..
drwxr-xr-x 2 portage portage 4096 Apr 11  2018 1.8.0_144
drwxr-xr-x 2 portage portage 4096 Aug  2 22:15 1.8.0_172
drwxr-xr-x 2 portage portage 4096 Aug 10 20:06 1.8.0_252
drwxr-xr-x 2 portage portage 4096 Oct 19 00:49 1.8.0_265
drwxr-xr-x 2 portage portage 4096 Jan 29 18:23 1.8.0_272

[strogdon]

eselect fontconfig list on Gentoo

  [1]   09-texlive.conf *
  [2]   10-autohint.conf *
  [3]   10-hinting-full.conf *
  [4]   10-hinting-medium.conf *
  [5]   10-hinting-none.conf *
  [6]   10-hinting-slight.conf *
  [7]   10-no-sub-pixel.conf
  [8]   10-scale-bitmap-fonts.conf *
  [9]   10-sub-pixel-bgr.conf *
  [10]  10-sub-pixel-rgb.conf *
  [11]  10-sub-pixel-vbgr.conf *
  [12]  10-sub-pixel-vrgb.conf *
  [13]  10-unhinted.conf *
  [14]  11-lcdfilter-default.conf *
  [15]  11-lcdfilter-legacy.conf *
  [16]  11-lcdfilter-light.conf *
  [17]  20-unhint-small-dejavu-sans.conf
  [18]  20-unhint-small-dejavu-sans-mono.conf
  [19]  20-unhint-small-dejavu-serif.conf
  [20]  20-unhint-small-vera.conf *
  [21]  25-unhint-nonlatin.conf
  [22]  30-metric-aliases.conf *
  [23]  40-nonlatin.conf *
  [24]  42-luxi-mono.conf *
  [25]  45-generic.conf *
  [26]  45-latin.conf *
  [27]  49-sansserif.conf *
  [28]  50-user.conf *
  [29]  51-local.conf *
  [30]  57-dejavu-sans.conf
  [31]  57-dejavu-sans-mono.conf
  [32]  57-dejavu-serif.conf
  [33]  60-generic.conf *
  [34]  60-latin.conf *
  [35]  60-liberation.conf *
  [36]  61-stix.conf *
  [37]  65-fonts-persian.conf *
  [38]  65-khmer.conf *
  [39]  65-nonlatin.conf *
  [40]  69-unifont.conf *
  [41]  70-no-bitmaps.conf
  [42]  70-yes-bitmaps.conf
  [43]  80-delicious.conf *
  [44]  90-synthetic.conf *

[strogdon]

eselect fontconfig list on Prefix

  [1]   09-texlive.conf
  [2]   10-autohint.conf
  [3]   10-hinting-full.conf
  [4]   10-hinting-medium.conf
  [5]   10-hinting-none.conf
  [6]   10-hinting-slight.conf *
  [7]   10-no-sub-pixel.conf
  [8]   10-scale-bitmap-fonts.conf *
  [9]   10-sub-pixel-bgr.conf
  [10]  10-sub-pixel-rgb.conf
  [11]  10-sub-pixel-vbgr.conf
  [12]  10-sub-pixel-vrgb.conf
  [13]  10-unhinted.conf
  [14]  11-lcdfilter-default.conf
  [15]  11-lcdfilter-legacy.conf
  [16]  11-lcdfilter-light.conf
  [17]  20-unhint-small-dejavu-sans.conf
  [18]  20-unhint-small-dejavu-sans-mono.conf
  [19]  20-unhint-small-dejavu-serif.conf
  [20]  20-unhint-small-vera.conf *
  [21]  25-unhint-nonlatin.conf
  [22]  30-metric-aliases.conf *
  [23]  40-nonlatin.conf *
  [24]  45-generic.conf *
  [25]  45-latin.conf *
  [26]  49-sansserif.conf *
  [27]  50-user.conf *
  [28]  51-local.conf *
  [29]  57-dejavu-sans.conf
  [30]  57-dejavu-sans-mono.conf
  [31]  57-dejavu-serif.conf
  [32]  60-generic.conf *
  [33]  60-latin.conf *
  [34]  61-stix.conf
  [35]  65-fonts-persian.conf *
  [36]  65-khmer.conf
  [37]  65-nonlatin.conf *
  [38]  69-unifont.conf *
  [39]  70-no-bitmaps.conf
  [40]  70-yes-bitmaps.conf
  [41]  80-delicious.conf *
  [42]  90-synthetic.conf *

[kiwifb]

/var/tmp/portage/.java now that's different. fonconfig is a wild shot but who knows I may end up finding something in there.

I believe it works when I build as a user because somehow I have this

ll .java/fonts/1.8.0_272/
total 160K
drwxr-xr-x  2 fbissey fbissey 4.0K Jan 22 17:20  .
drwxr-xr-x 20 fbissey fbissey 4.0K Jan 22 17:20  ..
-rw-------  1 fbissey fbissey 152K Jan 22 17:20 'fcinfo-1-moonloop-"Gentoo"-null-en.properties'

[strogdon]

Can you change the protections and ownership to see if that works? For reference I have

ls -al .java/fonts/1.8.0_272/
total 88
drwxr-xr-x 2 portage portage  4096 Jan 29 18:32 .
drwxr-xr-x 7 portage portage  4096 Jan 29 18:22 ..
-rw------- 1 portage portage 79238 Dec  7 15:18 fcinfo-1-localhost-Linux-5.4.38-gentoo-en.properties

Timestamps seem a bit weird.

[kiwifb]

Yes I am trying to put it inside the portage user to see what happens.

[kiwifb]

Nope. Trying to write temporary file in the location.

[strogdon]

I don't know how Prefix deals with the java font issue, but I have rebuilt s-o-g and no .java folder is created anywhere. I have searched the entire Prefix tree.

[kiwifb]

I have patched around my issue and I am hopeful that it will be short lived as we move away from jmol.

[strogdon]

It appears, unless I read things incorrectly, that going forward (your forcejavatmp.patch) jmol with write temp stuff to .sage. If so, any idea where this is now being done? It seems that jmol is somewhat inconsistent. I'm just curious.

[kiwifb]

During build that would go in /var/tmp/portage/sci-mathematics/sage-9999/homedir/.sage. I could have made it HOME one way or another.

[strogdon]

OK, on Prefix with your patch a fcinfo-1-... file is written to homedir/.sage/.java/fonts/1.8.0_272 which was not there previously. Apparently, not needed on vanilla or Prefix where sandbox may be vaguely defined.

[strogdon]

I'm not sure what component generates the fcinfo-1-... file but on Prefix the generation is not quite correct in that some fonts from the host appear in the file. I'm wondering whether this file is ever used.

[kiwifb]

It is supposed to be an index of fonts. But that's not obviously used by jmol when building the doc. I don't know that jmol creates it, all I know is plenty of .tmp are created and destroyed in the same folder during doc building.

[strogdon]

The .tmp files are interesting. Your patch can be applied to vanilla sage. The only file I see that is created is ~/.sage/.java/fonts/1.8.0_272/fcinfo-1-hp-envy-Linux-5.4.38-gentoo-en.properties. During building the html-docs if I delete the file it is immediately recreated. But no .tmp files. And I have watched in ~/.sage, ~/.sage/.java, ~/.sage/.java/fonts and in ~/.sage/.java/fonts/1.8.0_272 during the build. But of course this does not use portage.

[strogdon]

There are a bunch of folders created under ~/.sage/temp/'computer-name'/ that contain numerous tmp_* files, some of which are .pngs.

[kiwifb]

I think we should close this now.