verify-fingerprints fails

[hockeyrob]

This is an onboarding issue I have Zowe 2.0.0 installed. Running on a z/OS V2R5 system. zwe support verify-fingerprints fails. I try connecting to OMVS with ssh. I get the java 'not-enough-storage' barrage. "ulimit -M" shows I have 512M above the bar...although it should be "unlimited", according to our PARMLIB settings. I try the same from TSO OMVS. Here, "unlimit -M" says I have 10G above the bar....but the result is exactly the same. Not enough storage.

[jackjia-ibm]

zwe support verify-fingerprints requires java to function properly to calculate hashes. @hockeyrob could you please provide zwe support verify-fingerprints -vv log so we may see more details how it fails? Thanks.

[hockeyrob]

This is the beginning of the processing. Identifying file names and such redacted, a little:

SERVER:/usr/lpp/zowe> zwe support verify-fingerprints -vv

VERIFY ZOWE FILE FINGERPRINTS

• Create Zowe directory file list

  create_tmp_file on /tmp

  • test /tmp/zwe-support-verify-fingerprints-21584
  • good
  • File list created as /tmp/zwe-support-verify-fingerprints-21584

• Calculate hashes of Zowe files

  create_tmp_file on /tmp

  • test /tmp/zwe-support-verify-fingerprints-21584
  • test /tmp/zwe-support-verify-fingerprints-21584
  • test /tmp/zwe-support-verify-fingerprints-20890
  • good JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2022/07/29 10:40:05 - please wait. JVMDUMP032I JVM requested System dump using 'ZOS.JVM.DATASET.D220729.T104005.X&DS' in response to an event IEATDUMP in progress with options SDATA=(LPA,GRSQ,LSQA,NUC,PSA,RGN,SQA,SUM,SWA,TRT) IEATDUMP success for DSN='ZOS.JVM.DATASET.D220729.T104005.X&DS' JVMDUMP040I System dump written to dataset(s) using name template ZOS.JVM.DATASET.D220729.T104005.X&DS JVMDUMP030W Cannot write dump to file /usr/lpp/zowe/heapdump.20220729.104005.83886325.0002.phd: EDC5141I Read-only file system. (errno2=0x05620060) JVMDUMP032I JVM requested Heap dump using '/tmp/heapdump.20220729.104005.83886325.0002.phd' in response to an event JVMDUMP010I Heap dump written to /tmp/heapdump.20220729.104005.83886325.0002.phd JVMDUMP030W Cannot write dump to file /usr/lpp/zowe/javacore.20220729.104005.83886325.0003.txt: EDC5141I Read-only file system. (errno2=0x05620060) JVMDUMP032I JVM requested Java dump using '/tmp/javacore.20220729.104005.83886325.0003.txt' in response to an event JVMDUMP010I Java dump written to /tmp/javacore.20220729.104005.83886325.0003.txt JVMDUMP032I JVM requested Snap dump using '/usr/lpp/zowe/Snap.20220729.104005.83886325.0004.trc' in response to an event JVMDUMP030W Cannot write dump to file /usr/lpp/zowe/Snap.20220729.104005.83886325.0004.trc: EDC5141I Read-only file system. (errno2=0x05620060) JVMDUMP010I Snap dump written to /tmp/Snap.20220729.104005.83886325.0004.trc JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError".

At this point I had to stop it before it started listing all the files under /usr/lpp/zowe...because there are some 11900 of them. Since I interrupted it, I was able to see the two /tmp/zwe-support-verify-fingerprint-* temp files, one of which was empty, and the other contained the list of 11952 files.

JAVA_HOME is set to /usr/lpp/java/current_64, which: SERVER:/usr/lpp/zowe> ls -lA $JAVA_HOME lrwxrwxrwx 1 BPXROOT OMVSGRP 9 May 3 19:25 /usr/lpp/java/current_64@ -> ./J8.0_64 SERVER:/usr/lpp/zowe>

We are running this on z/OS V2R5 at the latest RSU; this is the Java release that came with it.

[hockeyrob]

Interesting how Git reformatted the output; i guess it's still pretty clear what it was supposed to be.

[hockeyrob]

Upgraded to Zowe V2.2, with no change.

[Martin-Zeithaml]

When I have limitted the -Xmx to low values, I was getting JVMDUMP055I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError", exception "Java heap space" at 2024/08/08 08:08:08 - please wait.....

It seems the zwe support verify-fingerprints command is ok and this issue is about the java setting.

However in such case, the processing should stop here. But is continues due to:

• Crashing java program returns 0
• Output file is empty, but we are checking if file exist

[JoeNemo]

The java crash part of this bug is not really a Zowe issue. The rest of this has been fixed.