eclipse-openj9 / openj9

Eclipse OpenJ9: A Java Virtual Machine for OpenJDK that's optimized for small footprint, fast start-up, and high throughput. Builds on Eclipse OMR (https://github.com/eclipse/omr) and combines with the Extensions for OpenJDK for OpenJ9 repo.
Other
3.28k stars 722 forks source link

JDK8 Segmentation error: vmState=0x000514ff #15311

Closed connglli closed 2 years ago

connglli commented 2 years ago

Java -version output

openjdk version "1.8.0_342-internal"
OpenJDK Runtime Environment (build 1.8.0_342-internal-_2022_06_10_15_18-b00)
Eclipse OpenJ9 VM (build master-3d06b2f9c, JRE 1.8.0 Linux amd64-64-Bit Compressed References 20220610_000000 (JIT enabled, AOT enabled)
OpenJ9   - 3d06b2f9c
OMR      - cf8ddbd1a
JCL      - 2bb179375a based on jdk8u342-b05)

Summary of problem

The following test case crashes OpenJ9's JIT compiler

$ cat Test.java
import java.text.DecimalFormat;
import java.text.DecimalFormatSymbols;
import java.text.NumberFormat;

class Test {
  void mainTest(String[] strArr1) {
    int i30;
    for (i30 = 2; 46 > i30; i30++) {
      DecimalFormat ax$25 = (DecimalFormat) NumberFormat.getInstance();
      DecimalFormatSymbols ax$24 = ax$25.getDecimalFormatSymbols();
      if (ax$24.getPatternSeparator() != ';') {}
    }
  }

  public static void main(String[] strArr) {
    Test _instance = new Test();
    for (int i; ; ) _instance.mainTest(strArr);
  }
}

Diagnostic files

By issuing

$ java -Xmx1G Test

the following crash log is given:

#0: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x7ab6d5) [0x7f81eee6e6d5]
#1: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x7b69c0) [0x7f81eee799c0]
#2: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x1125d9) [0x7f81ee7d55d9]
#3: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9prt29.so(+0x2911a) [0x7f81f42b211a]
#4: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14420) [0x7f81f4726420]
#5: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x74b851) [0x7f81eee0e851]
#6: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x78b77e) [0x7f81eee4e77e]
#7: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x7a56d3) [0x7f81eee686d3]
#8: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x75223b) [0x7f81eee1523b]
#9: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x774fa4) [0x7f81eee37fa4]
#10: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x7a58fb) [0x7f81eee688fb]
#11: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x7a5d21) [0x7f81eee68d21]
#12: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x78f72e) [0x7f81eee5272e]
#13: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x792b08) [0x7f81eee55b08]
#14: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x7914ba) [0x7f81eee544ba]
#15: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x791667) [0x7f81eee54667]
#16: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x792b08) [0x7f81eee55b08]
#17: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x7914ba) [0x7f81eee544ba]
#18: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x792584) [0x7f81eee55584]
#19: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x792815) [0x7f81eee55815]
#20: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x7969f2) [0x7f81eee599f2]
#21: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x635447) [0x7f81eecf8447]
#22: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x6359d9) [0x7f81eecf89d9]
#23: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x6359d9) [0x7f81eecf89d9]
#24: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x636d8b) [0x7f81eecf9d8b]
#25: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x42ba25) [0x7f81eeaeea25]
#26: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x121e6a) [0x7f81ee7e4e6a]
#27: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x122c73) [0x7f81ee7e5c73]
#28: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9prt29.so(+0x29c53) [0x7f81f42b2c53]
#29: /zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so(+0x120a69) [0x7f81ee7e3a69]
Unhandled exception
Type=Segmentation error vmState=0x000514ff
J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000001
Handler1=00007F81F44C8020 Handler2=00007F81F42B1EF0 InaccessibleAddress=000000000000000C
RDI=0000000000000000 RSI=00007F81EC431670 RAX=00007F81EC431670 RBX=00007F81EC431670
RCX=0000000000000000 RDX=000000000000000A R8=00000000FFFFFFFF R9=00007F81ECCAB4E0
R10=0000000000000040 R11=00007F81EC22B3B0 R12=00007F81EBEF7020 R13=00007F81EC4194C0
R14=00000000FFFFFFFF R15=0000000000000000
RIP=00007F81EEE0E851 GS=0000 FS=0000 RSP=00007F81ECCAAE50
EFlags=0000000000010202 CS=0033 RBP=0000000000000000 ERR=0000000000000004
TRAPNO=000000000000000E OLDMASK=0000000000000000 CR2=000000000000000C
xmm0 0000000000000000 (f: 0.000000, d: 0.000000e+00)
xmm1 ff00000000ffff00 (f: 16776960.000000, d: -5.486124e+303)
xmm2 000000000000ff00 (f: 65280.000000, d: 3.225261e-319)
xmm3 ffff000000000000 (f: 0.000000, d: -nan)
xmm4 00007f81ec414050 (f: 3963699200.000000, d: 6.926602e-310)
xmm5 00007f81ec4141a0 (f: 3963699712.000000, d: 6.926602e-310)
xmm6 00003b6e6f697470 (f: 1869182080.000000, d: 3.228497e-310)
xmm7 0000000000000000 (f: 0.000000, d: 0.000000e+00)
xmm8 252e732a2e250073 (f: 774176896.000000, d: 1.372768e-129)
xmm9 0000000000000000 (f: 0.000000, d: 0.000000e+00)
xmm10 08041b062f050804 (f: 788858880.000000, d: 4.757186e-270)
xmm11 0000000000000000 (f: 0.000000, d: 0.000000e+00)
xmm12 0000000000000000 (f: 0.000000, d: 0.000000e+00)
xmm13 0000000000000000 (f: 0.000000, d: 0.000000e+00)
xmm14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
xmm15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
Module=/zdata/congli/OpenJ9/jdk8/jre/lib/amd64/default/libj9jit29.so
Module_base_address=00007F81EE6C3000

Method_being_compiled=Test.mainTest([Ljava/lang/String;)V
Target=2_90_20220610_000000 (Linux 5.4.0-117-generic)
CPU=amd64 (8 logical CPUs) (0x3e45b9000 RAM)
----------- Stack Backtrace -----------
_ZN2TR12VPConstraint9intersectEPS0_PN3OMR16ValuePropagationE+0x21 (0x00007F81EEE0E851 [libj9jit29.so+0x74b851])
_ZN3OMR16ValuePropagation19addGlobalConstraintEPN2TR4NodeEiPNS1_12VPConstraintEi+0xce (0x00007F81EEE4E77E [libj9jit29.so+0x78b77e])
_ZN3OMR16ValuePropagation13getConstraintEPN2TR4NodeERbS3_+0xa3 (0x00007F81EEE686D3 [libj9jit29.so+0x7a56d3])
_ZL25canFoldNonOverriddenGuardPN3OMR16ValuePropagationEPN2TR4NodeES4_+0x4b (0x00007F81EEE1523B [libj9jit29.so+0x75223b])
_ZL18constrainIfcmpeqnePN3OMR16ValuePropagationEPN2TR4NodeEb+0x1ea4 (0x00007F81EEE37FA4 [libj9jit29.so+0x774fa4])
_ZN3OMR16ValuePropagation10launchNodeEPN2TR4NodeES3_i+0x9b (0x00007F81EEE688FB [libj9jit29.so+0x7a58fb])
_ZN3OMR16ValuePropagation12processTreesEPN2TR7TreeTopES3_+0x191 (0x00007F81EEE68D21 [libj9jit29.so+0x7a5d21])
_ZN2TR22GlobalValuePropagation12processBlockEP24TR_StructureSubGraphNodebb+0x31e (0x00007F81EEE5272E [libj9jit29.so+0x78f72e])
_ZN2TR22GlobalValuePropagation17processRegionNodeEP24TR_StructureSubGraphNodebb+0x178 (0x00007F81EEE55B08 [libj9jit29.so+0x792b08])
_ZN2TR22GlobalValuePropagation21processRegionSubgraphEP24TR_StructureSubGraphNodebbb+0x10a (0x00007F81EEE544BA [libj9jit29.so+0x7914ba])
_ZN2TR22GlobalValuePropagation18processNaturalLoopEP24TR_StructureSubGraphNodebb+0xb7 (0x00007F81EEE54667 [libj9jit29.so+0x791667])
_ZN2TR22GlobalValuePropagation17processRegionNodeEP24TR_StructureSubGraphNodebb+0x178 (0x00007F81EEE55B08 [libj9jit29.so+0x792b08])
_ZN2TR22GlobalValuePropagation21processRegionSubgraphEP24TR_StructureSubGraphNodebbb+0x10a (0x00007F81EEE544BA [libj9jit29.so+0x7914ba])
_ZN2TR22GlobalValuePropagation20processAcyclicRegionEP24TR_StructureSubGraphNodebb+0x34 (0x00007F81EEE55584 [libj9jit29.so+0x792584])
_ZN2TR22GlobalValuePropagation20determineConstraintsEv+0xa5 (0x00007F81EEE55815 [libj9jit29.so+0x792815])
_ZN2TR22GlobalValuePropagation7performEv+0x442 (0x00007F81EEE599F2 [libj9jit29.so+0x7969f2])
_ZN3OMR9Optimizer19performOptimizationEPK20OptimizationStrategyiii+0x767 (0x00007F81EECF8447 [libj9jit29.so+0x635447])
_ZN3OMR9Optimizer19performOptimizationEPK20OptimizationStrategyiii+0xcf9 (0x00007F81EECF89D9 [libj9jit29.so+0x6359d9])
_ZN3OMR9Optimizer19performOptimizationEPK20OptimizationStrategyiii+0xcf9 (0x00007F81EECF89D9 [libj9jit29.so+0x6359d9])
_ZN3OMR9Optimizer8optimizeEv+0x1db (0x00007F81EECF9D8B [libj9jit29.so+0x636d8b])
_ZN3OMR11Compilation7compileEv+0x925 (0x00007F81EEAEEA25 [libj9jit29.so+0x42ba25])
_ZN2TR28CompilationInfoPerThreadBase7compileEP10J9VMThreadPNS_11CompilationEP17TR_ResolvedMethodR11TR_J9VMBaseP19TR_OptimizationPlanRKNS_16SegmentAllocatorE+0x4fa (0x00007F81EE7E4E6A [libj9jit29.so+0x121e6a])
_ZN2TR28CompilationInfoPerThreadBase14wrappedCompileEP13J9PortLibraryPv+0x323 (0x00007F81EE7E5C73 [libj9jit29.so+0x122c73])
omrsig_protect+0x1e3 (0x00007F81F42B2C53 [libj9prt29.so+0x29c53])
_ZN2TR28CompilationInfoPerThreadBase7compileEP10J9VMThreadP21TR_MethodToBeCompiledRN2J917J9SegmentProviderE+0x309 (0x00007F81EE7E3A69 [libj9jit29.so+0x120a69])
_ZN2TR24CompilationInfoPerThread12processEntryER21TR_MethodToBeCompiledRN2J917J9SegmentProviderE+0x207 (0x00007F81EE7E40D7 [libj9jit29.so+0x1210d7])
_ZN2TR24CompilationInfoPerThread14processEntriesEv+0x38b (0x00007F81EE7E2DAB [libj9jit29.so+0x11fdab])
_ZN2TR24CompilationInfoPerThread3runEv+0x2a (0x00007F81EE7E309A [libj9jit29.so+0x12009a])
_Z30protectedCompilationThreadProcP13J9PortLibraryPN2TR24CompilationInfoPerThreadE+0x82 (0x00007F81EE7E3162 [libj9jit29.so+0x120162])
omrsig_protect+0x1e3 (0x00007F81F42B2C53 [libj9prt29.so+0x29c53])
_Z21compilationThreadProcPv+0x1d2 (0x00007F81EE7E35A2 [libj9jit29.so+0x1205a2])
thread_wrapper+0x162 (0x00007F81F44742B2 [libj9thr29.so+0xf2b2])
start_thread+0xd9 (0x00007F81F471A609 [libpthread.so.0+0x8609])
clone+0x43 (0x00007F81F485A133 [libc.so.6+0x11f133])
---------------------------------------
JVMDUMP039I Processing dump event "gpf", detail "" at 2022/06/13 22:12:45 - please wait.
JVMDUMP032I JVM requested System dump using '/zdata/congli/ax-exp/ax-eval/2-ax-only/63.openj9/mutant/red/aaa/core.20220613.221245.555411.0001.dmp' in response to an event
JVMDUMP010I System dump written to /zdata/congli/ax-exp/ax-eval/2-ax-only/63.openj9/mutant/red/aaa/core.20220613.221245.555411.0001.dmp
JVMDUMP032I JVM requested Java dump using '/zdata/congli/ax-exp/ax-eval/2-ax-only/63.openj9/mutant/red/aaa/javacore.20220613.221245.555411.0002.txt' in response to an event
JVMDUMP010I Java dump written to /zdata/congli/ax-exp/ax-eval/2-ax-only/63.openj9/mutant/red/aaa/javacore.20220613.221245.555411.0002.txt
JVMDUMP032I JVM requested Snap dump using '/zdata/congli/ax-exp/ax-eval/2-ax-only/63.openj9/mutant/red/aaa/Snap.20220613.221245.555411.0003.trc' in response to an event
JVMDUMP010I Snap dump written to /zdata/congli/ax-exp/ax-eval/2-ax-only/63.openj9/mutant/red/aaa/Snap.20220613.221245.555411.0003.trc
JVMDUMP032I JVM requested JIT dump using '/zdata/congli/ax-exp/ax-eval/2-ax-only/63.openj9/mutant/red/aaa/jitdump.20220613.221245.555411.0004.dmp' in response to an event
JVMDUMP051I JIT dump occurred in 'JIT Compilation Thread-000' thread 0x000000000001E600
JVMDUMP049I JIT dump notified all waiting threads of the current method to be compiled
JVMDUMP054I JIT dump is tracing the IL of the method on the crashed compilation thread
JVMDUMP048I JIT dump method being compiled is an ordinary method
JVMDUMP053I JIT dump is recompiling Test.mainTest([Ljava/lang/String;)V
JVMDUMP010I JIT dump written to /zdata/congli/ax-exp/ax-eval/2-ax-only/63.openj9/mutant/red/aaa/jitdump.20220613.221245.555411.0004.dmp
JVMDUMP013I Processed dump event "gpf", detail "".

Please also check openj9-bug-63.tar.gz for all the logs (jitdump, snap, etc.) and the testcase (Test.java, Test.class).

Notice

The given Test.java (which is reduced by us) is NOT always reproducible, and often take ~1min to crash when it is reproducible. So please be patient. If it is not reproducible, please use Test.java.orig in the above link.

connglli commented 2 years ago

We can reproduce the reduced Test.java in one of our desktop machine but cannot on our performant server. So please use the unreduced Test.java.orig if you cannot reproduce the crash.

pshipton commented 2 years ago

The crash didn't reproduce for me. Trying Test.java.orig, it doesn't compile - error: package FuzzerUtils does not exist

pshipton commented 2 years ago

Actually, I was able to reproduce the crash using the reduced test case on plinux (using 11.0.14.1).

pshipton commented 2 years ago

vmState [0x514ff]: {J9VMSTATE_JIT} {globalValuePropagation}

@0xdaryl fyi

pshipton commented 2 years ago

It doesn't reproduce on plinux using the reduced test case with the latest build.

connglli commented 2 years ago

@pshipton Sorry for fogetting FuzzerUtils.java. See this

/*
* Copyright (C) 2016 Intel Corporation
* Modifications copyright (C) 2017-2018 Azul Systems
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
*      http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

import java.util.concurrent.atomic.AtomicLong;
import java.io.PrintStream;
import java.util.Random;

// Utilities for the tests generated by Java* Fuzzer for Android*
// with modifications for Java* Fuzzer test generator

public class FuzzerUtils {

    public static PrintStream out = System.out;
    public static Random random = new Random(1);
    public static long seed = 1L;
    public static int UnknownZero = 0;

    // Init seed
    public static void seed(long seed){
        random = new Random(seed);
        FuzzerUtils.seed = seed;
    }

    public static int nextInt(){
        return random.nextInt();
    }
    public static long nextLong(){
        return random.nextLong();
    }
    public static float nextFloat(){
        return random.nextFloat();
    }
    public static double nextDouble(){
        return random.nextDouble();
    }
    public static boolean nextBoolean(){
        return random.nextBoolean();
    }
    public static byte nextByte(){
        return (byte)random.nextInt();
    }
    public static short nextShort(){
        return (short)random.nextInt();
    }
    public static char nextChar(){
        return (char)random.nextInt();
    }

    // Array initialization

    // boolean -----------------------------------------------
    public static void init(boolean[] a, boolean seed) {
        for (int j = 0; j < a.length; j++) {
            a[j] = (j % 2 == 0) ? seed : (j % 3 == 0);
        }
    }

    public static void init(boolean[][] a, boolean seed) {
        for (int j = 0; j < a.length; j++) {
            init(a[j], seed);
        }
    }

    // long --------------------------------------------------
    public static void init(long[] a, long seed) {
        for (int j = 0; j < a.length; j++) {
            a[j] = (j % 2 == 0) ? seed + j : seed - j;
        }
    }

    public static void init(long[][] a, long seed) {
        for (int j = 0; j < a.length; j++) {
            init(a[j], seed);
        }
    }

    // int --------------------------------------------------
    public static void init(int[] a, int seed) {
        for (int j = 0; j < a.length; j++) {
            a[j] = (j % 2 == 0) ? seed + j : seed - j;
        }
    }

    public static void init(int[][] a, int seed) {
        for (int j = 0; j < a.length; j++) {
            init(a[j], seed);
        }
    }

    // short --------------------------------------------------
    public static void init(short[] a, short seed) {
        for (int j = 0; j < a.length; j++) {
            a[j] = (short) ((j % 2 == 0) ? seed + j : seed - j);
        }
    }

    public static void init(short[][] a, short seed) {
        for (int j = 0; j < a.length; j++) {
            init(a[j], seed);
        }
    }

    // char --------------------------------------------------
    public static void init(char[] a, char seed) {
        for (int j = 0; j < a.length; j++) {
            a[j] = (char) ((j % 2 == 0) ? seed + j : seed - j);
        }
    }

    public static void init(char[][] a, char seed) {
        for (int j = 0; j < a.length; j++) {
            init(a[j], seed);
        }
    }

    // byte --------------------------------------------------
    public static void init(byte[] a, byte seed) {
        for (int j = 0; j < a.length; j++) {
            a[j] = (byte) ((j % 2 == 0) ? seed + j : seed - j);
        }
    }

    public static void init(byte[][] a, byte seed) {
        for (int j = 0; j < a.length; j++) {
            init(a[j], seed);
        }
    }

    // double --------------------------------------------------
    public static void init(double[] a, double seed) {
        for (int j = 0; j < a.length; j++) {
            a[j] = (j % 2 == 0) ? seed + j : seed - j;
        }
    }

    public static void init(double[][] a, double seed) {
        for (int j = 0; j < a.length; j++) {
            init(a[j], seed);
        }
    }

    // float --------------------------------------------------
    public static void init(float[] a, float seed) {
        for (int j = 0; j < a.length; j++) {
            a[j] = (j % 2 == 0) ? seed + j : seed - j;
        }
    }

    public static void init(float[][] a, float seed) {
        for (int j = 0; j < a.length; j++) {
            init(a[j], seed);
        }
    }
    // Object -------------------------------------------------

    public static void init(Object[][] a, Object seed) {
        for (int j = 0; j < a.length; j++) {
            init(a[j], seed);
        }
    }

    public static void init(Object[] a, Object seed) {
        for (int j = 0; j < a.length; j++)
            try {
                a[j] = seed.getClass().newInstance();
            } catch (Exception ex) {
                a[j] = seed;
            }
    }

    // Calculate array checksum

    // boolean -----------------------------------------------
    public static long checkSum(boolean[] a) {
        long sum = 0;
        for (int j = 0; j < a.length; j++) {
            sum += (a[j] ? j + 1 : 0);
        }
        return sum;
    }

    public static long checkSum(boolean[][] a) {
        long sum = 0;
        for (int j = 0; j < a.length; j++) {
            sum += checkSum(a[j]);
        }
        return sum;
    }

    // long --------------------------------------------------
    public static long checkSum(long[] a) {
        long sum = 0;
        for (int j = 0; j < a.length; j++) {
            sum += (a[j] / (j + 1) + a[j] % (j + 1));
        }
        return sum;
    }

    public static long checkSum(long[][] a) {
        long sum = 0;
        for (int j = 0; j < a.length; j++) {
            sum += checkSum(a[j]);
        }
        return sum;
    }

    // int --------------------------------------------------
    public static long checkSum(int[] a) {
        long sum = 0;
        for (int j = 0; j < a.length; j++) {
            sum += (a[j] / (j + 1) + a[j] % (j + 1));
        }
        return sum;
    }

    public static long checkSum(int[][] a) {
        long sum = 0;
        for (int j = 0; j < a.length; j++) {
            sum += checkSum(a[j]);
        }
        return sum;
    }

    // short --------------------------------------------------
    public static long checkSum(short[] a) {
        long sum = 0;
        for (int j = 0; j < a.length; j++) {
            sum += (short) (a[j] / (j + 1) + a[j] % (j + 1));
        }
        return sum;
    }

    public static long checkSum(short[][] a) {
        long sum = 0;
        for (int j = 0; j < a.length; j++) {
            sum += checkSum(a[j]);
        }
        return sum;
    }

    // char --------------------------------------------------
    public static long checkSum(char[] a) {
        long sum = 0;
        for (int j = 0; j < a.length; j++) {
            sum += (char) (a[j] / (j + 1) + a[j] % (j + 1));
        }
        return sum;
    }

    public static long checkSum(char[][] a) {
        long sum = 0;
        for (int j = 0; j < a.length; j++) {
            sum += checkSum(a[j]);
        }
        return sum;
    }

    // byte --------------------------------------------------
    public static long checkSum(byte[] a) {
        long sum = 0;
        for (int j = 0; j < a.length; j++) {
            sum += (byte) (a[j] / (j + 1) + a[j] % (j + 1));
        }
        return sum;
    }

    public static long checkSum(byte[][] a) {
        long sum = 0;
        for (int j = 0; j < a.length; j++) {
            sum += checkSum(a[j]);
        }
        return sum;
    }

    // double --------------------------------------------------
    public static double checkSum(double[] a) {
        double sum = 0;
        for (int j = 0; j < a.length; j++) {
            sum += (a[j] / (j + 1) + a[j] % (j + 1));
        }
        return sum;
    }

    public static double checkSum(double[][] a) {
        double sum = 0;
        for (int j = 0; j < a.length; j++) {
            sum += checkSum(a[j]);
        }
        return sum;
    }

    // float --------------------------------------------------
    public static double checkSum(float[] a) {
        double sum = 0;
        for (int j = 0; j < a.length; j++) {
            sum += (a[j] / (j + 1) + a[j] % (j + 1));
        }
        return sum;
    }

    public static double checkSum(float[][] a) {
        double sum = 0;
        for (int j = 0; j < a.length; j++) {
            sum += checkSum(a[j]);
        }
        return sum;
    }

    // Object --------------------------------------------------
    public static long checkSum(Object[][] a) {
        long sum = 0;
        for (int j = 0; j < a.length; j++) {
            sum += checkSum(a[j]);
        }
        return sum;
    }

    public static long checkSum(Object[] a) {
        long sum = 0;
        for (int j = 0; j < a.length; j++) {
            sum += checkSum(a[j]) * Math.pow(2, j);
        }
        return sum;
    }

    public static long checkSum(Object a) {
        if (a == null)
            return 0L;
        return (long) a.getClass().getCanonicalName().length();
    }

    // Array creation ------------------------------------------
    public static byte[] byte1array(int sz, byte seed) {
        byte[] ret = new byte[sz];
        init(ret, seed);
        return ret;
    }

    public static byte[][] byte2array(int sz, byte seed) {
        byte[][] ret = new byte[sz][sz];
        init(ret, seed);
        return ret;
    }

    public static short[] short1array(int sz, short seed) {
        short[] ret = new short[sz];
        init(ret, seed);
        return ret;
    }

    public static short[][] short2array(int sz, short seed) {
        short[][] ret = new short[sz][sz];
        init(ret, seed);
        return ret;
    }

    public static int[] int1array(int sz, int seed) {
        int[] ret = new int[sz];
        init(ret, seed);
        return ret;
    }

    public static int[][] int2array(int sz, int seed) {
        int[][] ret = new int[sz][sz];
        init(ret, seed);
        return ret;
    }

    public static long[] long1array(int sz, long seed) {
        long[] ret = new long[sz];
        init(ret, seed);
        return ret;
    }

    public static long[][] long2array(int sz, long seed) {
        long[][] ret = new long[sz][sz];
        init(ret, seed);
        return ret;
    }

    public static float[] float1array(int sz, float seed) {
        float[] ret = new float[sz];
        init(ret, seed);
        return ret;
    }

    public static float[][] float2array(int sz, float seed) {
        float[][] ret = new float[sz][sz];
        init(ret, seed);
        return ret;
    }

    public static double[] double1array(int sz, double seed) {
        double[] ret = new double[sz];
        init(ret, seed);
        return ret;
    }

    public static double[][] double2array(int sz, double seed) {
        double[][] ret = new double[sz][sz];
        init(ret, seed);
        return ret;
    }

    public static char[] char1array(int sz, char seed) {
        char[] ret = new char[sz];
        init(ret, seed);
        return ret;
    }

    public static char[][] char2array(int sz, char seed) {
        char[][] ret = new char[sz][sz];
        init(ret, seed);
        return ret;
    }

    public static Object[] Object1array(int sz, Object seed) {
        Object[] ret = new Object[sz];
        init(ret, seed);
        return ret;
    }

    public static Object[][] Object2array(int sz, Object seed) {
        Object[][] ret = new Object[sz][sz];
        init(ret, seed);
        return ret;
    }

    public static boolean[] boolean1array(int sz, boolean seed) {
        boolean[] ret = new boolean[sz];
        init(ret, seed);
        return ret;
    }

    public static boolean[][] boolean2array(int sz, boolean seed) {
        boolean[][] ret = new boolean[sz][sz];
        init(ret, seed);
        return ret;
    }

    public static AtomicLong runningThreads = new AtomicLong(0);

    public static synchronized void runThread(Runnable r) {
        final Thread t = new Thread(r);
        t.start();
        runningThreads.incrementAndGet();
        Thread t1 = new Thread(new Runnable() {
            public void run() {
                try {
                    t.join();
                    runningThreads.decrementAndGet();
                } catch (InterruptedException e) {
                }
            }
        });
        t1.start();
    }

    public static void joinThreads() {
        while (runningThreads.get() > 0) {
            try {
                Thread.sleep(1000);
            } catch (InterruptedException e) {
            }
        }
    }
}
pshipton commented 2 years ago

Using the original testcase, I can reproduce it now on xlinux on 0.32 and the latest JVM,

0xdaryl commented 2 years ago

@hzongaro : could you have a look at this reproducible GVP crash please? Note it is initially targeted to 0.33.

hzongaro commented 2 years ago

This might be the same as issue #14489. I am readily able to reproduce it and am continuing to investigate.

BradleyWood commented 2 years ago

@hzongaro I looked through the value propagation git history in OMR and tried reverting a few recent PR's. I was not able to reproduce after reverting eclipse/omr#6121.

hzongaro commented 2 years ago

I looked through the value propagation git history in OMR and tried reverting a few recent PR's. I was not able to reproduce after reverting https://github.com/eclipse/omr/pull/6121.

Thanks, Brad @BradleyWood! I'll take a look at whether that causes the problem, or if that change was correct, and exposed the problem. . . .

hzongaro commented 2 years ago

I took at look at transformations that GVP performs when OMR pull request #6121 is reverted. In that case, although there's no crash, I can see that it removes a NULLCHK even though there is a path through which a null reference is assigned to the temporary that's involved - that's the same null reference that causes the crash in attempting to intersect constraints when that pull request is in place. So I believe that the problem already existed, but that pull request just changed the symptom of the problem - at least in this case.

n7996n    BBStart <block_593> (freq 3) (catches java/lang/Exception) (cold)
  ...
n8175n    astore  <temp slot 30>[#1309  Auto] [flags 0x7 0x0 ] (X==0 sharedMemory )           [0x7f84fada0b00]
n8021n      aconst NULL (X==0 sharedMemory )                                                  [0x7f84fad9dae0]
n8176n    goto --> block_580 BBStart at n8171n                                                [0x7f84fada0b50]
n7997n    BBEnd </block_593> (cold) =====                                                     [0x7f84fad9d360]
   ...
n8171n    BBStart <block_580> (freq 6) (in loop 14)                                           [0x7f84fada09c0]
n8189n    astore  <temp slot 30>[#1309  Auto] [flags 0x7 0x0 ]                                [0x7f84fada0f60]
n8179n      aload  <temp slot 30>[#1309  Auto] [flags 0x7 0x0 ]                               [0x7f84fada0c40]
n1161n    astore  <auto slot 24>[#496  Auto] [flags 0x7 0x0 ]                                 [0x7f84fa737aa0]
n8194n      aload  <temp slot 30>[#1309  Auto] [flags 0x7 0x0 ]                               [0x7f84fada10f0]
   ...
n8218n    ificmpne --> block_601 BBStart at n8212n (inlineNonoverriddenGuard )                [0x7f84fada1870]
   ...
n8209n    BBEnd </block_580> =====
   ...
n8212n    BBStart <block_601> (freq 1) (cold) (in loop 14)                                    [0x7f84fada1690]
n8225n    NULLCHK on n8232n [#32]                                                             [0x7f84fada1aa0]
n1164n      icalli  java/text/DecimalFormatSymbols.getPatternSeparator()C[#498  virtual Method -160] [flags 0x500 0x0 ] (virtualCallNodeForAGuardedInlinedCall )  [0x7f84fa737b90]
n1163n        aloadi  <vft-symbol>[#309  Shadow] [flags 0x18607 0x0 ]                         [0x7f84fa737b40]
n8233n          aload  <auto slot 24>[#496  Auto] [flags 0x7 0x0 ] (X>=0 sharedMemory )       [0x7f84fada1d20] 
n8232n        aload  <auto slot 24>[#496  Auto] [flags 0x7 0x0 ]                              [0x7f84fada1cd0]
   ...
Node n8232n has already been processed by mergeDefConstraints - returning NULL
   aload [00007F84FADA1CD0] has existing global constraint: value 1434 is class Ljava/text/DecimalFormatSymbols; (non-NULL) {HeapObject}
syncRequired is already setup at node [00007F84FA737B90]
[ 18764] O^O VALUE PROPAGATION: Removing redundant null check node [00007F84FADA1AA0]

Now I'm trying to understand why mergeDefConstraints is marking these constraints as global.

hzongaro commented 2 years ago

I'm still able to reproduce the failure with this reduced test:

import java.text.DecimalFormatSymbols;
import java.util.Locale;
import java.text.DecimalFormat;
import java.text.NumberFormat;

// Generated by Java* Fuzzer test generator (1.0.001). Sat Jun 11 23:30:48 2022
public class Test {

    public void mainTest(String[] strArr1) {
        for (int i30 = 0; i30 < 1000; i30++) {
            java.io.PrintStream ax$29 = System.out;
            java.io.PrintStream ax$30 = System.err;
            for (int ax$27 = 0; ax$27 < 1000; ax$27++) {
                try {
                    DecimalFormat ax$25 = (DecimalFormat) NumberFormat.getInstance(Locale.JAPAN);
                    DecimalFormatSymbols ax$24 = ax$25.getDecimalFormatSymbols();
                    if (ax$24.getPatternSeparator() != ';') {
                    }
                } catch (Throwable ax$26) {
                } finally {
                }
            }
            System.setOut(ax$29);
            System.setErr(ax$30);
        }
    }

    public static void main(String[] strArr) {
        Test _instance = new Test();
        for (int i = 0; i < 10; i++) {
            _instance.mainTest(strArr);
        }
    }

    private static Boolean ax$32 = false;
}

I'll post some more details showing what I think is going wrong.

0xdaryl commented 2 years ago

This is not a 0.33 regression. It is reproducible at least as far back as 0.24 on JDK11. As @hzongaro mentions above, eclipse/omr#6121 is likely a red herring.

hzongaro commented 2 years ago

I'm still struggling a bit with how to resolve this issue. I don't think it can be quickly resolved safely, and as it's a long-standing problem I don't think it should be considered a blocker - and I see the Peter @pshipton has already removed the blocker label.

I'll post some more details about what I see happening, and reach out to Vijay @vijaysun-omr for some advice.

hzongaro commented 2 years ago

Inside the inner loop (loop 6), we have several paths through inlined code and guarded calls that assigns values to #397. The reference in #397 is used on the taken side of a guarded icalli for DecimalFormatSymbols.getPatternSeparator in block_47. One of the inlined code paths results in a null reference being assigned to #397 in a catch block (block_39); on another path, the reference assigned to #397 (in block_94) is the result of a call to Object.clone(), so known to be non-null.

The relevant portion of the CFG looks this:

31→32, 32→92, 92→93, 93→94, 94→47 39→26, 26→47 44→47 89→47

The branches from blocks 26, 89 and 94 are conditional; the branch from block 44 (which is on the taken side of the inlined guard for DecimalFormat.getDecimalFormatSymbols is unconditional. Defs for #397 appear in blocks 94, 89, 26 and 44.

In processing block_94 the last time through, constrainIfcmpeqne calls canFoldNonOverriddenGuard, which calls getConstraint for the first argument to the guarded call in block_47:

n3633n    BBStart <block_94> (freq 9990) (in loop 6)                                          [0x7f1372050f50] bci=[-1,42,16] rc=0 vc=494 vn=172 li=- udi=- nc=0
n3634n    astore  <temp slot 11>[#737  Auto] [flags 0x7 0x0 ]                                 [0x7f1372050fa0] bci=[2,0,566] rc=0 vc=494 vn=28 li=28 udi=17 nc=1
n3635n      aload  <temp slot 11>[#737  Auto] [flags 0x7 0x0 ]                                [0x7f1372050ff0] bci=[2,0,566] rc=1 vc=494 vn=28 li=70 udi=91 nc=0
n3636n    astore  <auto slot 7>[#397  Auto] [flags 0x7 0x0 ]                                  [0x7f1372051040] bci=[-1,45,16] rc=0 vc=494 vn=28 li=29 udi=18 nc=1
n3637n      aload  <temp slot 11>[#737  Auto] [flags 0x7 0x0 ]                                [0x7f1372051090] bci=[-1,42,16] rc=2 vc=494 vn=28 li=71 udi=92 nc=0
n3638n    astore  <pending push temp 0>[#392  Auto] [flags 0x7 0x800 ]                        [0x7f13720510e0] bci=[-1,42,16] rc=0 vc=494 vn=28 li=- udi=19 nc=1
n3637n      ==>aload
n3639n    ificmpne --> block_47 BBStart at n3190n (inlineNonoverriddenGuard )                 [0x7f1372051130] bci=[3,0,303] rc=0 vc=494 vn=173 li=- udi=- nc=2 flg=0x5020
n3640n      iand (X>=0 cannotOverflow )                                                       [0x7f1372051180] bci=[3,0,303] rc=1 vc=494 vn=33 li=- udi=- nc=2 flg=0x1100
n3641n        l2i                                                                             [0x7f13720511d0] bci=[3,0,303] rc=1 vc=494 vn=32 li=- udi=- nc=1
n3642n          lload      0x1d7158[#739  Static] [flags 0x307 0x0 ] (cannotOverflow )        [0x7f1372051220] bci=[-1,49,17] rc=1 vc=494 vn=31 li=- udi=- nc=0 flg=0x1000
n3643n        iconst 4 (X!=0 X>=0 )                                                           [0x7f1372051270] bci=[3,0,303] rc=1 vc=494 vn=22 li=- udi=- nc=0 flg=0x104
n3644n      iconst 0 (X==0 X>=0 X<=0 )                                                        [0x7f13720512c0] bci=[3,0,303] rc=1 vc=494 vn=3 li=- udi=- nc=0 flg=0x302
n3645n    BBEnd </block_94> =====                                                             [0x7f1372051310] bci=[3,4,303] rc=0 vc=494 vn=174 li=- udi=- nc=0

...

n3190n    BBStart <block_47> (freq 1) (cold)                                                  [0x7f13721d84d0] bci=[-1,49,17] rc=0 vc=0 vn=- li=- udi=- nc=0
n3203n    NULLCHK on n3210n [#32]                                                             [0x7f13721d88e0] bci=[-1,49,17] rc=0 vc=0 vn=- li=- udi=- nc=1
n61n        icalli  java/text/DecimalFormatSymbols.getPatternSeparator()C[#399  virtual Method -160] [flags 0x500 0x0 ] (virtualCallNodeForAGuardedInlinedCall )  [0x7f1371d2b2d0] bci=[-1,49,17] rc=3 vc=44 vn=- li=- udi=- nc=2 flg=0x800
n60n          aloadi  <vft-symbol>[#309  Shadow] [flags 0x18607 0x0 ]                         [0x7f1371d2b280] bci=[-1,49,17] rc=1 vc=44 vn=- li=- udi=- nc=1
n3211n          aload  <auto slot 7>[#397  Auto] [flags 0x7 0x0 ]                             [0x7f13721d8b60] bci=[-1,49,17] rc=1 vc=0 vn=- li=- udi=- nc=0
n3210n        aload  <auto slot 7>[#397  Auto] [flags 0x7 0x0 ]                               [0x7f13721d8b10] bci=[-1,49,17] rc=1 vc=0 vn=- li=- udi=- nc=0
n3204n    istore  <pending push temp 0>[#400  Auto] [flags 0x3 0x800 ]                        [0x7f13721d8930] bci=[-1,49,17] rc=0 vc=0 vn=- li=- udi=- nc=1
n61n        ==>icalli
n3205n    istore  <temp slot 12>[#741  Auto] [flags 0x3 0x0 ]                                 [0x7f13721d8980] bci=[-1,49,17] rc=0 vc=0 vn=- li=- udi=- nc=1
n61n        ==>icalli
n3206n    goto --> block_46 BBStart at n3188n                                                 [0x7f13721d89d0] bci=[-1,49,17] rc=0 vc=0 vn=- li=- udi=- nc=0
n3191n    BBEnd </block_47> (cold) =====                                                      [0x7f13721d8520] bci=[-1,49,17] rc=0 vc=0 vn=- li=- udi=- nc=0

getConstraint calls mergeDefConstraints for node n3210n.

Where things go awry in mergeDefConstraints, there are two places where unseenDefsFound might be set to true: firstly for autos or parms that are not defined on all paths, and secondly for a store which has not been seen.

In this case, #397 is marked as defined on all paths, and the def from n3636n is considered to be seen, but the stores from the other paths are not considered to be seen - so unseenDefsFound is set to true. The one set of constraints that come from the def at node n3636n is Ljava/text/DecimalFormatSymbols; (non-NULL) {HeapObject}

Later mergeDefConstraints, as it's processing a loop the last-time through, attempting to look at back-edge constraints, it only considers autos and parms whose def node symref is not defined on all paths. All the def nodes use #397, so they end up being ignored, unseenDefsFound is left as false, and isGlobal is left as true. That results in mergeDefConstraints returning Ljava/text/DecimalFormatSymbols; (non-NULL) {HeapObject} as a global constraint.

Later, in processing block_26 the last-time through, the same thing happens

n2990n    BBStart <block_39> (freq 3) (catches java/lang/Exception) (cold)                    [0x7f13721d4650] bci=[1,11,2430] rc=0 vc=466 vn=- li=- udi=- nc=0
   ...
n3153n    astore  <temp slot 11>[#737  Auto] [flags 0x7 0x0 ] (X==0 sharedMemory )            [0x7f13721d7940] bci=[1,12,2431] rc=0 vc=466 vn=- li=- udi=- nc=1 flg=0x2
n3015n      aconst NULL (X==0 sharedMemory )                                                  [0x7f13721d4e20] bci=[1,12,2431] rc=1 vc=466 vn=- li=- udi=- nc=0 flg=0x2
n3154n    goto --> block_26 BBStart at n3149n                                                 [0x7f13721d7990] bci=[1,13,2431] rc=0 vc=466 vn=- li=- udi=- nc=0
n2991n    BBEnd </block_39> (cold) =====                                                      [0x7f13721d46a0] bci=[1,13,2431] rc=0 vc=466 vn=- li=- udi=- nc=0
  ...
n3149n    BBStart <block_26> (freq 6) (in loop 6)                                             [0x7f13721d7800] bci=[-1,42,16] rc=0 vc=494 vn=193 li=- udi=- nc=0
n3167n    astore  <temp slot 11>[#737  Auto] [flags 0x7 0x0 ]                                 [0x7f13721d7da0] bci=[2,0,566] rc=0 vc=494 vn=48 li=36 udi=29 nc=1
n3157n      aload  <temp slot 11>[#737  Auto] [flags 0x7 0x0 ]                                [0x7f13721d7a80] bci=[2,0,566] rc=1 vc=494 vn=48 li=82 udi=103 nc=0
n58n      astore  <auto slot 7>[#397  Auto] [flags 0x7 0x0 ]                                  [0x7f1371d2b1e0] bci=[-1,45,16] rc=0 vc=494 vn=48 li=37 udi=30 nc=1
n3172n      aload  <temp slot 11>[#737  Auto] [flags 0x7 0x0 ]                                [0x7f13721d7f30] bci=[-1,42,16] rc=2 vc=494 vn=48 li=83 udi=104 nc=0
n55n      astore  <pending push temp 0>[#392  Auto] [flags 0x7 0x800 ]                        [0x7f1371d2b0f0] bci=[-1,42,16] rc=0 vc=494 vn=48 li=- udi=31 nc=1
n3172n      ==>aload
n3196n    ificmpne --> block_47 BBStart at n3190n (inlineNonoverriddenGuard )                 [0x7f13721d86b0] bci=[3,0,303] rc=0 vc=494 vn=194 li=- udi=- nc=2 flg=0x5020
n3576n      iand (X>=0 cannotOverflow )                                                       [0x7f137204fd80] bci=[3,0,303] rc=1 vc=494 vn=51 li=- udi=- nc=2 flg=0x1100
n3575n        l2i                                                                             [0x7f137204fd30] bci=[3,0,303] rc=1 vc=494 vn=50 li=- udi=- nc=1
n3192n          lload      0x1d7158[#739  Static] [flags 0x307 0x0 ] (cannotOverflow )        [0x7f13721d8570] bci=[-1,49,17] rc=1 vc=494 vn=49 li=- udi=- nc=0 flg=0x1000
n3573n        iconst 4 (X!=0 X>=0 )                                                           [0x7f137204fc90] bci=[3,0,303] rc=1 vc=494 vn=22 li=- udi=- nc=0 flg=0x104
n3574n      iconst 0 (X==0 X>=0 X<=0 )                                                        [0x7f137204fce0] bci=[3,0,303] rc=1 vc=494 vn=3 li=- udi=- nc=0 flg=0x302
n3187n    BBEnd </block_26> =====                                                             [0x7f13721d83e0] bci=[3,4,303] rc=0 vc=494 vn=195 li=- udi=- nc=0

But this time, the constraint from the def at node n58n is (NULL), mergeDefConstraints leaves isGlobal true and unseenDefsFound false again, getConstraint attempts to intersect that new (incorrectly) global constraint with the previously seen (again, incorrectly) global constraint Ljava/text/DecimalFormatSymbols; (non-NULL) {HeapObject}, and crashes.

What I'm trying to understand is whether the later processing in mergeDefConstraints should ensure isGlobal ends up being set to false in these cases, but still return the constraint found from the definition on that path? Or if it should end up leaving it as unconstrained? Or something else. . . .

Vijay @vijaysun-omr, I would appreciate your thoughts, comments or questions about this

hzongaro commented 2 years ago

In case anyone is interested, here is the complete jitdump from which the information in the previous comment was extracted. It contains some additional tracing from getConstraint and mergeDefConstraints.

jitdump.20220620.202541.15478.0004.dmp.txt

hzongaro commented 2 years ago

Fixed by OMR pull request 6619