search

eclipse-openj9 / openj9

Eclipse OpenJ9: A Java Virtual Machine for OpenJDK that's optimized for small footprint, fast start-up, and high throughput. Builds on Eclipse OMR (https://github.com/eclipse/omr) and combines with the Extensions for OpenJDK for OpenJ9 repo.
Other
3.28k stars 721 forks source link

[JDK18] IsRecordTest Segfaults Intermittently #14354

Closed babsingh closed 2 years ago

babsingh commented 2 years ago

OpenJDK Test

java/lang/reflect/records/IsRecordTest.java

This failure is intermittent, and it was noticed with JDK18.

Errors

https://hyc-runtimes-jenkins.swg-devops.com/job/Test_openjdk18_j9_sanity.openjdk_x86-64_linux_Personal/15/console

14:02:32  STDOUT:
14:02:32  
14:02:32  --- testBasicRecords ---
14:02:32  test IsRecordTest.testBasicRecords(): success
...
...
14:02:32  --- testDirectSubClass isFinal=true, isAbstract=false, extendsJLR=false, withRecordAttr=false, expectIsRecord=false ---
14:02:32  cls=class C, Record::isAssignable=false, isRecord=false
14:02:32  test IsRecordTest.testDirectSubClass(true, false, false, false, false): success
14:02:32  
14:02:32  --- testDirectSubClass isFinal=true, isAbstract=false, extendsJLR=true, withRecordAttr=false, expectIsRecord=false ---
14:02:32  cls=class C, Record::isAssignable=true, isRecord=false
14:02:32  test IsRecordTest.testDirectSubClass(true, false, true, false, false): success
14:02:32  
14:02:32  --- testIndirectSubClass isFinal=false, isAbstract=false withRecordAttr=true ---
14:02:32  STDERR:
14:02:32  Unhandled exception
14:02:32  Type=Segmentation error vmState=0x00000000
14:02:32  J9Generic_Signal_Number=00000018 Signal_Number=0000000b Error_Value=00000000 Signal_Code=00000080
14:02:32  Handler1=00007F86DFA37250 Handler2=00007F86DF793CB0 InaccessibleAddress=0000000000000000
14:02:32  RDI=00007F86C2E1A703 RSI=00007F86C2E1A708 RAX=1000000000000001 RBX=0000000000000005
14:02:32  RCX=00000000000B0600 RDX=00007F86C2E1A708 R8=00000000FFDE0E50 R9=0000000000000002
14:02:32  R10=00000000FFDE0C58 R11=0000000000272000 R12=00000000E00B9228 R13=00000000FFDE0D40
14:02:32  R14=00000000E00B9228 R15=0000000000000001
14:02:32  RIP=00007F86DD888F08 GS=0000 FS=0000 RSP=000000000022F2E0
14:02:32  EFlags=0000000000010213 CS=0033 RBP=00000000001D7400 ERR=0000000000000000
14:02:32  TRAPNO=000000000000000D OLDMASK=0000000000000000 CR2=00007F1FD4E26000
14:02:32  xmm0 0000000000000005 (f: 5.000000, d: 2.470328e-323)
14:02:32  xmm1 0063006500520073 (f: 5374067.000000, d: 8.455966e-307)
14:02:32  xmm2 ffffffffffffffff (f: 4294967296.000000, d: -nan)
14:02:32  xmm3 bfe02157844dea3a (f: 2219698688.000000, d: -5.040701e-01)
14:02:32  xmm4 000000003f740320 (f: 1064567552.000000, d: 5.259663e-315)
14:02:32  xmm5 bff0000000000000 (f: 0.000000, d: -1.000000e+00)
14:02:32  xmm6 bfa252f438e10c1e (f: 954272768.000000, d: -3.578914e-02)
14:02:32  xmm7 0000000000000000 (f: 0.000000, d: 0.000000e+00)
14:02:32  xmm8 006d0065004d002f (f: 5046319.000000, d: 1.290611e-306)
14:02:32  xmm9 0000000000000000 (f: 0.000000, d: 0.000000e+00)
14:02:32  xmm10 0000000000000000 (f: 0.000000, d: 0.000000e+00)
14:02:32  xmm11 ca62c1d6ca62c1d6 (f: 3395469824.000000, d: -2.193092e+50)
14:02:32  xmm12 0000000000000000 (f: 0.000000, d: 0.000000e+00)
14:02:32  xmm13 0000000000000000 (f: 0.000000, d: 0.000000e+00)
14:02:32  xmm14 0000000000000000 (f: 0.000000, d: 0.000000e+00)
14:02:32  xmm15 0000000000000000 (f: 0.000000, d: 0.000000e+00)
14:02:32  Module=/home/jenkins/workspace/Test_openjdk18_j9_sanity.openjdk_x86-64_linux_Personal/openjdkbinary/j2sdk-image/lib/default/libj9jit29.so
14:02:32  Module_base_address=00007F86DD3E8000
14:02:32  Target=2_90_20220124_35 (Linux 5.4.0-66-generic)
14:02:32  CPU=amd64 (4 logical CPUs) (0xf5fb4000 RAM)
14:02:32  ----------- Stack Backtrace -----------
14:02:32  (0x00007F86DF7906C2 [libj9prt29.so+0x266c2])
14:02:32  (0x00007F86DF794A13 [libj9prt29.so+0x2aa13])
14:02:32  (0x00007F86DF79073E [libj9prt29.so+0x2673e])
14:02:32  (0x00007F86DF790834 [libj9prt29.so+0x26834])
14:02:32  (0x00007F86DF794A13 [libj9prt29.so+0x2aa13])
14:02:32  (0x00007F86DF79059B [libj9prt29.so+0x2659b])
14:02:32  (0x00007F86DFA36D94 [libj9vm29.so+0x3fd94])
14:02:32  (0x00007F86DF794A13 [libj9prt29.so+0x2aa13])
14:02:32  (0x00007F86DFA37005 [libj9vm29.so+0x40005])
14:02:32  (0x00007F86DFB2C7E5 [libj9vm29.so+0x1357e5])
14:02:32  ---------------------------------------
14:02:32  JVMDUMP039I Processing dump event "gpf", detail "" at 2022/01/24 11:02:16 - please wait.
14:02:32  JVMDUMP032I JVM requested System dump using '/home/jenkins/workspace/Test_openjdk18_j9_sanity.openjdk_x86-64_linux_Personal/aqa-tests/TKG/output_16430499373839/jdk_lang_0/work/scratch/core.20220124.110216.2162780.0001.dmp' in response to an event
14:02:32  JVMDUMP010I System dump written to /home/jenkins/workspace/Test_openjdk18_j9_sanity.openjdk_x86-64_linux_Personal/aqa-tests/TKG/output_16430499373839/jdk_lang_0/work/scratch/core.20220124.110216.2162780.0001.dmp
14:02:32  JVMDUMP032I JVM requested Java dump using '/home/jenkins/workspace/Test_openjdk18_j9_sanity.openjdk_x86-64_linux_Personal/aqa-tests/TKG/output_16430499373839/jdk_lang_0/work/scratch/javacore.20220124.110216.2162780.0002.txt' in response to an event
14:02:32  JVMDUMP010I Java dump written to /home/jenkins/workspace/Test_openjdk18_j9_sanity.openjdk_x86-64_linux_Personal/aqa-tests/TKG/output_16430499373839/jdk_lang_0/work/scratch/javacore.20220124.110216.2162780.0002.txt
14:02:32  JVMDUMP032I JVM requested Snap dump using '/home/jenkins/workspace/Test_openjdk18_j9_sanity.openjdk_x86-64_linux_Personal/aqa-tests/TKG/output_16430499373839/jdk_lang_0/work/scratch/Snap.20220124.110216.2162780.0003.trc' in response to an event
14:02:32  JVMDUMP010I Snap dump written to /home/jenkins/workspace/Test_openjdk18_j9_sanity.openjdk_x86-64_linux_Personal/aqa-tests/TKG/output_16430499373839/jdk_lang_0/work/scratch/Snap.20220124.110216.2162780.0003.trc
14:02:32  JVMDUMP032I JVM requested JIT dump using '/home/jenkins/workspace/Test_openjdk18_j9_sanity.openjdk_x86-64_linux_Personal/aqa-tests/TKG/output_16430499373839/jdk_lang_0/work/scratch/jitdump.20220124.110216.2162780.0004.dmp' in response to an event
14:02:32  JVMDUMP051I JIT dump occurred in 'MainThread' thread 0x00000000001D7400
14:02:32  JVMDUMP010I JIT dump written to /home/jenkins/workspace/Test_openjdk18_j9_sanity.openjdk_x86-64_linux_Personal/aqa-tests/TKG/output_16430499373839/jdk_lang_0/work/scratch/jitdump.20220124.110216.2162780.0004.dmp
14:02:32  JVMDUMP013I Processed dump event "gpf", detail "".

Related: #13946

EricYangIBM commented 2 years ago

Java stack:

at java/lang/String.formatted(String.java:4960)
at IsRecordTest.testIndirectSubClass(IsRecordTest.java:110)
at java/lang/invoke/LambdaForm$DMH/0x000000002810f670.invokeVirtual(LambdaForm$DMH)
at java/lang/invoke/LambdaForm$MH/0x0000000028119960.invoke(LambdaForm$MH)
at java/lang/invoke/LambdaForm$MH/0x000000002800c440.invokeExact_MT(LambdaForm$MH)
at jdk/internal/reflect/DirectMethodHandleAccessor.invokeImpl(DirectMethodHandleAccessor.java:158)
at jdk/internal/reflect/DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:104)
at java/lang/reflect/Method.invoke(Method.java:577)
at org/testng/internal/MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:132)
at org/testng/internal/TestInvoker.invokeMethod(TestInvoker.java:599)

Failing thread:

> !j9vmthread 0x00000000001D7400
J9VMThread at 0x1d7400 {
  Fields for J9VMThread:
        0x0: struct JNINativeInterface_* functions = !jninativeinterface_ 0x00007F2AABAC0AA0
        0x8: struct J9JavaVM* javaVM = !j9javavm 0x00007F2AAC02DF90
        0x10: UDATA* arg0EA = !j9x 0x000000000022F330
        0x18: UDATA* bytecodes = !j9x 0x0000000000000000
        0x20: UDATA* sp = !j9x 0x000000000022F2A0
        0x28: U8* pc = !j9x 0x00007F2A55478114 // "�v&9����!"
        0x30: struct J9Method* literals = !j9method 0x000000000004EC00 // java/lang/String.formatted([Ljava/lang/Object;)Ljava/lang/String;
        0x38: UDATA jitStackFrameFlags = 0x0000000000000000 (0)
        0x40: struct J9Object* jitException = !j9object 0x0000000000000000
        0x48: struct J9Object* currentException = !j9object 0x0000000000000000
        0x50: UDATA* stackOverflowMark = !j9x 0xFFFFFFFFFFFFFFFF
        0x58: UDATA* stackOverflowMark2 = !j9x 0x000000000022B718
        0x60: U8* heapAlloc = !j9x 0x00000000FFF29338 // ""
        0x68: U8* heapTop = !j9x 0x00000000FFF42EA0 // ""
        0x70: IDATA tlhPrefetchFTA = 0xFFFFFFFFFFFFF768 (-2200)
        0x78: U8* nonZeroHeapAlloc = !j9x 0x0000000000000000
        0x80: U8* nonZeroHeapTop = !j9x 0x0000000000000000
        0x88: IDATA nonZeroTlhPrefetchFTA = 0x0000000000000000 (0)
        0x90: struct J9ThreadMonitor* publicFlagsMutex = !j9threadmonitor 0x00007F2AAC255978
        0x98: UDATA publicFlags = 0x0000000000000020 (32)
        0xa0: UDATA compressObjectReferences = 0x0000000000000001 (1)
        0xa8: struct J9Object* threadObject = !j9object 0x00000000E00CE5F0 // java/lang/Thread
        0xb0: void* lowTenureAddress = !j9x 0x00000000E0000000
        0xb8: void* highTenureAddress = !j9x 0x00000000E0600000
        0xc0: void* heapBaseForActiveCardTable = !j9x 0x0000000000000000
        0xc8: void* activeCardTableBase = !j9x 0x00007F2AA764A000
        0xd0: UDATA heapSizeForActiveCardTable = 0x0000000000000000 (0)
        0xd8: void* heapBaseForBarrierRange0 = !j9x 0x00000000E0000000
        0xe0: UDATA heapSizeForBarrierRange0 = 0x0000000000600000 (6291456)
        0xe8: UDATA* jniLocalReferences = !j9x 0x0000000000000000
        0xf0: UDATA tempSlot = 0x00007F2A8E918640 (139820757255744)
        0xf8: void* jitReturnAddress = !j9x 0x00007F2AA93F7EFD
        0x100: void* floatTemp1 = !j9x 0x00000000FFF28F88
        0x108: void* floatTemp2 = !j9x 0x00007F2A8E917FF8
        0x110: void* floatTemp3 = !j9x 0x00007F2A8E917224
        0x118: void* floatTemp4 = !j9x 0x00000000FFF28600
        0x120: UDATA returnValue = 0xFFFFFFFFFFFFFAB6 (-1354)
        0x128: UDATA returnValue2 = 0x0000000000000001 (1)
        0x130: UDATA* objectFlagSpinLockAddress = !j9x 0x0000000000000000
        0x138: struct J9JavaStack* stackObject = !j9javastack 0x0000000000228F18
        0x140: struct J9Thread* osThread = !j9thread 0x00007F2AAC328338
        0x148: UDATA inspectionSuspendCount = 0x0000000000000000 (0)
        0x150: UDATA inspectorCount = 0x0000000000000000 (0)
        0x158: U32 eventFlags = 0x00000010 (16)
        0x15c: U32 osrFrameIndex = 0x00000000 (0)
        0x160: void* codertTOC = !j9x 0x0000000000000000
        0x168: U8* cardTableVirtualStart = !j9x 0x00007F2AA6F4A000 // "ated/VM_ArrayCopyHelpersFullPointer;
"
        0x170: struct J9Object* allocateObjectSavePrivate1 = !j9object 0x0000000000000000
        0x178: struct J9Object* allocateObjectSavePrivate2 = !j9object 0x0000000000000000
        0x180: struct J9Object* stopThrowable = !j9object 0x0000000000000000
        0x188: struct J9Object* outOfMemoryError = !j9object 0x00000000E00CE6D8 // java/lang/OutOfMemoryError
        0x190: UDATA* jniCurrentReference = !j9x 0x0000000000000000
        0x198: UDATA* jniLimitReference = !j9x 0x0000000000000000
        0x1a0: struct J9VMThread* linkNext = !j9vmthread 0x0000000000016F00
        0x1a8: struct J9VMThread* linkPrevious = !j9vmthread 0x0000000000208200
        0x1b0: UDATA privateFlags = 0x0000000000000000 (0)
        0x1b8: UDATA jitTOC = 0x0000000000000000 (0)
        0x1c0: UDATA ferReturnType = 0x0000000000000000 (0)
        0x1c8: U64 ferReturnValue = 0x0000000000000000 (0)
        0x1d0: U64 mgmtBlockedTimeTotal = 0x0000000000000000 (0)
        0x1d8: U64 mgmtBlockedTimeStart = 0x0000000000000000 (0)
        0x1e0: U64 mgmtWaitedTimeTotal = 0x0000000000000000 (0)
        0x1e8: U64 mgmtWaitedTimeStart = 0x0000000000000000 (0)
        0x1f0: UDATA jniVMAccessCount = 0x0000000000000000 (0)
        0x1f8: UDATA debugEventData1 = 0x0000000000000000 (0)
        0x200: UDATA debugEventData2 = 0x0000000000000000 (0)
        0x208: UDATA debugEventData3 = 0x0000000000000000 (0)
        0x210: UDATA debugEventData4 = 0x0000000000000000 (0)
        0x218: UDATA debugEventData5 = 0x0000000000000000 (0)
        0x220: UDATA debugEventData6 = 0x0000000000000000 (0)
        0x228: UDATA debugEventData7 = 0x0000000000000000 (0)
        0x230: UDATA debugEventData8 = 0x0000000000000000 (0)
        0x238: struct J9StackElement* classLoadingStack = !j9stackelement 0x0000000000000000
        0x240: UDATA jitTransitionJumpSlot = 0x0000000000000000 (0)
        0x248: struct J9ThreadMonitor* gcClassUnloadingMutex = !j9threadmonitor 0x0000000000000000
        0x250: struct J9VMThread* gcClassUnloadingThreadPrevious = !j9vmthread 0x0000000000000000
        0x258: struct J9VMThread* gcClassUnloadingThreadNext = !j9vmthread 0x0000000000000000
        0x260: struct J9StackWalkState* stackWalkState = !j9stackwalkstate 0x00000000001D7710
        0x268: struct J9VMEntryLocalStorage* entryLocalStorage = !j9vmentrylocalstorage 0x00007F2A549BEB70
        0x270: UDATA gpProtected = 0x0000000000000001 (1)
        0x278: struct J9VMGCSublistFragment gcRememberedSet = !j9vmgcsublistfragment 0x00000000001D7678
        0x2a8: struct MM_GCRememberedSetFragment sATBBarrierRememberedSetFragment = !mm_gcrememberedsetfragment 0x00000000001D76A8
        0x2d8: void* gcTaskListPtr = !j9x 0x0000000000000000
        0x2e0: UDATA* dropBP = !j9x 0x0000000000000000
        0x2e8: UDATA dropFlags = 0x0000000000000000 (0)
        0x2f0: struct J9Pool* monitorEnterRecordPool = !j9pool 0x00007F2AAC32DF00
        0x2f8: struct J9MonitorEnterRecord* monitorEnterRecords = !j9monitorenterrecord 0x0000000000000000
        0x300: UDATA* jniArrayCache = !j9x 0x00007F2AAC25F350
        0x308: UDATA* jniArrayCache2 = !j9x 0x0000000000000000
        0x310: struct J9StackWalkState inlineStackWalkState = !j9stackwalkstate 0x00000000001D7710
        0x538: struct J9JITDecompilationInfo* decompilationStack = !j9jitdecompilationinfo 0x0000000000000000
        0x540: struct J9ModronThreadLocalHeap allocateThreadLocalHeap = !j9modronthreadlocalheap 0x00000000001D7940
        0x570: struct J9ModronThreadLocalHeap nonZeroAllocateThreadLocalHeap = !j9modronthreadlocalheap 0x00000000001D7970
        0x5a0: void* sidecarEvent = !j9x 0x0000000000000000
        0x5a8: struct PortlibPTBuffers_struct* ptBuffers = !portlibptbuffers_ 0x0000000000000000
        0x5b0: struct J9Object* blockingEnterObject = !j9object 0x0000000000000000
        0x5b8: void* gcExtensions = !j9x 0x00007F2AAC3629A8
        0x5c0: void* gpInfo = !j9x 0x000000000022EC70
        0x5c8: void* jitVMwithThreadInfo = !j9x 0x00007F2A280033B0
        0x5d0: U8* profilingBufferEnd = !j9x 0x00007F2A280031C0 // "!Ce�;|ń"
        0x5d8: U8* profilingBufferCursor = !j9x 0x00007F2A28002FBD // ""
        0x5e0: UDATA* j2iFrame = !j9x 0x0000000000000000
        0x5e8: UDATA currentOSStackFree = 0x000000000003CBD0 (248784)
        0x5f0: UDATA mgmtBlockedCount = 0x0000000000000000 (0)
        0x5f8: UDATA mgmtWaitedCount = 0x0000000000000000 (0)
        0x600: UDATA mgmtBlockedStart = 0x0000000000000000 (0)
        0x608: UDATA mgmtWaitedStart = 0x0000000000000000 (0)
        0x610: UDATA cardTableShiftSize = 0x0000000000000009 (9)
        0x618: void* aotVMwithThreadInfo = !j9x 0x00007F2A28004450
        0x620: UDATA asyncEventFlags = 0x0000000000000004 (4)
        0x628: struct J9Object* forceEarlyReturnObjectSlot = !j9object 0x0000000000000000
        0x630: struct J9MonitorEnterRecord* jniMonitorEnterRecords = !j9monitorenterrecord 0x0000000000000000
        0x638: struct J9DLTInformationBlock dltBlock = !j9dltinformationblock 0x00000000001D7A38
        0x7f8: struct J9VMGCSegregatedAllocationCacheEntry* segregatedAllocationCache = !j9vmgcsegregatedallocationcacheentry 0x0000000000000000
        0x800: struct J9StackWalkState* activeWalkState = !j9stackwalkstate 0x0000000000000000
        0x808: void* jniCalloutArgs = !j9x 0x0000000000000000
        0x810: struct J9VMThread* exclusiveVMAccessQueueNext = !j9vmthread 0x0000000000000000
        0x818: struct J9VMThread* exclusiveVMAccessQueuePrevious = !j9vmthread 0x0000000000000000
        0x820: struct J9Object* javaLangThreadLocalCache = !j9object 0x0000000000000000
        0x828: UDATA jitCountDelta = 0x0000000000000002 (2)
        0x830: UDATA maxProfilingCount = 0x00000000000007D1 (2001)
        0x838: j9objectmonitor_t[] objectMonitorLookupCache = !j9x 0x00000000001D7C38
        0x938: UDATA jniCriticalCopyCount = 0x0000000000000000 (0)
        0x940: UDATA jniCriticalDirectCount = 0x0000000000000000 (0)
        0x948: struct J9Pool* jniReferenceFrames = !j9pool 0x0000000000000000
        0x950: U32 ludclInlineDepth = 0x00000000 (0)
        0x954: U32 ludclBPOffset = 0x00000000 (0)
        0x958: UDATA* osrJittedFrameCopy = !j9x 0x0000000000000000
        0x960: struct J9OSRBuffer* osrBuffer = !j9osrbuffer 0x0000000000000000
        0x968: void* osrReturnAddress = !j9x 0x0000000000000000
        0x970: void* osrScratchBuffer = !j9x 0x0000000000000000
        0x978: void* jitArtifactSearchCache = !j9x 0x00007F2A28004AC0
        0x980: void* jitExceptionHandlerCache = !j9x 0x0000000000000000
        0x988: void* jitPrivateData = !j9x 0x00007F2A54B4E5F0
        0x990: struct J9Method* jitMethodToBeCompiled = !j9method 0x0000000000000000
        0x998: UDATA privateFlags2 = 0x0000000000000000 (0)
        0x9a0: struct OMR_VMThread* omrVMThread = !omr_vmthread 0x00000000001D7E18
        0x9a8: void* gpuInfo = !j9x 0x0000000000000000
        0x9b0: void* startOfMemoryBlock = !j9x 0x00000000001D73C8
        0x9b8: UDATA inNative = 0x0000000000000000 (0)
        0x9c0: struct J9JITDecompilationInfo* lastDecompilation = !j9jitdecompilationinfo 0x0000000000000000
        0x9c8: struct J9GSParameters gsParameters = !j9gsparameters 0x00000000001D7DC8
        0x9d8: UDATA readBarrierRangeCheckBase = 0xFFFFFFFFFFFFFFFF (-1)
        0x9e0: UDATA readBarrierRangeCheckTop = 0x0000000000000000 (0)
        0x9e8: U32 readBarrierRangeCheckBaseCompressed = 0xFFFFFFFF (4294967295)
        0x9ec: U32 readBarrierRangeCheckTopCompressed = 0x00000000 (0)
        0x9f0: UDATA safePointCount = 0x0000000000000000 (0)
        0x9f8: volatile struct J9HashTable* utfCache = !j9hashtable 0x00007F2A2801F9B0
        0xa00: U64* ffiArgs = !j9x 0x0000000000000000
        0xa08: UDATA ffiArgCount = 0x0000000000000000 (0)
}

Stackslots:

> !stackslots 0x00000000001D7400
<1d7400> *** BEGIN STACK WALK, flags = 00400001 walkThread = 0x00000000001D7400 ***
<1d7400>        ITERATE_O_SLOTS
<1d7400>        RECORD_BYTECODE_PC_OFFSET
<1d7400> Initial values: walkSP = 0x000000000022F2A0, PC = 0x00007F2A55478114, literals = 0x000000000004EC00, A0 = 0x000000000022F330, j2iFrame = 0x0000000000000000, ELS = 0x00007F2A549BEB70, decomp = 0x0000000000000000
<1d7400> Bytecode frame: bp = 0x000000000022F320, sp = 0x000000000022F2A0, pc = 0x00007F2A55478114, cp = 0x000000000004A730, arg0EA = 0x000000000022F330, flags = 0x0000000000000000
<1d7400>        Method: java/lang/String.formatted([Ljava/lang/Object;)Ljava/lang/String; !j9method 0x000000000004EC00
<1d7400>        Bytecode index = 12
<1d7400>        Using local mapper
<1d7400>        Locals starting at 0x000000000022F330 for 0x0000000000000002 slots
<1d7400>                I-Slot: a0[0x000000000022F330] = 0x00000000E01AAE78
<1d7400>                I-Slot: a1[0x000000000022F328] = 0x00000000FFF28F68
Jan 24, 2022 1:12:39 PM com.ibm.j9ddr.vm29.events.DefaultEventListener corruptData
WARNING: CorruptData encountered iterating o-slots. walkThread = 0x00000000001D7400
com.ibm.j9ddr.CorruptDataException: Operand stack underflow in StackMap

<1d7400> Bytecode frame: bp = 0x000000000022F350, sp = 0x000000000022F338, pc = 0x00007F2A5491BC4F, cp = 0x000000000021B190, arg0EA = 0x000000000022F3C8, flags = 0x0000000000000000
<1d7400>        Method: IsRecordTest.testIndirectSubClass(ZZZZZ)V !j9method 0x000000000021B948
<1d7400>        Bytecode index = 31
<1d7400>        Using local mapper
<1d7400>        Locals starting at 0x000000000022F3C8 for 0x000000000000000F slots
<1d7400>                O-Slot: a0[0x000000000022F3C8] = 0x00000000FFD553A8
<1d7400>                I-Slot: a1[0x000000000022F3C0] = 0x0000000000000000
<1d7400>                I-Slot: a2[0x000000000022F3B8] = 0x0000000000000000
<1d7400>                I-Slot: a3[0x000000000022F3B0] = 0x0000000000000001
<1d7400>                I-Slot: a4[0x000000000022F3A8] = 0x0000000000000001
<1d7400>                I-Slot: a5[0x000000000022F3A0] = 0x0000000000000000
<1d7400>                I-Slot: t6[0x000000000022F398] = 0x00000000FFF247D0
<1d7400>                I-Slot: t7[0x000000000022F390] = 0x0000000000357BC8
<1d7400>                I-Slot: t8[0x000000000022F388] = 0x0000000000000000
<1d7400>                I-Slot: t9[0x000000000022F380] = 0x00000000E005F840
<1d7400>                I-Slot: t10[0x000000000022F378] = 0x00007F2AA98B56B0
<1d7400>                I-Slot: t11[0x000000000022F370] = 0x0000000000000206
<1d7400>                I-Slot: t12[0x000000000022F368] = 0x0000000000000246
<1d7400>                I-Slot: t13[0x000000000022F360] = 0x00007F2A557C134C
<1d7400>                I-Slot: t14[0x000000000022F358] = 0x0000000000000202
<1d7400>        Pending stack starting at 0x000000000022F338 for UDATA(0x0000000000000001) slots
<1d7400>                O-Slot: p0[0x000000000022F338] = 0x00000000E004FEE8
<1d7400> Bytecode frame: bp = 0x000000000022F3E0, sp = 0x000000000022F3D0, pc = 0x00007F2A2810F7FC, cp = 0x000000000034F4C0, arg0EA = 0x000000000022F420, flags = 0x0000000000000000
EricYangIBM commented 2 years ago

Core files: https://ibm.box.com/s/b4lq6ufajqx5spxo3vamnm6vj6m2hc7s

EricYangIBM commented 2 years ago

Native stack:

(gdb) bt
#0  0x00007fed4fca9aa1 in pthread_kill () from /lib64/libpthread.so.0
#1  0x00007fed4e1e2f4d in omrdump_create (portLibrary=0x7fed4f4b43a0 <hashTableAddNodeInList+128>, 
    filename=0x7fed0422ef30 "/home/jenkins/workspace/Grinder/aqa-tests/TKG/output_16431411604513/jdk_lang_1/work/scratch/0/", dumpType=<optimized out>, 
    userData=<optimized out>) at /home/jenkins/workspace/Build_JDK18_x86-64_linux_Personal/omr/port/unix/omrosdump.c:188
#2  0x00007fed4d73c4e2 in doSystemDump (agent=0x7fed48030f10, 
    label=0x7fed0422ef30 "/home/jenkins/workspace/Grinder/aqa-tests/TKG/output_16431411604513/jdk_lang_1/work/scratch/0/", context=0x7fed0422f3f0)
    at /home/jenkins/workspace/Build_JDK18_x86-64_linux_Personal/openj9/runtime/rasdump/dmpagent.c:747
#3  0x00007fed4d7382d5 in protectedDumpFunction (portLibrary=portLibrary@entry=0x7fed4f4b43a0 <hashTableAddNodeInList+128>, 
    userData=userData@entry=0x7fed0422ee90) at /home/jenkins/workspace/Build_JDK18_x86-64_linux_Personal/openj9/runtime/rasdump/dmpagent.c:2843
#4  0x00007fed4e1e4a13 in omrsig_protect (portLibrary=0x7fed4f4b43a0 <hashTableAddNodeInList+128>, fn=0x7fed4d7382c0 <protectedDumpFunction>, 
    fn_arg=0x7fed0422ee90, handler=0x7fed4d7382e0 <signalHandler>, handler_arg=0x0, flags=505, result=0x7fed0422ee88)
    at /home/jenkins/workspace/Build_JDK18_x86-64_linux_Personal/omr/port/unix/omrsignal.c:425
#5  0x00007fed4d73b94b in runDumpFunction (agent=<optimized out>, 
    label=0x7fed0422ef30 "/home/jenkins/workspace/Grinder/aqa-tests/TKG/output_16431411604513/jdk_lang_1/work/scratch/0/", context=<optimized out>)
    at /home/jenkins/workspace/Build_JDK18_x86-64_linux_Personal/openj9/runtime/rasdump/dmpagent.c:2821
#6  0x00007fed4d73badf in runDumpAgent (vm=vm@entry=0x7fed4801a4c0, agent=agent@entry=0x7fed48030f10, context=context@entry=0x7fed0422f3f0, 
    state=state@entry=0x7fed0422f3e8, detail=detail@entry=0x7fed0422f470 "", timeNow=timeNow@entry=1643141578604)
    at /home/jenkins/workspace/Build_JDK18_x86-64_linux_Personal/openj9/runtime/rasdump/dmpagent.c:2751
#7  0x00007fed4d75430e in triggerDumpAgents (vm=0x7fed4801a4c0, self=0x7fed484b4900, eventFlags=8192, eventData=<optimized out>)
    at /home/jenkins/workspace/Build_JDK18_x86-64_linux_Personal/openj9/runtime/rasdump/trigger.c:1012
#8  0x00007fed4e486df2 in generateDiagnosticFiles (portLibrary=portLibrary@entry=0x7fed4f4b43a0 <hashTableAddNodeInList+128>, 
    userData=userData@entry=0x7fed0422f950) at /home/jenkins/workspace/Build_JDK18_x86-64_linux_Personal/openj9/runtime/vm/gphandle.c:1159
#9  0x00007fed4e1e4a13 in omrsig_protect (portLibrary=0x7fed4f4b43a0 <hashTableAddNodeInList+128>, fn=0x7fed4e486d10 <generateDiagnosticFiles>, 
    fn_arg=0x7fed0422f950, handler=0x7fed4e4863d0 <recursiveCrashHandler>, handler_arg=0x7fed0422f920, flags=505, result=0x7fed0422f918)
    at /home/jenkins/workspace/Build_JDK18_x86-64_linux_Personal/omr/port/unix/omrsignal.c:425
#10 0x00007fed4e487005 in vmSignalHandler (portLibrary=0x7fed4f4b43a0 <hashTableAddNodeInList+128>, gpType=24, gpInfo=<optimized out>, userData=<optimized out>)
    at /home/jenkins/workspace/Build_JDK18_x86-64_linux_Personal/openj9/runtime/vm/gphandle.c:833
#11 0x00007fed4e57c7e5 in swapStacksAndRunHandler ()
    at /home/jenkins/workspace/Build_JDK18_x86-64_linux_Personal/build/linux-x86_64-server-release/vm/runtime/vm/stackswap.s:144
#12 0x00007fed484b4900 in ?? ()
#13 0x00007fed04230b70 in ?? ()
#14 0x00007fed0757a0a8 in ?? ()
#15 0x00007fed40fa7f68 in ?? ()
#16 0x00007fed482ae200 in ?? ()
#17 0x00007fed07579f70 in ?? ()
#18 0x00007fed07579f6b in ?? ()
#19 0x00007fed07579f70 in ?? ()
#20 0x00007fed4e5febca in j9_void_void () from /home/jenkins/eric/jdk/lib/default/libj9vm29.so
#21 0x00007fed00000000 in ?? ()
#22 0x00007fecac069f28 in ?? ()
#23 0x0000000000000006 in ?? ()
#24 0x0000000000000049 in ?? ()
#25 0x0000000000000045 in ?? ()
#26 0x00007fed40fa7f10 in ?? ()
#27 0x0000000000000006 in ?? ()
#28 0x0000000000000002 in ?? ()
#29 0x00007fed40fa7f68 in ?? ()
#30 0x0000000000000005 in ?? ()
#31 0x0064006e00490074 in ?? ()
#32 0x0000000000000000 in ?? ()
EricYangIBM commented 2 years ago

Maybe this is related to https://github.com/eclipse-openj9/openj9/issues/14124?

babsingh commented 2 years ago

Maybe this is related to #14124?

The above diagnostic information is not sufficient to indicate that this issue is the same as #14124. In the native stack trace, debug symbols for key functions are missing. Also, the segfault happens in libj9jit29.so but the stackslots do not have JIT frames. Not sure if it is a VM or JIT issue; thus, added comp:vm + triageRequired labels.

In #14124, the key symptom is <unknown>+0x0 in the stack backtrace, which is not seen in this issue.

17:33:29  ----------- Stack Backtrace -----------
...
17:33:29  (0x000003FF81AB24E4 [libj9prt29.so+0x324e4])
17:33:29  (0x000003FF829FDE90 [<unknown>+0x0])

More triage is required to identify the source/cause of this failure.

tajila commented 2 years ago

@EricYangIBM can you reproduce this with -Xint?

babsingh commented 2 years ago

https://github.com/eclipse-openj9/openj9/issues/14354#issuecomment-1020555710: String.formatted is being run in the top frame. The extra field in this J9Method is set which indicates the method was compiled at one point. During the segfault, it has a bytecode frame. Potentially, a sign of OSR transition, which may have caused the O-slot corruption.

> !j9method 0x000000000004EC00
J9Method at 0x4ec00 {
  Fields for J9Method:
    0x0: U8* bytecodes = !j9x 0x00007F2A55478108 // "�sY�t*+�u�v&9����!"
    0x8: struct J9ConstantPool* constantPool = !j9constantpool 0x000000000004A730 (flags = 0x0)
    0x10: void* methodRunAddress = !j9x 0x0000000000000006
    0x18: volatile void* extra = !j9x 0x0000000000000799
}
Signature: java/lang/String.formatted([Ljava/lang/Object;)Ljava/lang/String; !bytecodes 0x000000000004EC00
ROM Method: !j9rommethod 0x00007F2A554780F4
Next Method: !j9method 0x000000000004EC20
EricYangIBM commented 2 years ago

It doesn't seem to crash with -Xint: https://hyc-runtimes-jenkins.swg-devops.com/job/Grinder/20490/ Without -Xint this failure happens fairly frequently, maybe half of all iterations.

pshipton commented 2 years ago

I didn't see this in a build of the latest https://openj9-jenkins.osuosl.org/job/Pipeline-Build-Test-JDK18/13/ Closing assuming it's a dup of https://github.com/eclipse-openj9/openj9/issues/14124