dragonwell-project / dragonwell11

Alibaba Dragonwell11 JDK
https://www.aliyun.com/product/dragonwell
GNU General Public License v2.0
557 stars 112 forks source link

[11.0.21.18.9][lilliput]release版本x64平台vmTestbase/nsk/monitoring/stress/lowmem/lowmem007/TestDescription.java小概率随机crash:G1ParEvacuateFollowersClosure::do_void() #762

Open sendaoYan opened 8 months ago

sendaoYan commented 8 months ago

job:https://tone.aliyun-inc.com/ws/xesljfzh/test_result/263375?tab=1

Description release版本x64平台vmTestbase/nsk/monitoring/stress/lowmem/lowmem007/TestDescription.java小概率随机crash:G1ParEvacuateFollowersClosure::do_void()

# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00007f6f227a53df, pid=2198740, tid=2198848
#
# JRE version: OpenJDK Runtime Environment (Alibaba Dragonwell Extended Edition)-11.0.21.18+9-GA (11.0.21.18+9) (build 11.0.21.18+9)
# Java VM: OpenJDK 64-Bit Server VM (Alibaba Dragonwell Extended Edition)-11.0.21.18+9-GA (11.0.21.18+9, mixed mode, tiered, compressed oops, g1 gc, linux-amd64)
# Problematic frame:
# V  [libjvm.so+0x7a53df]  G1ParEvacuateFollowersClosure::do_void()+0x47f
#
# Core dump will be written. Default location: /tmp/tone/run/jtreg/jt-work/hotspot_jtreg/vmTestbase/nsk/monitoring/stress/lowmem/lowmem007/TestDescription/core.2198740
#
# If you would like to submit a bug report, please visit:
#   mailto:dragonwell_use@googlegroups.com

Steps to Reproduce Steps to reproduce the behavior:

export test=test/hotspot/jtreg/vmTestbase/nsk/monitoring/stress/lowmem/lowmem007/TestDescription.java
function runJtreg() { jtreg -ea -esa -timeoutFactor:4 -v:fail,error,time,nopass -nr -w $dir/index-$1 $test &> $dir/$1.log ; if [[ 0 -ne $? ]] ; then echo -n "$1 " ; else rm -rf $dir/index-$1 $dir/$1.log ; fi ; } ; export -f runJtreg ; export dir="tmp-jtreg-"`basename ${test##* } .java` ; rm -rf $dir ; mkdir -p $dir ; time seq 10000 | xargs -i -n 1 -P 1 bash -c "
runJtreg {}" ; echo total fail number: `ls $dir/*.log 2> /dev/null | wc | awk '{print $1}'`

Expected behavior all passed

tone.log

hs_err_pid2198740.log

JDK version

# uname -a ; cat /etc/os-release ; free -h ; lscpu | head -n 25 ; java -version ; java -Xinternalversion
Linux iZbp1go2piwffcpjzwyemxZ 5.10.134-15.an8.x86_64 #1 SMP Thu Jul 20 00:35:47 CST 2023 x86_64 x86_64 x86_64 GNU/Linux
NAME="Anolis OS"
VERSION="8.8"
ID="anolis"
ID_LIKE="rhel fedora centos"
VERSION_ID="8.8"
PLATFORM_ID="platform:an8"
PRETTY_NAME="Anolis OS 8.8"
ANSI_COLOR="0;31"
HOME_URL="https://openanolis.cn/"

              total        used        free      shared  buff/cache   available
Mem:           60Gi        15Gi        43Gi       2.0Mi       1.7Gi        44Gi
Swap:            0B          0B          0B
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              32
On-line CPU(s) list: 0-31
Thread(s) per core:  2
Core(s) per socket:  16
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
BIOS Vendor ID:      Alibaba Cloud
CPU family:          6
Model:               106
Model name:          Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
BIOS Model name:     pc-i440fx-2.1
Stepping:            6
CPU MHz:             3498.388
BogoMIPS:            5399.99
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           48K
L1i cache:           32K
L2 cache:            1280K
L3 cache:            49152K
NUMA node0 CPU(s):   0-31
openjdk version "11.0.21.18" 2023-10-17
OpenJDK Runtime Environment (Alibaba Dragonwell Extended Edition)-11.0.21.18+9-GA (build 11.0.21.18+9)
OpenJDK 64-Bit Server VM (Alibaba Dragonwell Extended Edition)-11.0.21.18+9-GA (build 11.0.21.18+9, mixed mode)
OpenJDK 64-Bit Server VM (11.0.21.18+9) for linux-amd64 JRE (11.0.21.18+9), built on Dec 27 2023 07:48:26 by "" with gcc 7.5.0
sendaoYan commented 8 months ago

二进制链接:

BINARY_URL=https://dragonwell.oss-cn-shanghai.aliyuncs.com/11.0.21.18.9/Alibaba_Dragonwell_Extended_11.0.21.18.9_x64_linux.tar.gz
TEST_IMAGE=https://dragonwell.oss-cn-shanghai.aliyuncs.com/11.0.21.18.9/Alibaba_Dragonwell_Extended_11.0.21.18.9_x64_linux-testimage.tar.gz

ip-list-temurin.txt ip-list-Xmixed.txt ip-list-UseCompactObjectHeaders.txt

从昨天晚上到现在,开启和关闭UseCompactObjectHeaders总共分别运行了1.5w该用例,均未复现该问题

sendaoYan commented 8 months ago

2024年01月03日14:30:44复现一次(总共大概运行了2w次):

887.log hs_err_pid472879.log

115.29.214.196

# uname -a ; cat /etc/os-release ; free -h ; lscpu | head -n 25 ; java -version ; java -Xinternalversion
Linux iZbp1igy5idwfm60r53sb7Z 5.10.134-15.an8.x86_64 #1 SMP Thu Jul 20 00:35:47 CST 2023 x86_64 x86_64 x86_64 GNU/Linux
NAME="Anolis OS"
VERSION="8.8"
ID="anolis"
ID_LIKE="rhel fedora centos"
VERSION_ID="8.8"
PLATFORM_ID="platform:an8"
PRETTY_NAME="Anolis OS 8.8"
ANSI_COLOR="0;31"
HOME_URL="https://openanolis.cn/"

              total        used        free      shared  buff/cache   available
Mem:           60Gi       980Mi        54Gi       2.0Mi       4.9Gi        58Gi
Swap:            0B          0B          0B
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              32
On-line CPU(s) list: 0-31
Thread(s) per core:  2
Core(s) per socket:  16
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
BIOS Vendor ID:      Alibaba Cloud
CPU family:          6
Model:               106
Model name:          Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
BIOS Model name:     pc-i440fx-2.1
Stepping:            6
CPU MHz:             3499.993
BogoMIPS:            5399.99
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           48K
L1i cache:           32K
L2 cache:            1280K
L3 cache:            49152K
NUMA node0 CPU(s):   0-31
openjdk version "11.0.21.18" 2023-10-17
OpenJDK Runtime Environment (Alibaba Dragonwell Extended Edition)-11.0.21.18+9-GA (build 11.0.21.18+9)
OpenJDK 64-Bit Server VM (Alibaba Dragonwell Extended Edition)-11.0.21.18+9-GA (build 11.0.21.18+9, mixed mode)
OpenJDK 64-Bit Server VM (11.0.21.18+9) for linux-amd64 JRE (11.0.21.18+9), built on Dec 27 2023 07:48:26 by "" with gcc 7.5.0
sendaoYan commented 7 months ago

yitian总共跑了大概1.5w,没有复现 x64 alinux3总共跑了大概2.5w次,复现了3次

# uname -a ; cat /etc/os-release ; free -h ; lscpu | head -n 25 ; java -version ; java -Xinternalversion
Linux iZbp1hsfceg13a47ez2dazZ 5.10.134-15.al8.x86_64 #1 SMP Thu Jul 20 00:44:04 CST 2023 x86_64 x86_64 x86_64 GNU/Linux
NAME="Alibaba Cloud Linux"
VERSION="3 (Soaring Falcon)"
ID="alinux"
ID_LIKE="rhel fedora centos anolis"
VERSION_ID="3"
PLATFORM_ID="platform:al8"
PRETTY_NAME="Alibaba Cloud Linux 3 (Soaring Falcon)"
ANSI_COLOR="0;31"
HOME_URL="https://www.aliyun.com/"

              total        used        free      shared  buff/cache   available
Mem:          247Gi        16Gi       208Gi       2.0Mi        21Gi       228Gi
Swap:            0B          0B          0B
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              32
On-line CPU(s) list: 0-31
Thread(s) per core:  2
Core(s) per socket:  16
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
BIOS Vendor ID:      Alibaba Cloud
CPU family:          6
Model:               106
Model name:          Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
BIOS Model name:     pc-i440fx-2.1
Stepping:            6
CPU MHz:             3484.701
BogoMIPS:            5399.99
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           48K
L1i cache:           32K
L2 cache:            1280K
L3 cache:            49152K
NUMA node0 CPU(s):   0-31
openjdk version "11.0.21.18" 2023-10-17
OpenJDK Runtime Environment (Alibaba Dragonwell Extended Edition)-11.0.21.18+9-GA (build 11.0.21.18+9)
OpenJDK 64-Bit Server VM (Alibaba Dragonwell Extended Edition)-11.0.21.18+9-GA (build 11.0.21.18+9, mixed mode)
OpenJDK 64-Bit Server VM (11.0.21.18+9) for linux-amd64 JRE (11.0.21.18+9), built on Dec 27 2023 07:48:26 by "" with gcc 7.5.0

hs_err_pid46383.log

sendaoYan commented 7 months ago

倚天运行大概3.7w次,未复现

sendaoYan commented 7 months ago

centos运行2.3w次复现一次:root@121.40.237.66

# uname -a ; cat /etc/os-release ; free -h ; lscpu | head -n 25 ; java -version ; java -Xinternalversion
Linux iZbp1aflbhgkeypurnjg2kZ 4.18.0-348.7.1.el8_5.x86_64 #1 SMP Wed Dec 22 13:25:12 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
NAME="CentOS Linux"
VERSION="8"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="8"
PLATFORM_ID="platform:el8"
PRETTY_NAME="CentOS Linux 8"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:8"
HOME_URL="https://centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"
CENTOS_MANTISBT_PROJECT="CentOS-8"
CENTOS_MANTISBT_PROJECT_VERSION="8"
              total        used        free      shared  buff/cache   available
Mem:          247Gi        24Gi       191Gi       2.0Mi        31Gi       221Gi
Swap:            0B          0B          0B
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              64
On-line CPU(s) list: 0-63
Thread(s) per core:  2
Core(s) per socket:  32
Socket(s):           1
NUMA node(s):        1
Vendor ID:           GenuineIntel
BIOS Vendor ID:      Alibaba Cloud
CPU family:          6
Model:               106
Model name:          Intel(R) Xeon(R) Platinum 8369B CPU @ 2.70GHz
BIOS Model name:     pc-i440fx-2.1
Stepping:            6
CPU MHz:             3021.284
BogoMIPS:            5399.99
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           48K
L1i cache:           32K
L2 cache:            1280K
L3 cache:            49152K
NUMA node0 CPU(s):   0-63
openjdk version "11.0.21.18" 2023-10-17
OpenJDK Runtime Environment (Alibaba Dragonwell Extended Edition)-11.0.21.18+9-GA (build 11.0.21.18+9)
OpenJDK 64-Bit Server VM (Alibaba Dragonwell Extended Edition)-11.0.21.18+9-GA (build 11.0.21.18+9, mixed mode)
OpenJDK 64-Bit Server VM (11.0.21.18+9) for linux-amd64 JRE (11.0.21.18+9), built on Dec 27 2023 07:48:26 by "" with gcc 7.5.0

hs_err_pid403367.log

573.log

sendaoYan commented 7 months ago

ecs.hfg8i.16xlarge这个规格的ecs,跑了大概12w次,目前没有复现

sendaoYan commented 7 months ago

(intel 8代)ecs.hfg8i.16xlarge规格 alinux3运行12w次未复现 (intel)ecs.g7.16xlarge规格,centos、alinux3、anolis8复现概率大概为1/1w (倚天)ecs.c8y.8xlarge规格,alinux3运行10w次未复现

mmyxym commented 7 months ago

目前只有g7 icelake小概率复现,并且sig handler和core显示的error address和上下指令寄存器不一致