adoptium / temurin-build

Eclipse Temurin™ build scripts - common across all releases/versions
Apache License 2.0
1.03k stars 249 forks source link

Cannot create worker GC thread. Out of system resources. #3020

Closed roginluo closed 2 years ago

roginluo commented 2 years ago

What are you trying to do? trying to start skywalking using https://github.com/apache/skywalking.git i hava edited .env file

ES_VERSION=7.4.2
OAP_IMAGE=apache/skywalking-oap-server:latest
UI_IMAGE=apache/skywalking-ui:latest

Expected behaviour: all container started Observed behaviour: apache/skywalking-oap-server:latest cann't start Any other comments:

docker error json log

{"log":"[Entrypoint] Apache SkyWalking Docker Image\n","stream":"stdout","time":"2022-07-02T04:16:58.830391698Z"}
{"log":"+ exec java -Xms4096m -Xmx4096m -classpath '/skywalking/ext-libs/*.jar:oap-libs/zstd-jni-1.4.3-1.jar:oap-libs/zookeeper-jute-3.5.7.jar:oap-libs/zookeeper-3.5.7.jar:oap-libs/zipkin-receiver-plugin-9.1.0.jar:oap-libs/zipkin-query-plugin-9.1.0.jar:oap-libs/zipkin-2.23.16.jar:oap-libs/vavr-match-0.10.3.jar:oap-libs/vavr-0.10.3.jar:oap-libs/tool-profile-snapshot-server-mock-9.1.0.jar:oap-libs/tool-profile-snapshot-bootstrap-9.1.0.jar:oap-libs/telemetry-prometheus-9.1.0.jar:oap-libs/telemetry-api-9.1.0.jar:oap-libs/swagger-annotations-1.6.3.jar:oap-libs/storage-tidb-plugin-9.1.0.jar:oap-libs/storage-jdbc-hikaricp-plugin-9.1.0.jar:oap-libs/storage-elasticsearch-plugin-9.1.0.jar:oap-libs/storage-banyandb-plugin-9.1.0.jar:oap-libs/snappy-java-1.1.7.3.jar:oap-libs/snakeyaml-1.28.jar:oap-libs/slf4j-api-1.7.30.jar:oap-libs/skywalking-zabbix-receiver-plugin-9.1.0.jar:oap-libs/skywalking-trace-receiver-plugin-9.1.0.jar:oap-libs/skywalking-sharing-server-plugin-9.1.0.jar:oap-libs/skywalking-profile-receiver-plugin-9.1.0.jar:oap-libs/skywalking-meter-receiver-plugin-9.1.0.jar:oap-libs/skywalking-mesh-receiver-plugin-9.1.0.jar:oap-libs/skywalking-management-receiver-plugin-9.1.0.jar:oap-libs/skywalking-log-recevier-plugin-9.1.0.jar:oap-libs/skywalking-jvm-receiver-plugin-9.1.0.jar:oap-libs/skywalking-event-receiver-plugin-9.1.0.jar:oap-libs/skywalking-ebpf-receiver-plugin-9.1.0.jar:oap-libs/skywalking-clr-receiver-plugin-9.1.0.jar:oap-libs/skywalking-browser-receiver-plugin-9.1.0.jar:oap-libs/simpleclient_httpserver-0.12.0.jar:oap-libs/simpleclient_hotspot-0.6.0.jar:oap-libs/simpleclient_common-0.6.0.jar:oap-libs/simpleclient-0.6.0.jar:oap-libs/server-starter-9.1.0.jar:oap-libs/server-health-checker-9.1.0.jar:oap-libs/server-core-9.1.0.jar:oap-libs/server-alarm-plugin-9.1.0.jar:oap-libs/retrofit-2.5.0.jar:oap-libs/receiver-proto-9.1.0.jar:oap-libs/reactive-streams-1.0.2.jar:oap-libs/query-graphql-plugin-9.1.0.jar:oap-libs/proto-google-common-protos-2.0.1.jar:oap-libs/protobuf-java-util-3.19.4.jar:oap-libs/protobuf-java-3.19.4.jar:oap-libs/prometheus-fetcher-plugin-9.1.0.jar:oap-libs/postgresql-42.3.3.jar:oap-libs/perfmark-api-0.25.0.jar:oap-libs/otel-receiver-plugin-9.1.0.jar:oap-libs/okio-1.17.2.jar:oap-libs/okhttp-3.14.9.jar:oap-libs/oal-rt-9.1.0.jar:oap-libs/oal-grammar-9.1.0.jar:oap-libs/netty-transport-native-unix-common-4.1.77.Final-linux-x86_64.jar:oap-libs/netty-transport-native-unix-common-4.1.77.Final.jar:oap-libs/netty-transport-native-epoll-4.1.77.Final-linux-x86_64.jar:oap-libs/netty-transport-native-epoll-4.1.77.Final.jar:oap-libs/netty-transport-classes-epoll-4.1.77.Final.jar:oap-libs/netty-transport-4.1.77.Final.jar:oap-libs/netty-tcnative-classes-2.0.52.Final.jar:oap-libs/netty-tcnative-boringssl-static-2.0.52.Final-windows-x86_64.jar:oap-libs/netty-tcnative-boringssl-static-2.0.52.Final-osx-x86_64.jar:oap-libs/netty-tcnative-boringssl-static-2.0.52.Final-osx-aarch_64.jar:oap-libs/netty-tcnative-boringssl-static-2.0.52.Final-linux-x86_64.jar:oap-libs/netty-tcnative-boringssl-static-2.0.52.Final-linux-aarch_64.jar:oap-libs/netty-tcnative-boringssl-static-2.0.52.Final.jar:oap-libs/netty-resolver-dns-native-macos-4.1.77.Final-osx-x86_64.jar:oap-libs/netty-resolver-dns-native-macos-4.1.77.Final-osx-aarch_64.jar:oap-libs/netty-resolver-dns-classes-macos-4.1.77.Final.jar:oap-libs/netty-resolver-dns-4.1.77.Final.jar:oap-libs/netty-resolver-4.1.77.Final.jar:oap-libs/netty-handler-proxy-4.1.77.Final.jar:oap-libs/netty-handler-4.1.77.Final.jar:oap-libs/netty-common-4.1.77.Final.jar:oap-libs/netty-codec-socks-4.1.77.Final.jar:oap-libs/netty-codec-http-4.1.77.Final.jar:oap-libs/netty-codec-http2-4.1.77.Final.jar:oap-libs/netty-codec-haproxy-4.1.77.Final.jar:oap-libs/netty-codec-dns-4.1.77.Final.jar:oap-libs/netty-codec-4.1.77.Final.jar:oap-libs/netty-buffer-4.1.77.Final.jar:oap-libs/nacos-common-1.4.2.jar:oap-libs/nacos-client-1.4.2.jar:oap-libs/nacos-api-1.4.2.jar:oap-libs/mvel2-2.4.8.Final.jar:oap-libs/micrometer-core-1.8.2.jar:oap-libs/meter-analyzer-9.1.0.jar:oap-libs/lz4-java-1.6.0.jar:oap-libs/logging-interceptor-4.9.1.jar:oap-libs/log-analyzer-9.1.0.jar:oap-libs/log4j-slf4j-impl-2.17.1.jar:oap-libs/log4j-over-slf4j-1.7.30.jar:oap-libs/log4j-core-2.17.1.jar:oap-libs/log4j-api-2.17.1.jar:oap-libs/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:oap-libs/library-util-9.1.0.jar:oap-libs/library-server-9.1.0.jar:oap-libs/library-module-9.1.0.jar:oap-libs/library-elasticsearch-client-9.1.0.jar:oap-libs/library-datacarrier-queue-9.1.0.jar:oap-libs/library-client-9.1.0.jar:oap-libs/LatencyUtils-2.0.3.jar:oap-libs/kotlinx-coroutines-reactive-1.5.0-native-mt.jar:oap-libs/kotlinx-coroutines-jdk8-1.5.0-native-mt.jar:oap-libs/kotlinx-coroutines-core-jvm-1.5.0-native-mt.jar:oap-libs/kotlinx-coroutines-core-1.5.0-native-mt.jar:oap-libs/kotlin-stdlib-jdk8-1.5.0.jar:oap-libs/kotlin-stdlib-jdk7-1.5.0.jar:oap-libs/kotlin-stdlib-common-1.5.0.jar:oap-libs/kotlin-stdlib-1.5.0.jar:oap-libs/kotlin-reflect-1.5.0.jar:oap-libs/kafka-fetcher-plugin-9.1.0.jar:oap-libs/kafka-clients-2.4.1.jar:oap-libs/jsr305-3.0.2.jar:oap-libs/jose4j-0.7.9.jar:oap-libs/joda-time-2.10.5.jar:oap-libs/jetcd-resolver-0.5.3.jar:oap-libs/jetcd-core-0.5.3.jar:oap-libs/jetcd-common-0.5.3.jar:oap-libs/jcl-over-slf4j-1.7.30.jar:oap-libs/javax.inject-1.jar:oap-libs/javassist-3.25.0-GA.jar:oap-libs/java-dataloader-3.1.0.jar:oap-libs/jackson-module-kotlin-2.13.1.jar:oap-libs/jackson-module-afterburner-2.12.2.jar:oap-libs/jackson-datatype-jdk8-2.9.10.jar:oap-libs/jackson-datatype-guava-2.9.10.jar:oap-libs/jackson-databind-2.13.2.2.jar:oap-libs/jackson-core-2.13.2.jar:oap-libs/jackson-annotations-2.13.2.jar:oap-libs/j2objc-annotations-1.3.jar:oap-libs/httpcore-nio-4.4.13.jar:oap-libs/httpcore-4.4.13.jar:oap-libs/httpclient-4.5.13.jar:oap-libs/httpasyncclient-4.1.3.jar:oap-libs/HikariCP-3.1.0.jar:oap-libs/HdrHistogram-2.1.12.jar:oap-libs/h2-2.1.212.jar:oap-libs/guice-4.1.0.jar:oap-libs/guava-31.1-jre.jar:oap-libs/gson-fire-1.8.5.jar:oap-libs/gson-2.9.0.jar:oap-libs/grpc-stub-1.46.0.jar:oap-libs/grpc-protobuf-lite-1.46.0.jar:oap-libs/grpc-protobuf-1.46.0.jar:oap-libs/grpc-netty-1.46.0.jar:oap-libs/grpc-grpclb-1.46.0.jar:oap-libs/grpc-core-1.46.0.jar:oap-libs/grpc-context-1.46.0.jar:oap-libs/grpc-configuration-sync-9.1.0.jar:oap-libs/grpc-api-1.46.0.jar:oap-libs/groovy-3.0.8.jar:oap-libs/graphql-java-tools-12.0.2.jar:oap-libs/graphql-java-extended-scalars-17.0.jar:oap-libs/graphql-java-17.3.jar:oap-libs/freemarker-2.3.31.jar:oap-libs/flatbuffers-java-1.12.0.jar:oap-libs/failureaccess-1.0.1.jar:oap-libs/failsafe-2.3.4.jar:oap-libs/exporter-9.1.0.jar:oap-libs/event-analyzer-9.1.0.jar:oap-libs/error_prone_annotations-2.11.0.jar:oap-libs/envoy-metrics-receiver-plugin-9.1.0.jar:oap-libs/data-generator-9.1.0.jar:oap-libs/curator-x-discovery-4.3.0.jar:oap-libs/curator-recipes-4.3.0.jar:oap-libs/curator-framework-4.3.0.jar:oap-libs/curator-client-4.3.0.jar:oap-libs/converter-jackson-2.5.0.jar:oap-libs/consul-client-1.4.2.jar:oap-libs/configuration-zookeeper-9.1.0.jar:oap-libs/configuration-nacos-9.1.0.jar:oap-libs/configuration-k8s-configmap-9.1.0.jar:oap-libs/configuration-etcd-9.1.0.jar:oap-libs/configuration-discovery-receiver-plugin-9.1.0.jar:oap-libs/configuration-consul-9.1.0.jar:oap-libs/configuration-apollo-9.1.0.jar:oap-libs/configuration-api-9.1.0.jar:oap-libs/commons-text-1.4.jar:oap-libs/commons-logging-1.2.jar:oap-libs/commons-lang3-3.12.0.jar:oap-libs/commons-io-2.7.jar:oap-libs/commons-compress-1.21.jar:oap-libs/commons-collections4-4.4.jar:oap-libs/commons-codec-1.11.jar:oap-libs/commons-beanutils-1.9.4.jar:oap-libs/cluster-zookeeper-plugin-9.1.0.jar:oap-libs/cluster-standalone-plugin-9.1.0.jar:oap-libs/cluster-nacos-plugin-9.1.0.jar:oap-libs/cluster-kubernetes-plugin-9.1.0.jar:oap-libs/cluster-etcd-plugin-9.1.0.jar:oap-libs/cluster-consul-plugin-9.1.0.jar:oap-libs/client-java-proto-14.0.0.jar:oap-libs/client-java-api-14.0.0.jar:oap-libs/client-java-14.0.0.jar:oap-libs/classmate-1.5.1.jar:oap-libs/checker-qual-3.12.0.jar:oap-libs/brotli4j-1.6.0.jar:oap-libs/bcutil-jdk15on-1.69.jar:oap-libs/bcprov-jdk15on-1.69.jar:oap-libs/bcprov-ext-jdk15on-1.69.jar:oap-libs/bcpkix-jdk15on-1.69.jar:oap-libs/banyandb-java-client-0.1.0.jar:oap-libs/audience-annotations-0.5.0.jar:oap-libs/armeria-protobuf-1.14.1.jar:oap-libs/armeria-graphql-protocol-1.14.1.jar:oap-libs/armeria-graphql-1.14.1.jar:oap-libs/armeria-1.14.1.jar:oap-libs/apollo-core-1.8.0.jar:oap-libs/apollo-client-1.8.0.jar:oap-libs/apm-network-9.1.0.jar:oap-libs/aopalliance-1.0.jar:oap-libs/antlr4-runtime-4.9.2.jar:oap-libs/annotations-13.0.jar:oap-libs/animal-sniffer-annotations-1.19.jar:oap-libs/agent-analyzer-9.1.0.jar:config:' org.apache.skywalking.oap.server.starter.OAPServerStartUp\n","stream":"stderr","time":"2022-07-02T04:16:58.855343376Z"}
{"log":"[0.005s][warning][os,thread] Failed to start thread - pthread_create failed (EPERM) for attributes: stacksize: 1024k, guardsize: 4k, detached.\n","stream":"stdout","time":"2022-07-02T04:16:58.86878564Z"}
{"log":"#\n","stream":"stdout","time":"2022-07-02T04:16:58.868984268Z"}
{"log":"# There is insufficient memory for the Java Runtime Environment to continue.\n","stream":"stdout","time":"2022-07-02T04:16:58.86900135Z"}
{"log":"# Cannot create worker GC thread. Out of system resources.\n","stream":"stdout","time":"2022-07-02T04:16:58.86900765Z"}
{"log":"# An error report file with more information is saved as:\n","stream":"stdout","time":"2022-07-02T04:16:58.869468693Z"}
{"log":"# /skywalking/hs_err_pid1.log\n","stream":"stdout","time":"2022-07-02T04:16:58.869486777Z"}

jvm error log


#
# There is insufficient memory for the Java Runtime Environment to continue.
# Cannot create worker GC thread. Out of system resources.
# Possible reasons:
#   The system is out of physical RAM or swap space
#   The process is running with CompressedOops enabled, and the Java Heap may be blocking the growth of the native heap
# Possible solutions:
#   Reduce memory load on the system
#   Increase physical memory or swap space
#   Check if swap backing store is full
#   Decrease Java heap size (-Xmx/-Xms)
#   Decrease number of Java threads
#   Decrease Java thread stack sizes (-Xss)
#   Set larger code cache with -XX:ReservedCodeCacheSize=
#   JVM is running with Unscaled Compressed Oops mode in which the Java heap is
#     placed in the first 4GB address space. The Java Heap base address is the
#     maximum limit for the native heap growth. Please use -XX:HeapBaseMinAddress
#     to set the Java Heap base and to place the Java Heap above 4GB virtual address.
# This output file may be truncated or incomplete.
#
#  Out of Memory Error (workerManager.hpp:70), pid=1, tid=1
#
# JRE version:  (11.0.15+10) (build )
# Java VM: OpenJDK 64-Bit Server VM (11.0.15+10, mixed mode, sharing, tiered, compressed oops, g1 gc, linux-amd64)
# Core dump will be written. Default location: /skywalking/core.1
#

---------------  S U M M A R Y ------------

Command Line: -Xms2048m -Xmx2048m org.apache.skywalking.oap.server.starter.OAPServerStartUp

Host: Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz, 16 cores, 98G, Ubuntu 22.04 LTS
Time: Fri Jul  1 05:51:19 2022 UTC elapsed time: 0.005089 seconds (0d 0h 0m 0s)

---------------  T H R E A D  ---------------

Current thread (0x0000563405091000):  JavaThread "Unknown thread" [_thread_in_vm, id=1, stack(0x00007ffc8d296000,0x00007ffc8da94000)]

Stack: [0x00007ffc8d296000,0x00007ffc8da94000],  sp=0x00007ffc8da8e3c0,  free space=8160k
Native frames: (J=compiled Java code, A=aot compiled Java code, j=interpreted, Vv=VM code, C=native code)
V  [libjvm.so+0xf424ca]  VMError::report_and_die(int, char const*, char const*, __va_list_tag*, Thread*, unsigned char*, void*, void*, char const*, int, unsigned long)+0x1ca
V  [libjvm.so+0xf4338b]  VMError::report_and_die(Thread*, char const*, int, unsigned long, VMErrorType, char const*, __va_list_tag*)+0x2b
V  [libjvm.so+0x6cf005]  report_vm_out_of_memory(char const*, int, unsigned long, VMErrorType, char const*, ...)+0xd5
V  [libjvm.so+0xf6f58b]  AbstractWorkGang::add_workers(unsigned int, bool)+0x14b
V  [libjvm.so+0x7aa8c5]  G1CollectedHeap::G1CollectedHeap(G1CollectorPolicy*)+0x5b5
V  [libjvm.so+0x79c3b9]  G1Arguments::create_heap()+0x69
V  [libjvm.so+0xefdc11]  Universe::initialize_heap()+0x31
V  [libjvm.so+0xefdec6]  universe_init()+0x46
V  [libjvm.so+0x8a4207]  init_globals()+0x57
V  [libjvm.so+0xed5537]  Threads::create_vm(JavaVMInitArgs*, bool*)+0x357
V  [libjvm.so+0x960262]  JNI_CreateJavaVM+0x52
C  [libjli.so+0x49af]  JavaMain+0x8f
C  [libjli.so+0x9d94]  CallJavaMainInNewThread+0x74
C  [libjli.so+0x637d]  ContinueInNewThread+0x4d
C  [libjli.so+0x7a00]  JLI_Launch+0x1470
C  [java+0xa73]

---------------  P R O C E S S  ---------------

Threads class SMR info:
_java_thread_list=0x0000563405054790, length=0, elements={
}

Java Threads: ( => current thread )

Other Threads:

=>0x0000563405091000 (exited) JavaThread "Unknown thread" [_thread_in_vm, id=1, stack(0x00007ffc8d296000,0x00007ffc8da94000)]

Threads with active compile tasks:

VM state:not at safepoint (not fully initialized)

VM Mutex/Monitor currently owned by a thread: None

[error occurred during error reporting (printing compressed oops mode), id 0xb, SIGSEGV (0xb) at pc=0x00007f98acefc543]

GC Heap History (0 events):
No events

Deoptimization events (0 events):
No events

Classes redefined (0 events):
No events

Internal exceptions (0 events):
No events

Events (2 events):
Event: 0.002 Protecting memory [0x00007ffc8d296000,0x00007ffc8d29a000] with protection modes 0
Event: 0.003 Loaded shared library /opt/java/openjdk/lib/libzip.so

Dynamic libraries:
56340325f000-563403260000 r-xp 00000000 08:03 36117702                   /opt/java/openjdk/bin/java
563403460000-563403461000 r--p 00001000 08:03 36117702                   /opt/java/openjdk/bin/java
563403461000-563403462000 rw-p 00002000 08:03 36117702                   /opt/java/openjdk/bin/java
56340504e000-5634050d1000 rw-p 00000000 00:00 0                          [heap]
7f989626e000-7f98964dd000 rw-p 00000000 00:00 0 
7f98964dd000-7f989674d000 rwxp 00000000 00:00 0 
7f989674d000-7f9896c1c000 ---p 00000000 00:00 0 
7f9896c1c000-7f9896e8c000 rwxp 00000000 00:00 0 
7f9896e8c000-7f989e07c000 ---p 00000000 00:00 0 
7f989e07c000-7f989e2ec000 rwxp 00000000 00:00 0 
7f989e2ec000-7f98a54dd000 ---p 00000000 00:00 0 
7f98a54dd000-7f98ab000000 r--s 00000000 08:03 585853954                  /opt/java/openjdk/lib/modules
7f98ab000000-7f98ab007000 r-xp 00000000 08:03 585853952                  /opt/java/openjdk/lib/libzip.so
7f98ab007000-7f98ab207000 ---p 00007000 08:03 585853952                  /opt/java/openjdk/lib/libzip.so
7f98ab207000-7f98ab208000 r--p 00007000 08:03 585853952                  /opt/java/openjdk/lib/libzip.so
7f98ab208000-7f98ab209000 rw-p 00008000 08:03 585853952                  /opt/java/openjdk/lib/libzip.so
7f98ab2b2000-7f98ab400000 rw-p 00000000 00:00 0 
7f98ab400000-7f98ab41b000 r-xp 00000000 08:03 585853929                  /opt/java/openjdk/lib/libjimage.so
7f98ab41b000-7f98ab61a000 ---p 0001b000 08:03 585853929                  /opt/java/openjdk/lib/libjimage.so
7f98ab61a000-7f98ab61c000 r--p 0001a000 08:03 585853929                  /opt/java/openjdk/lib/libjimage.so
7f98ab61c000-7f98ab61d000 rw-p 0001c000 08:03 585853929                  /opt/java/openjdk/lib/libjimage.so
7f98ab6fe000-7f98ab6ff000 ---p 00000000 00:00 0 
7f98ab6ff000-7f98ab800000 rw-p 00000000 00:00 0 
7f98ab800000-7f98ab828000 r-xp 00000000 08:03 585853923                  /opt/java/openjdk/lib/libjava.so
7f98ab828000-7f98aba28000 ---p 00028000 08:03 585853923                  /opt/java/openjdk/lib/libjava.so
7f98aba28000-7f98aba29000 r--p 00028000 08:03 585853923                  /opt/java/openjdk/lib/libjava.so
7f98aba29000-7f98aba2a000 rw-p 00029000 08:03 585853923                  /opt/java/openjdk/lib/libjava.so
7f98aba2a000-7f98aba2b000 rw-p 00000000 00:00 0 
7f98aba2e000-7f98aba33000 rw-p 00000000 00:00 0 
7f98aba33000-7f98abb17000 ---p 00000000 00:00 0 
7f98abb17000-7f98abb1c000 rw-p 00000000 00:00 0 
7f98abb1c000-7f98abc00000 ---p 00000000 00:00 0 
7f98abc00000-7f98abc0d000 r-xp 00000000 08:03 585853951                  /opt/java/openjdk/lib/libverify.so
7f98abc0d000-7f98abe0c000 ---p 0000d000 08:03 585853951                  /opt/java/openjdk/lib/libverify.so
7f98abe0c000-7f98abe0e000 r--p 0000c000 08:03 585853951                  /opt/java/openjdk/lib/libverify.so
7f98abe0e000-7f98abe0f000 rw-p 0000e000 08:03 585853951                  /opt/java/openjdk/lib/libverify.so
7f98abe19000-7f98abf19000 rw-p 00000000 00:00 0 
7f98abf19000-7f98abf27000 r--p 00000000 08:03 580295711                  /usr/lib/x86_64-linux-gnu/libm.so.6
7f98abf27000-7f98abfa3000 r-xp 0000e000 08:03 580295711                  /usr/lib/x86_64-linux-gnu/libm.so.6
7f98abfa3000-7f98abffe000 r--p 0008a000 08:03 580295711                  /usr/lib/x86_64-linux-gnu/libm.so.6
7f98abffe000-7f98abfff000 r--p 000e4000 08:03 580295711                  /usr/lib/x86_64-linux-gnu/libm.so.6
7f98abfff000-7f98ac000000 rw-p 000e5000 08:03 580295711                  /usr/lib/x86_64-linux-gnu/libm.so.6
7f98ac000000-7f98ad270000 r-xp 00000000 08:03 585853960                  /opt/java/openjdk/lib/server/libjvm.so
7f98ad270000-7f98ad470000 ---p 01270000 08:03 585853960                  /opt/java/openjdk/lib/server/libjvm.so
7f98ad470000-7f98ad535000 r--p 01270000 08:03 585853960                  /opt/java/openjdk/lib/server/libjvm.so
7f98ad535000-7f98ad570000 rw-p 01335000 08:03 585853960                  /opt/java/openjdk/lib/server/libjvm.so
7f98ad570000-7f98ad5c7000 rw-p 00000000 00:00 0 
7f98ad5d8000-7f98ad600000 r--p 00000000 08:03 580151838                  /usr/lib/x86_64-linux-gnu/libc.so.6
7f98ad600000-7f98ad795000 r-xp 00028000 08:03 580151838                  /usr/lib/x86_64-linux-gnu/libc.so.6
7f98ad795000-7f98ad7ed000 r--p 001bd000 08:03 580151838                  /usr/lib/x86_64-linux-gnu/libc.so.6
7f98ad7ed000-7f98ad7f1000 r--p 00214000 08:03 580151838                  /usr/lib/x86_64-linux-gnu/libc.so.6
7f98ad7f1000-7f98ad7f3000 rw-p 00218000 08:03 580151838                  /usr/lib/x86_64-linux-gnu/libc.so.6
7f98ad7f3000-7f98ad800000 rw-p 00000000 00:00 0 
7f98ad800000-7f98ad810000 r-xp 00000000 08:03 36123812                   /opt/java/openjdk/lib/jli/libjli.so
7f98ad810000-7f98ada0f000 ---p 00010000 08:03 36123812                   /opt/java/openjdk/lib/jli/libjli.so
7f98ada0f000-7f98ada10000 r--p 0000f000 08:03 36123812                   /opt/java/openjdk/lib/jli/libjli.so
7f98ada10000-7f98ada11000 rw-p 00010000 08:03 36123812                   /opt/java/openjdk/lib/jli/libjli.so
7f98ada78000-7f98ada7d000 rw-p 00000000 00:00 0 
7f98ada7d000-7f98ada87000 ---p 00000000 00:00 0 
7f98ada87000-7f98ada8f000 rw-s 00000000 08:03 313133055                  /tmp/hsperfdata_root/1
7f98ada8f000-7f98ada90000 r--p 00000000 08:03 580347790                  /usr/lib/x86_64-linux-gnu/librt.so.1
7f98ada90000-7f98ada91000 r-xp 00001000 08:03 580347790                  /usr/lib/x86_64-linux-gnu/librt.so.1
7f98ada91000-7f98ada92000 r--p 00002000 08:03 580347790                  /usr/lib/x86_64-linux-gnu/librt.so.1
7f98ada92000-7f98ada93000 r--p 00002000 08:03 580347790                  /usr/lib/x86_64-linux-gnu/librt.so.1
7f98ada93000-7f98ada94000 rw-p 00003000 08:03 580347790                  /usr/lib/x86_64-linux-gnu/librt.so.1
7f98ada94000-7f98ada99000 rw-p 00000000 00:00 0 
7f98ada99000-7f98ada9a000 r--p 00000000 08:03 580151854                  /usr/lib/x86_64-linux-gnu/libdl.so.2
7f98ada9a000-7f98ada9b000 r-xp 00001000 08:03 580151854                  /usr/lib/x86_64-linux-gnu/libdl.so.2
7f98ada9b000-7f98ada9c000 r--p 00002000 08:03 580151854                  /usr/lib/x86_64-linux-gnu/libdl.so.2
7f98ada9c000-7f98ada9d000 r--p 00002000 08:03 580151854                  /usr/lib/x86_64-linux-gnu/libdl.so.2
7f98ada9d000-7f98ada9e000 rw-p 00003000 08:03 580151854                  /usr/lib/x86_64-linux-gnu/libdl.so.2
7f98ada9e000-7f98ada9f000 r--p 00000000 08:03 580345503                  /usr/lib/x86_64-linux-gnu/libpthread.so.0
7f98ada9f000-7f98adaa0000 r-xp 00001000 08:03 580345503                  /usr/lib/x86_64-linux-gnu/libpthread.so.0
7f98adaa0000-7f98adaa1000 r--p 00002000 08:03 580345503                  /usr/lib/x86_64-linux-gnu/libpthread.so.0
7f98adaa1000-7f98adaa2000 r--p 00002000 08:03 580345503                  /usr/lib/x86_64-linux-gnu/libpthread.so.0
7f98adaa2000-7f98adaa3000 rw-p 00003000 08:03 580345503                  /usr/lib/x86_64-linux-gnu/libpthread.so.0
7f98adaa3000-7f98adaa5000 r--p 00000000 08:03 580349309                  /usr/lib/x86_64-linux-gnu/libz.so.1.2.11
7f98adaa5000-7f98adab6000 r-xp 00002000 08:03 580349309                  /usr/lib/x86_64-linux-gnu/libz.so.1.2.11
7f98adab6000-7f98adabc000 r--p 00013000 08:03 580349309                  /usr/lib/x86_64-linux-gnu/libz.so.1.2.11
7f98adabc000-7f98adabd000 ---p 00019000 08:03 580349309                  /usr/lib/x86_64-linux-gnu/libz.so.1.2.11
7f98adabd000-7f98adabe000 r--p 00019000 08:03 580349309                  /usr/lib/x86_64-linux-gnu/libz.so.1.2.11
7f98adabe000-7f98adabf000 rw-p 0001a000 08:03 580349309                  /usr/lib/x86_64-linux-gnu/libz.so.1.2.11
7f98adabf000-7f98adac0000 ---p 00000000 00:00 0 
7f98adac0000-7f98adac1000 r--p 00000000 00:00 0 
7f98adac1000-7f98adac3000 rw-p 00000000 00:00 0 
7f98adac3000-7f98adac5000 r--p 00000000 08:03 580151816                  /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
7f98adac5000-7f98adaef000 r-xp 00002000 08:03 580151816                  /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
7f98adaef000-7f98adafa000 r--p 0002c000 08:03 580151816                  /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
7f98adafb000-7f98adafd000 r--p 00037000 08:03 580151816                  /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
7f98adafd000-7f98adaff000 rw-p 00039000 08:03 580151816                  /usr/lib/x86_64-linux-gnu/ld-linux-x86-64.so.2
7ffc8d296000-7ffc8d29a000 ---p 00000000 00:00 0 
7ffc8da73000-7ffc8da96000 rw-p 00000000 00:00 0                          [stack]
7ffc8dbfc000-7ffc8dbfe000 r-xp 00000000 00:00 0                          [vdso]
ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0                  [vsyscall]

VM Arguments:
jvm_args: -Xms2048m -Xmx2048m 
java_command: org.apache.skywalking.oap.server.starter.OAPServerStartUp
java_class_path (initial): /skywalking/ext-libs/*.jar:oap-libs/zstd-jni-1.4.3-1.jar:oap-libs/zookeeper-jute-3.5.7.jar:oap-libs/zookeeper-3.5.7.jar:oap-libs/zipkin-receiver-plugin-9.1.0.jar:oap-libs/zipkin-query-plugin-9.1.0.jar:oap-libs/zipkin-2.23.16.jar:oap-libs/vavr-match-0.10.3.jar:oap-libs/vavr-0.10.3.jar:oap-libs/tool-profile-snapshot-server-mock-9.1.0.jar:oap-libs/tool-profile-snapshot-bootstrap-9.1.0.jar:oap-libs/telemetry-prometheus-9.1.0.jar:oap-libs/telemetry-api-9.1.0.jar:oap-libs/swagger-annotations-1.6.3.jar:oap-libs/storage-tidb-plugin-9.1.0.jar:oap-libs/storage-jdbc-hikaricp-plugin-9.1.0.jar:oap-libs/storage-elasticsearch-plugin-9.1.0.jar:oap-libs/storage-banyandb-plugin-9.1.0.jar:oap-libs/snappy-java-1.1.7.3.jar:oap-libs/snakeyaml-1.28.jar:oap-libs/slf4j-api-1.7.30.jar:oap-libs/skywalking-zabbix-receiver-plugin-9.1.0.jar:oap-libs/skywalking-trace-receiver-plugin-9.1.0.jar:oap-libs/skywalking-sharing-server-plugin-9.1.0.jar:oap-libs/skywalking-profile-receiver-plugin-9.1.0.jar:oap-libs/skywalking-meter-receiver-plugin-9.1.0.jar:oap-libs/skywalking-mesh-receiver-plugin-9.1.0.jar:oap-libs/skywalking-management-receiver-plugin-9.1.0.jar:oap-libs/skywalking-log-recevier-plugin-9.1.0.jar:oap-libs/skywalking-jvm-receiver-plugin-9.1.0.jar:oap-libs/skywalking-event-receiver-plugin-9.1.0.jar:oap-libs/skywalking-ebpf-receiver-plugin-9.1.0.jar:oap-libs/skywalking-clr-receiver-plugin-9.1.0.jar:oap-libs/skywalking-browser-receiver-plugin-9.1.0.jar:oap-libs/simpleclient_httpserver-0.12.0.jar:oap-libs/simpleclient_hotspot-0.6.0.jar:oap-libs/simpleclient_common-0.6.0.jar:oap-libs/simpleclient-0.6.0.jar:oap-libs/server-starter-9.1.0.jar:oap-libs/server-health-checker-9.1.0.jar:oap-libs/server-core-9.1.0.jar:oap-libs/server-alarm-plugin-9.1.0.jar:oap-libs/retrofit-2.5.0.jar:oap-libs/receiver-proto-9.1.0.jar:oap-libs/reactive-streams-1.0.2.jar:oap-libs/query-graphql-plugin-9.1.0.jar:oap-libs/proto-google-common-protos-2.0.1.jar:oap-libs/protobuf-java-util-3.19.4.jar:
Launcher Type: SUN_STANDARD

[Global flags]
     intx CICompilerCount                          = 12                                        {product} {ergonomic}
     uint G1ConcRefinementThreads                  = 13                                        {product} {ergonomic}
   size_t G1HeapRegionSize                         = 1048576                                   {product} {ergonomic}
    uintx GCDrainStackTargetSize                   = 64                                        {product} {ergonomic}
   size_t InitialHeapSize                          = 2147483648                                {product} {command line}
   size_t MaxHeapSize                              = 2147483648                                {product} {command line}
   size_t MinHeapDeltaBytes                        = 1048576                                   {product} {ergonomic}
    uintx NonNMethodCodeHeapSize                   = 7594288                                {pd product} {ergonomic}
    uintx NonProfiledCodeHeapSize                  = 122031976                              {pd product} {ergonomic}
    uintx ProfiledCodeHeapSize                     = 122031976                              {pd product} {ergonomic}
    uintx ReservedCodeCacheSize                    = 251658240                              {pd product} {ergonomic}
     bool SegmentedCodeCache                       = true                                      {product} {ergonomic}
     bool UseCompressedClassPointers               = true                                 {lp64_product} {ergonomic}
     bool UseCompressedOops                        = true                                 {lp64_product} {ergonomic}
     bool UseG1GC                                  = true                                      {product} {ergonomic}

Logging:
Log output configuration:
 #0: stdout all=warning uptime,level,tags
 #1: stderr all=off uptime,level,tags

Environment Variables:
JAVA_HOME=/opt/java/openjdk
PATH=/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
LANG=en_US.UTF-8
LC_ALL=en_US.UTF-8

Signal Handlers:
SIGSEGV: [libjvm.so+0xf44160], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGBUS: [libjvm.so+0xf44160], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGFPE: [libjvm.so+0xf44160], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGPIPE: [libjvm.so+0xc669b0], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGXFSZ: [libjvm.so+0xc669b0], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGILL: [libjvm.so+0xf44160], sa_mask[0]=11111111011111111101111111111110, sa_flags=SA_RESTART|SA_SIGINFO
SIGUSR2: [libjvm.so+0xc66850], sa_mask[0]=00000000000000000000000000000000, sa_flags=SA_RESTART|SA_SIGINFO
SIGHUP: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
SIGINT: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
SIGTERM: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none
SIGQUIT: SIG_DFL, sa_mask[0]=00000000000000000000000000000000, sa_flags=none

---------------  S Y S T E M  ---------------

OS:DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04 LTS"
uname:Linux 3.10.0-1127.el7.x86_64 #1 SMP Tue Mar 31 23:36:51 UTC 2020 x86_64
OS uptime: 116 days 0:44 hours
libc:glibc 2.35 NPTL 2.35 
rlimit (soft/hard): STACK 8192k/infinity , CORE infinity/infinity , NPROC infinity/infinity , NOFILE 1048576/1048576 , AS infinity/infinity , CPU infinity/infinity , DATA infinity/infinity , FSIZE infinity/infinity , MEMLOCK 64k/64k
load average:0.12 0.09 0.06

/proc/meminfo:
MemTotal:       102945540 kB
MemFree:        76121656 kB
MemAvailable:   97823172 kB
Buffers:            2076 kB
Cached:         16185204 kB
SwapCached:            0 kB
Active:         11590136 kB
Inactive:        4824488 kB
Active(anon):     901108 kB
Inactive(anon):   377560 kB
Active(file):   10689028 kB
Inactive(file):  4446928 kB
Unevictable:      940132 kB
Mlocked:          940160 kB
SwapTotal:      33554428 kB
SwapFree:       33554428 kB
Dirty:                36 kB
Writeback:             0 kB
AnonPages:       1176724 kB
Mapped:           309772 kB
Shmem:            880484 kB
Slab:            7748008 kB
SReclaimable:    7242500 kB
SUnreclaim:       505508 kB
KernelStack:       10656 kB
PageTables:        10048 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:    85027196 kB
Committed_AS:    3584344 kB
VmallocTotal:   34359738367 kB
VmallocUsed:      394548 kB
VmallocChunk:   34359310332 kB
Percpu:            36352 kB
HardwareCorrupted:     0 kB
AnonHugePages:    675840 kB
CmaTotal:              0 kB
CmaFree:               0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      151360 kB
DirectMap2M:     5091328 kB
DirectMap1G:    101711872 kB

/sys/kernel/mm/transparent_hugepage/enabled:
[always] madvise never
/sys/kernel/mm/transparent_hugepage/defrag (defrag/compaction efforts parameter):
[always] madvise never

Process Memory:
Virtual Size: 384192K (peak: 384192K)
Resident Set Size: 11884K (peak: 11884K) (anon: 5596K, file: 6288K, shmem: 0K)
Swapped out: 0K
C-Heap outstanding allocations: 419K

/proc/sys/kernel/threads-max (system-wide limit on the number of threads):
804094
/proc/sys/vm/max_map_count (maximum number of memory map areas a process may have):
262144
/proc/sys/kernel/pid_max (system-wide limit on number of process identifiers):
4194303

container (cgroup) information:
container_type: cgroupv1
cpu_cpuset_cpus: 0-15
cpu_memory_nodes: 0
active_processor_count: 16
cpu_quota: -1
cpu_period: 100000
cpu_shares: -1
memory_limit_in_bytes: -1
memory_and_swap_limit_in_bytes: -1
memory_soft_limit_in_bytes: -1
memory_usage_in_bytes: 6201344
memory_max_usage_in_bytes: 6201344

VMWare virtualization detected
Steal ticks since vm start: 0
Steal ticks percentage since vm start:  0.000

CPU:total 16 (initial active 16) (8 cores per cpu, 1 threads per core) family 6 model 85 stepping 4 microcode 0x2006a08, cmov, cx8, fxsr, mmx, sse, sse2, sse3, ssse3, sse4.1, sse4.2, popcnt, avx, avx2, aes, clmul, erms, 3dnowpref, lzcnt, tsc, tscinvbit, bmi1, bmi2, adx, fma
CPU Model and flags from /proc/cpuinfo:
model name  : Intel(R) Xeon(R) Gold 6130 CPU @ 2.10GHz
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon nopl xtopology tsc_reliable nonstop_tsc eagerfpu pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp tpr_shadow vnmi ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 invpcid avx512f avx512dq rdseed adx smap clflushopt clwb avx512cd avx512bw avx512vl xsaveopt xsavec arat pku ospke md_clear spec_ctrl intel_stibp flush_l1d arch_capabilities
Online cpus:
0-15
Offline cpus:
16-127
BIOS frequency limitation:
<Not Available>
Frequency switch latency (ns):
<Not Available>
Available cpu frequencies:
<Not Available>
Current governor:
<Not Available>
Core performance/turbo boost:
<Not Available>

Memory: 4k page, physical 102945540k(76121656k free), swap 33554428k(33554428k free)

vm_info: OpenJDK 64-Bit Server VM (11.0.15+10) for linux-amd64 JRE (11.0.15+10), built on Apr 19 2022 21:36:55 by "" with gcc 7.5.0

END.

docker compose file


# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

version: '3.8'
services:
  elasticsearch:
    image: docker.elastic.co/elasticsearch/elasticsearch-oss:${ES_VERSION}
    container_name: elasticsearch
    ports:
      - "9200:9200"
    healthcheck:
      test: [ "CMD-SHELL", "curl --silent --fail localhost:9200/_cluster/health || exit 1" ]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 10s
    environment:
      - discovery.type=single-node
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1

  oap:
    image: ${OAP_IMAGE}
    container_name: oap
    depends_on:
      elasticsearch:
        condition: service_healthy
    links:
      - elasticsearch
    ports:
      - "11800:11800"
      - "12800:12800"
    healthcheck:
      test: [ "CMD-SHELL", "/skywalking/bin/swctl ch" ]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 10s
    environment:
      SW_STORAGE: elasticsearch
      SW_STORAGE_ES_CLUSTER_NODES: elasticsearch:9200
      SW_HEALTH_CHECKER: default
      SW_TELEMETRY: prometheus
      JAVA_OPTS: "-Xms4096m -Xmx4096m"

  ui:
    image: ${UI_IMAGE}
    container_name: ui
    depends_on:
      oap:
        condition: service_healthy
    links:
      - oap
    ports:
      - "8080:8080"
    environment:
      SW_OAP_ADDRESS: http://oap:12800
roginluo commented 2 years ago

docker inpection

[deployer@Clone docker]$ docker inspect b051202f3cb6
[
    {
        "Id": "b051202f3cb6b94d9ff8e3be5f67e3ddc94ca229cf6527ab6856e9a6aa1e9a5b",
        "Created": "2022-07-02T04:43:45.885360938Z",
        "Path": "bash",
        "Args": [
            "docker-entrypoint.sh"
        ],
        "State": {
            "Status": "exited",
            "Running": false,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 0,
            "ExitCode": 1,
            "Error": "",
            "StartedAt": "2022-07-02T04:43:47.024587936Z",
            "FinishedAt": "2022-07-02T04:43:47.104826059Z",
            "Health": {
                "Status": "unhealthy",
                "FailingStreak": 0,
                "Log": []
            }
        },
        "Image": "sha256:b21800a57a2b56ee84031cbdea65834fd15eb8731f9c9bf2deaa0aeae12d0aa6",
        "ResolvConfPath": "/var/lib/docker/containers/b051202f3cb6b94d9ff8e3be5f67e3ddc94ca229cf6527ab6856e9a6aa1e9a5b/resolv.conf",
        "HostnamePath": "/var/lib/docker/containers/b051202f3cb6b94d9ff8e3be5f67e3ddc94ca229cf6527ab6856e9a6aa1e9a5b/hostname",
        "HostsPath": "/var/lib/docker/containers/b051202f3cb6b94d9ff8e3be5f67e3ddc94ca229cf6527ab6856e9a6aa1e9a5b/hosts",
        "LogPath": "/var/lib/docker/containers/b051202f3cb6b94d9ff8e3be5f67e3ddc94ca229cf6527ab6856e9a6aa1e9a5b/b051202f3cb6b94d9ff8e3be5f67e3ddc94ca229cf6527ab6856e9a6aa1e9a5b-json.log",
        "Name": "/oap",
        "RestartCount": 0,
        "Driver": "overlay2",
        "Platform": "linux",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "",
        "ExecIDs": null,
        "HostConfig": {
            "Binds": [],
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "json-file",
                "Config": {
                    "max-file": "3",
                    "max-size": "10m"
                }
            },
            "NetworkMode": "docker_default",
            "PortBindings": {
                "11800/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "11800"
                    }
                ],
                "12800/tcp": [
                    {
                        "HostIp": "",
                        "HostPort": "12800"
                    }
                ]
            },
            "RestartPolicy": {
                "Name": "",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": null,
            "CapDrop": null,
            "Capabilities": null,
            "Dns": null,
            "DnsOptions": null,
            "DnsSearch": null,
            "ExtraHosts": [],
            "GroupAdd": null,
            "IpcMode": "private",
            "Cgroup": "",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": null,
            "UTSMode": "",
            "UsernsMode": "",
            "ShmSize": 67108864,
            "Runtime": "runc",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": null,
            "DeviceCgroupRules": null,
            "DeviceRequests": null,
            "KernelMemory": 0,
            "KernelMemoryTCP": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": null,
            "OomKillDisable": false,
            "PidsLimit": null,
            "Ulimits": null,
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "MaskedPaths": [
                "/proc/asound",
                "/proc/acpi",
                "/proc/kcore",
                "/proc/keys",
                "/proc/latency_stats",
                "/proc/timer_list",
                "/proc/timer_stats",
                "/proc/sched_debug",
                "/proc/scsi",
                "/sys/firmware"
            ],
            "ReadonlyPaths": [
                "/proc/bus",
                "/proc/fs",
                "/proc/irq",
                "/proc/sys",
                "/proc/sysrq-trigger"
            ]
        },
        "GraphDriver": {
            "Data": {
                "LowerDir": "/var/lib/docker/overlay2/d27bbe912e7abe70db1d5a156796492ea9a7a8ac935364b4706254092fc119a3-init/diff:/var/lib/docker/overlay2/1528c65ec6c9b3d1b05ad876eee753074fd83e9a5475b9ad00d422aec2da355e/diff:/var/lib/docker/overlay2/dec8913b60805d40caa1e0be07c44d1e275a8de22b70bd304ced6a5714e96ecf/diff:/var/lib/docker/overlay2/767c6f09349401c1edc318ff51d6a80ddf6cf2722ae1fbf18ac3ca184e3db2ca/diff:/var/lib/docker/overlay2/2d5fc282d3b64c48aee6d98f778c30a4db7e4ba052e22df01d1ee0688af911f1/diff:/var/lib/docker/overlay2/ee19c2b5d69a0b5264c4191e528eefae9af57cd6f43078d7434afc89b1c41293/diff:/var/lib/docker/overlay2/c82afee51c47573d8e2b43c04ff4cb7295443668ccf8cc07466624c30ca9bb32/diff:/var/lib/docker/overlay2/ac3427b1a041fcab8f5bb78445c7ef6d732c9b10819d954125a53283fc52e84c/diff:/var/lib/docker/overlay2/f7b74f90c27bf12253c850c43be7da91ce18faa56f799937c44160eada566e23/diff:/var/lib/docker/overlay2/1832b872b543d20531771c9aa95705083ee96609cd0227c3a3ce03d040854638/diff:/var/lib/docker/overlay2/633e3ae6cceb175bdb8fbaac40c8c61982cb9157ea08060bf17c97b32b5ec36e/diff:/var/lib/docker/overlay2/b2c7e990412ba8c7681d178bb349b0811f42186dba7f2ff9b85870fcc9e21000/diff",
                "MergedDir": "/var/lib/docker/overlay2/d27bbe912e7abe70db1d5a156796492ea9a7a8ac935364b4706254092fc119a3/merged",
                "UpperDir": "/var/lib/docker/overlay2/d27bbe912e7abe70db1d5a156796492ea9a7a8ac935364b4706254092fc119a3/diff",
                "WorkDir": "/var/lib/docker/overlay2/d27bbe912e7abe70db1d5a156796492ea9a7a8ac935364b4706254092fc119a3/work"
            },
            "Name": "overlay2"
        },
        "Mounts": [],
        "Config": {
            "Hostname": "b051202f3cb6",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": true,
            "AttachStderr": true,
            "ExposedPorts": {
                "11800/tcp": {},
                "1234/tcp": {},
                "12800/tcp": {}
            },
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "SW_STORAGE=elasticsearch",
                "SW_STORAGE_ES_CLUSTER_NODES=elasticsearch:9200",
                "SW_HEALTH_CHECKER=default",
                "SW_TELEMETRY=prometheus",
                "JAVA_OPTS=-Xms4096m -Xmx4096m",
                "PATH=/opt/java/openjdk/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "LANG=en_US.UTF-8",
                "LANGUAGE=en_US:en",
                "LC_ALL=en_US.UTF-8",
                "JAVA_VERSION=jdk-11.0.15+10",
                "JAVA_HOME=/opt/java/openjdk"
            ],
            "Cmd": null,
            "Healthcheck": {
                "Test": [
                    "CMD-SHELL",
                    "/skywalking/bin/swctl ch"
                ],
                "Interval": 30000000000,
                "Timeout": 10000000000,
                "StartPeriod": 10000000000,
                "Retries": 3
            },
            "Image": "apache/skywalking-oap-server:latest",
            "Volumes": null,
            "WorkingDir": "/skywalking",
            "Entrypoint": [
                "bash",
                "docker-entrypoint.sh"
            ],
            "OnBuild": null,
            "Labels": {
                "com.docker.compose.config-hash": "a28d27032a8f8a9829f2cebb783f446f805da83eb87f088fb5c7614328bb70aa",
                "com.docker.compose.container-number": "1",
                "com.docker.compose.depends_on": "elasticsearch:service_healthy",
                "com.docker.compose.oneoff": "False",
                "com.docker.compose.project": "docker",
                "com.docker.compose.project.config_files": "/home/deployer/lj/skywalking/skywalking/docker/docker-compose.yml",
                "com.docker.compose.project.working_dir": "/home/deployer/lj/skywalking/skywalking/docker",
                "com.docker.compose.service": "oap",
                "com.docker.compose.version": "2.6.0"
            }
        },
        "NetworkSettings": {
            "Bridge": "",
            "SandboxID": "d3798f401296166a6ca5d9bcf12c138104a76fc0b84282cb35e9fafdbb6e50df",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {},
            "SandboxKey": "/var/run/docker/netns/d3798f401296",
            "SecondaryIPAddresses": null,
            "SecondaryIPv6Addresses": null,
            "EndpointID": "",
            "Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "IPAddress": "",
            "IPPrefixLen": 0,
            "IPv6Gateway": "",
            "MacAddress": "",
            "Networks": {
                "docker_default": {
                    "IPAMConfig": null,
                    "Links": [
                        "elasticsearch:elasticsearch",
                        "elasticsearch:elasticsearch-1",
                        "elasticsearch:docker-elasticsearch-1"
                    ],
                    "Aliases": [
                        "oap",
                        "oap",
                        "b051202f3cb6"
                    ],
                    "NetworkID": "f050e0bbaadbf3b1c7ba13d6e9088ac6a687d6cae16782f1483982045ba02da6",
                    "EndpointID": "",
                    "Gateway": "",
                    "IPAddress": "",
                    "IPPrefixLen": 0,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "",
                    "DriverOpts": null
                }
            }
        }
    }
]
roginluo commented 2 years ago

docker-entrypoint.sh

#!/bin/sh
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements.  See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership.  The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

set -e

echo "[Entrypoint] Apache SkyWalking Docker Image"

EXT_LIB_DIR=/skywalking/ext-libs
EXT_CONFIG_DIR=/skywalking/ext-config

# Override configuration files
if [ "$(ls -A $EXT_CONFIG_DIR)" ]; then
  cp -vfRL ${EXT_CONFIG_DIR}/* config/
fi

CLASSPATH="config:$CLASSPATH"
for i in oap-libs/*.jar
do
    CLASSPATH="$i:$CLASSPATH"
done
for i in "${EXT_LIB_DIR}"/*.jar
do
    CLASSPATH="$i:$CLASSPATH"
done

set -ex

exec java ${JAVA_OPTS} -classpath ${CLASSPATH} org.apache.skywalking.oap.server.starter.OAPServerStartUp "$@"
roginluo commented 2 years ago

this is my hosted os info

[deployer@Clone ~]$ cat /etc/os-release 
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"

[deployer@Clone ~]$ free -h
              total        used        free      shared  buff/cache   available
Mem:            98G        3.2G         72G        859M         22G         93G
Swap:           31G          0B         31G
[deployer@Clone ~]$ uname -a
Linux Clone 3.10.0-1127.el7.x86_64 #1 SMP Tue Mar 31 23:36:51 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
karianna commented 2 years ago

are you running in a docker container? if so are you on the latest engine

roginluo commented 2 years ago

here is my docker version

[deployer@Clone ~]$ docker version
Client: Docker Engine - Community
 Version:           19.03.8
 API version:       1.40
 Go version:        go1.12.17
 Git commit:        afacb8b7f0
 Built:             Wed Mar 11 01:22:56 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.8
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.17
  Git commit:       afacb8b7f0
  Built:            Wed Mar 11 01:30:32 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.2.13
  GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683
karianna commented 2 years ago

Ah OK, yes please upgrade to the 20.x engine

roginluo commented 2 years ago

after docker upgraded,the problem has been resolved,but,Is the older version of docker no longer supported?

[deployer@Clone docker]$ docker ps -a
CONTAINER ID   IMAGE                                                     COMMAND                  CREATED              STATUS                        PORTS                                                                                                    NAMES
b8afa9c5e046   apache/skywalking-ui:latest                               "bash docker-entrypo…"   About a minute ago   Up 7 seconds                  0.0.0.0:8080->8080/tcp, :::8080->8080/tcp                                                                ui
3a2cf8e05d68   apache/skywalking-oap-server:latest                       "bash docker-entrypo…"   About a minute ago   Up 38 seconds (healthy)       0.0.0.0:11800->11800/tcp, :::11800->11800/tcp, 1234/tcp, 0.0.0.0:12800->12800/tcp, :::12800->12800/tcp   oap
e98b1e87d376   docker.elastic.co/elasticsearch/elasticsearch-oss:7.4.2   "/usr/local/bin/dock…"   About a minute ago   Up About a minute (healthy)   0.0.0.0:9200->9200/tcp, :::9200->9200/tcp, 9300/tcp                                                      elasticsearch
karianna commented 2 years ago

basically it's a bug in the docker engine not interacting with how it reports things to Java.

Zhannan3 commented 1 year ago

I also encountered the same problem, but my version is 20.10.0. Which version did you upgrade to to solve this problem

karianna commented 1 year ago

@Zhannan3 please try the latest 20.10.21

RohMoon commented 1 year ago

sorry , i had the same problem, and fix now from this post . and i wonder ' What does it have to do with version and memory inffluct?'

karianna commented 1 year ago

sorry , i had the same problem, and fix now from this post . and i wonder ' What does it have to do with version and memory inffluct?'

There was apparently a bug in how an older Docker version was surfacing data to the JVM

Querela commented 1 year ago

I was previously using the openjdk:11-jre-slim-bullseye (now deprecated!) image without issues and migrated to eclipse-temurin:11-jre-jammy. Suddenly this error popped up and I was first suspecting my code and not the image, so was a bit stumped until I found this issue here. I'm running docker 18.09.1 and updating is currently not possible. Is it known what causes the issue on the docker image side? (What is the difference from this image to the openjdk one?) So that I may in the meantime patch/reconfigure something to be able to use eclipse-temurin? (I did not yet test the other images suggested by the openjdk deprecation notice to see if the same issue pops up there, too.)

karianna commented 1 year ago

@Querela I can only recommend you go back to the old image if you are unable to upgrade docker core.

Querela commented 1 year ago

Ok. Well, that's currently my approach but from a security perspective not that good, since the openjdk image was deprecated for a reason. But trying a semi-permanent "temporary fix" to get the temurin image to run might also not be good. I will need to look into updating...

zimu312500 commented 1 year ago

I used this base image eclipse-temurin:11-jre-centos7 in my env with docker 19.03 to soloved this problem.The skywalking-oap-server Dockerfile use eclipse-temurin:11-jre default will occured upper probelm.

danmana commented 1 year ago

I was running into the same issue with Docker Engine 20.10.1 and eclipse-temurin:17-jre Using seccomp=unconfined was not an option on my side, because it doesn't seem to work with docker swarm.

However eclipse-temurin:17-jre-centos7 works