apache / doris

Apache Doris is an easy-to-use, high performance and unified analytics database.
https://doris.apache.org
Apache License 2.0
12.77k stars 3.29k forks source link

[Bug] be crash #30285

Open xingyingone opened 10 months ago

xingyingone commented 10 months ago

Search before asking

Version

doris-2.0.4-rc02-e529646

What's Wrong?

the doris cluster is just execute query on external table and the cluster load is low

be crash at Jan 23 20:14 image

error log image

be.out image

What You Expected?

no crash

How to Reproduce?

I don't known, There is a point that I copy hadoop-lzo-0.4.20.jar into the /home/olap/doris/be/custom_lib/

Anything Else?

no

Are you willing to submit PR?

Code of Conduct

xingyingone commented 10 months ago

be.out start time: Wed Jan 17 20:18:19 CST 2024 INFO: java_cmd /home/olap/doris/java8/bin/java INFO: jdk_version 8 SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/olap/doris/be/lib/java_extensions/preload-extensions/preload-extensions-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/olap/doris/be/lib/java_extensions/java-udf/java-udf-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/olap/doris/be/lib/hadoop_hdfs/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory] Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /home/olap/doris/be/lib/hadoop_hdfs/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now. It's highly recommended that you fix the library with 'execstack -c ', or link it with '-z noexecstack'. start time: Tue Jan 23 14:03:31 CST 2024 INFO: java_cmd /home/olap/doris/java8/bin/java INFO: jdk_version 8 SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found binding in [jar:file:/home/olap/doris/be/lib/java_extensions/preload-extensions/preload-extensions-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/olap/doris/be/lib/java_extensions/java-udf/java-udf-jar-with-dependencies.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: Found binding in [jar:file:/home/olap/doris/be/lib/hadoop_hdfs/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class] SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation. SLF4J: Actual binding is of type [org.slf4j.impl.Reload4jLoggerFactory] Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /home/olap/doris/be/lib/hadoop_hdfs/native/libhadoop.so.1.0.0 which might have disabled stack guard. The VM will try to fix the stack guard now. It's highly recommended that you fix the library with 'execstack -c ', or link it with '-z noexecstack'. Query id: 0-0 tablet id: 0 Aborted at 1706011977 (unix time) try "date -d @1706011977" if you are using GNU date Current BE git commitID: e529646 SIGSEGV unknown detail explain (@0x0) received by PID 59922 (TID 453745 OR 0x7f232ef3c700) from PID 0; stack trace: 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t, void) at /root/src/doris-2.0/be/src/common/signal_handler.h:417 1# os::Linux::chained_handler(int, siginfo, void) in /home/olap/doris/java8/jre/lib/amd64/server/libjvm.so 2# JVM_handle_linux_signal in /home/olap/doris/java8/jre/lib/amd64/server/libjvm.so 3# signalHandler(int, siginfo, void) in /home/olap/doris/java8/jre/lib/amd64/server/libjvm.so 4# 0x00007F245E72E400 in /lib64/libc.so.6 5# Arena::~Arena() in /home/olap/doris/java8/jre/lib/amd64/server/libjvm.so 6# Thread::~Thread() in /home/olap/doris/java8/jre/lib/amd64/server/libjvm.so 7# JavaThread::~JavaThread() in /home/olap/doris/java8/jre/lib/amd64/server/libjvm.so 8# JavaThread::thread_main_inner() in /home/olap/doris/java8/jre/lib/amd64/server/libjvm.so 9# JavaThread::run() in /home/olap/doris/java8/jre/lib/amd64/server/libjvm.so 10# java_start(Thread*) in /home/olap/doris/java8/jre/lib/amd64/server/libjvm.so 11# start_thread in /lib64/libpthread.so.0 12# clone in /lib64/libc.so.6

Vallishp commented 10 months ago

from past experience, arena crash can come, when libs/jars replaced in run time in some flavors of linux, may be u can check timestamp of jar and core dump time to make sure its the same root cause.

xingyingone commented 10 months ago

from past experience, arena crash can come, when libs/jars replaced in run time in some flavors of linux, may be u can check timestamp of jar and core dump time to make sure its the same root cause.

I just copy libs/jars and then restart be, and be crash after several hours later, so not the same timestamp

morningman commented 9 months ago

Is it crash when doing query? If yes, please provide:

  1. create catalog stmt
  2. sql
  3. table schema