search

namhyung / uftrace

Function graph tracer for C/C++/Rust/Python
https://uftrace.github.io/slide/
GNU General Public License v2.0
3.06k stars 474 forks source link

Segfault with qt application #1008

Open Lorac opened 5 years ago

Lorac commented 5 years ago

The output of uftrace -v -t 1ms record ./MyQtApp

uftrace: checking binary ./MyQtApp
uftrace: removing uftrace.data.old directory
uftrace: using /usr/lib/libmcount.so library for tracing
uftrace: creating 1 thread(s) for recording
mcount: initializing mcount library
demangle: demangle failed: _ZTSPDu
demangle: demangle failed: _ZTSPKDu
demangle: demangle failed: _ZTIPKDu
demangle: demangle failed: _ZTIDu
demangle: demangle failed: _ZTIPDu
demangle: demangle failed: _ZTSDu
plthook: setup PLT hooking "/root/MyQtApp"
mcount: mcount setup done
mcount: new session started: fc7d8ed7392c7af7: MyQtApp
Segmentation fault: address not mapped (addr: (nil))
Backtrace from uftrace:
=====================================
[1] (QCoreApplication::translate[556ccfe630] <= _GLOBAL__sub_I_context.cpp[556cd004d4])
[0] (_GLOBAL__sub_I_context.cpp[556cd004a8] <= __libc_csu_init[556ce90060])
child terminated by signal: 11: Segmentation fault
uftrace: cannot find build-id section
uftrace: reading uftrace.data/task.txt file
uftrace: flushing /uftrace-fc7d8ed7392c7af7-5265-000

This is the replay uftrace replay MyQtApp

# DURATION     TID     FUNCTION
            [  5265] | _GLOBAL__sub_I_context.cpp() {
            [  5265] |   QCoreApplication::translate() {
            [  5265] |     /* linux:task-exit */

uftrace stopped tracing with remaining functions
================================================
task: 5265
[1] QCoreApplication::translate
[0] _GLOBAL__sub_I_context.cpp

If I use --no-libcall it works, but I lose a lot of information...

This is on AArch64.

namhyung commented 5 years ago

@Lorac thanks for using uftrace and the report. Which version did you use? You might use more -v option to see more detailed debug messages. It'd be nice to know which library function it tried to execute and if there's any clue for the function.

Lorac commented 5 years ago

I tried using 0.8.3 and 0.9.3 and having the same problem. Do I need to activate the instrumentation for all the libraries that I'm calling?

namhyung commented 5 years ago

Any chance you can run current master version?

Do I need to activate the instrumentation for all the libraries that I'm calling?

No you don't need to do it unless you want to trace inside the libraries.

honggyukim commented 5 years ago

It can be a lot more helpful if you can share the following info.

Lorac commented 5 years ago

These are the defines

-DQT_CORE_LIB -DQT_DBUS_LIB -DQT_DISABLE_DEPRECATED_BEFORE=0x060000 -DQT_GUI_LIB -DQT_NETWORK_LIB -DQT_NO_FOREACH -DQT_NO_NARROWING_CONVERSIONS_IN_CONNECT -DQT_QML_LIB -DQT_QUICK_LIB -DQT_USE_QSTRINGBUILDER -DQT_XML_LIB

compilation flags -finstrument-functions -O0 -g -pipe -Wall -Wextra -Wpedantic -fPIC -std=gnu++14

GCC -v

Target: aarch64-piko-linux
Configured with: ../../../../../../work-shared/gcc-8.2.0-r0/gcc-8.2.0/configure --build=x86_64-linux --host=x86_64-pikosdk-linux --target=aarch64-piko-linux --prefix=/opt/piko/2.6.1/sysroots/x86_64-pikosdk-linux/usr --exec_prefix=/opt/piko/2.6.1/sysroots/x86_64-pikosdk-linux/usr --bindir=/opt/piko/2.6.1/sysroots/x86_64-pikosdk-linux/usr/bin/aarch64-piko-linux --sbindir=/opt/piko/2.6.1/sysroots/x86_64-pikosdk-linux/usr/bin/aarch64-piko-linux --libexecdir=/opt/piko/2.6.1/sysroots/x86_64-pikosdk-linux/usr/libexec/aarch64-piko-linux --datadir=/opt/piko/2.6.1/sysroots/x86_64-pikosdk-linux/usr/share --sysconfdir=/opt/piko/2.6.1/sysroots/x86_64-pikosdk-linux/etc --sharedstatedir=/opt/piko/2.6.1/sysroots/x86_64-pikosdk-linux/com --localstatedir=/opt/piko/2.6.1/sysroots/x86_64-pikosdk-linux/var --libdir=/opt/piko/2.6.1/sysroots/x86_64-pikosdk-linux/usr/lib/aarch64-piko-linux --includedir=/opt/piko/2.6.1/sysroots/x86_64-pikosdk-linux/usr/include --oldincludedir=/opt/piko/2.6.1/sysroots/x86_64-pikosdk-linux/usr/include --infodir=/opt/piko/2.6.1/sysroots/x86_64-pikosdk-linux/usr/share/info --mandir=/opt/piko/2.6.1/sysroots/x86_64-pikosdk-linux/usr/share/man --disable-silent-rules --disable-dependency-tracking --with-libtool-sysroot=/cache/build/MASKED/tmp/work/x86_64-nativesdk-pikosdk-linux/gcc-cross-canadian-aarch64/8.2.0-r0/recipe-sysroot --with-gnu-ld --enable-shared --enable-languages=c,c++ --enable-threads=posix --enable-multilib --enable-default-pie --enable-c99 --enable-long-long --enable-symvers=gnu --enable-libstdcxx-pch --program-prefix=aarch64-piko-linux- --without-local-prefix --enable-lto --disable-libssp --enable-libitm --disable-bootstrap --disable-libmudflap --with-system-zlib --with-linker-hash-style=gnu --enable-linker-build-id --with-ppl=no --with-cloog=no --enable-checking=release --enable-cheaders=c_global --without-isl --with-gxx-include-dir=/not/exist/usr/include/c++/8.2.0 --with-build-time-tools=/cache/build/MASKED/tmp/work/x86_64-nativesdk-pikosdk-linux/gcc-cross-canadian-aarch64/8.2.0-r0/recipe-sysroot-native/usr/aarch64-piko-linux/bin --with-sysroot=/not/exist --with-build-sysroot=/cache/build/MASKED/tmp/work/x86_64-nativesdk-pikosdk-linux/gcc-cross-canadian-aarch64/8.2.0-r0/recipe-sysroot --without-long-double-128 libgcc_cv_powerpc_float128=no --enable-poison-system-directories --disable-static --enable-nls --enable-initfini-array --enable-__cxa_atexit
Thread model: posix
gcc version 8.2.0 (GCC)

Output with -vvv

...
plthook: [idx:  216] enter 55715f8360: QVariant::QVariant@plt (mod: 7f8349b170)
mcount: <1> enter 55715f8360
mcount:  tr->flags: 0, filter mode: 0, count: 0/0, depth: 1023
mcount: <2> exit  55715f8360
mcount: task 3971 recorded 32 bytes (record count = 2)
mcount: rstack[1] ENTRY 55715f8360
mcount: rstack[1] EXIT  55715f8360
plthook: [idx:  216] exit  55715f9bc4: QVariant::QVariant     (resolved addr: 7f81847a28)
plthook: [idx:  406] enter 55715f8f40: QSettings::value@plt (mod: 7f8349b170)
mcount: <1> enter 55715f8f40
mcount:  tr->flags: 0, filter mode: 0, count: 0/0, depth: 1023
mcount: task 3971 recorded 16 bytes (record count = 1)
mcount: rstack[1] ENTRY 55715f8f40
WARN: Segmentation fault: address not mapped (addr: (nil))
WARN: Backtrace from uftrace:
WARN: =====================================
WARN: [1] (QSettings::value[55715f8f40] <= main[55715f9bdc])
WARN: [0] (main[55715f9ac4] <= __libc_start_main[7f811cbce4])
uftrace: all process/thread exited
WARN: child terminated by signal: 11: Segmentation fault
...
namhyung commented 5 years ago

Thanks for the info. Do you see the segfault at the same point everytime? Can you show me your code or provide a simple reproducer?

Lorac commented 5 years ago

It always segfault at the same place. I'll try to build a simple reproducer as soon as I can.

dofmind commented 5 years ago

@Lorac any update?

Lorac commented 5 years ago 
#include <QSettings>
#include <QtCore/QCoreApplication>

int main(int argc, char *argv[]) {
  QCoreApplication a(argc, argv);
  QSettings settings;

  settings.value("something", "");

  return a.exec();
}

This application crashes.

mcount: [mod: 7fbacff170, idx: 6] enter 5573805db0: QSettings::QSettings
mcount: [mod: 7fbacff170, idx: 14] enter 5573805e30: QString::fromAscii_helper
mcount: [mod: 7fbacff170, idx: 8] enter 5573805dd0: QVariant::QVariant
mcount: [mod: 7fbacff170, idx: 0] enter 5573805d50: QSettings::value
Segmentation fault: address not mapped (addr: (nil))
Backtrace from uftrace:
=====================================
[1] (QSettings::value[5573805d50] <= main[5573806054])
[0] (main[5573805f94] <= __libc_start_main[7fba2c4ce4])
uftrace: all process/thread exited
child terminated by signal: 11: Segmentation fault
uftrace: stop writer thread 0
uftrace: cannot find build-id section
uftrace: reading uftrace.data/task.txt file
session: new task: tid = 2380
uftrace: flushing /uftrace-972f657370b6841f-2380-000
uftrace: unlink for session: 972f657370b6841f
uftrace: reading symbols for session 972f657370b6841f
Lorac commented 5 years ago

The last trace was from 0.8.3.

This is from the latest master:

plthook: [idx:    6] exit  557f53dffc: QSettings::QSettings     (resolved addr: 7fbd2e0718)
mcount: <1> enter 557f53e11c
mcount:  tr->flags: 0, filter mode: 0, count: 0/0, depth: 1023
plthook: [idx:   14] enter 557f53de30: QString::fromAscii_helper@plt (mod: 7fbd750170)
mcount: <2> enter 557f53de30
mcount:  tr->flags: 0, filter mode: 0, count: 0/0, depth: 1022
mcount: <3> exit  557f53de30
mcount: task 3206 recorded 48 bytes (record count = 3)
mcount: rstack[1] ENTRY 557f53e11c
mcount: rstack[2] ENTRY 557f53de30
mcount: rstack[2] EXIT  557f53de30
plthook: [idx:   14] exit  557f53e01c: QString::fromAscii_helper     (resolved addr: 7fbd1f8f50)
mcount: <2> exit  557f53e11c
mcount: task 3206 recorded 16 bytes (record count = 1)
mcount: rstack[1] EXIT  557f53e11c
plthook: [idx:    8] enter 557f53ddd0: QVariant::QVariant@plt (mod: 7fbd750170)
mcount: <1> enter 557f53ddd0
mcount:  tr->flags: 0, filter mode: 0, count: 0/0, depth: 1023
mcount: <2> exit  557f53ddd0
mcount: task 3206 recorded 32 bytes (record count = 2)
mcount: rstack[1] ENTRY 557f53ddd0
mcount: rstack[1] EXIT  557f53ddd0
plthook: [idx:    8] exit  557f53e040: QVariant::QVariant     (resolved addr: 7fbd385a28)
plthook: [idx:    0] enter 557f53dd50: QSettings::value@plt (mod: 7fbd750170)
mcount: <1> enter 557f53dd50
mcount:  tr->flags: 0, filter mode: 0, count: 0/0, depth: 1023
mcount: task 3206 recorded 16 bytes (record count = 1)
mcount: rstack[1] ENTRY 557f53dd50
WARN: Segmentation fault: address not mapped (addr: (nil))
WARN: Backtrace from uftrace:
WARN: =====================================
WARN: [1] (QSettings::value[557f53dd50] <= main[557f53e054])
WARN: [0] (main[557f53df94] <= __libc_start_main[7fbcd09ce4])
uftrace: all process/thread exited
WARN: child terminated by signal: 11: Segmentation fault
uftrace: stop writer thread 0
uftrace: fill header (metadata) info in uftrace.data/info
uftrace: cannot find build-id section
uftrace: reading uftrace.data/task.txt file
session: new session: pid = 3206, session = 464c39c7f781e59a
session: task session: tid = 3206, session = 464c39c7f781e59a
session: new task: tid = 3206 (untitled3), session = 464c39c7f781e59a
uftrace: flushing /uftrace-464c39c7f781e59a-3206-000
uftrace: make a new write buffer
uftrace: unlink for session: 464c39c7f781e59a
uftrace: unlink /uftrace-464c39c7f781e59a-3206-000
uftrace: unlink /uftrace-464c39c7f781e59a-3206-001
uftrace: reading symbols for session 464c39c7f781e59a
namhyung commented 5 years ago

Could you please upload the sample binary? I cannot build it :)

Lorac commented 5 years ago

This is an aarch64 binary. uftrace_dgb.zip

namhyung commented 5 years ago

Thanks I'll take a look, but now I'm in a business trip so it'll take some time. :)

honggyukim commented 5 years ago

Hi @Lorac, I tried to run your program, but it's not compile with -pg so cannot trace it.

Lorac commented 5 years ago

I compiled it with -finstrument-functions

honggyukim commented 5 years ago

I checked it again, but the file you attached do not have any tracable code.

$ aarch64-linux-gnu-nm uftrace_dgb | grep mcount
(nothing printed)

$ aarch64-linux-gnu-nm uftrace_dgb | grep cyg_
(nothing printed)
Lorac commented 4 years ago

Ok this one has mcount in it. Sorry.

uftrace_dbg.zip