search

dotnet / diagnostics

This repository contains the source code for various .NET Core runtime diagnostic tools and documents.
MIT License
1.18k stars 355 forks source link

Cannot use SOS from netcore 2.1.300 under lldb-3.9 on Ubuntu18 #22

Closed raffaeler closed 5 years ago

raffaeler commented 6 years ago

I created two scripts to attach a managed process (netcore 2.1) from lldb-3.9: This one is used to load sos from the self contained deployment of the app

#!/bin/bash
pid=$1
sudo lldb-3.9 -o "process attach -p $pid" -o "plugin load ~/crash/scd/libsosplugin.so" -o "setclrpath ~/crash/scd/" -o "process handle -s false SIGUSR1 SIGUSR2"

This is the one for non-scd deployment (sos is loaded from the sdk) which does not work because of a 'Failed to load data access DLL, 0x801331c4f'

#!/bin/bash
pid=$1
sudo lldb-3.9 -o "process attach -p $pid" -o "plugin load /usr/share/dotnet/shared/Microsoft.NETCore.App/2.1.0/libsosplugin.so" -o "setclrpath /usr/share/dotnet/shared/Microsoft.NETCore.App/2.1.0/" -o "process handle -s false SIGUSR1 SIGUSR2"

After running a managed process compiled with 2.1.300, I get the pid with ps -ejH and attach it using one of the above scripts (~/lldb)

The process is attached and the plugin gets loaded:

raf@ubuntu18:~/crash/scd$ ~/lldb.sh 16436
(lldb) process attach -p 16436
Process 16436 stopped
* thread #1: tid = 16436, 0x00007fbf76ceb384 libpthread.so.0`__libc_read + 68, name = 'crash', stop reason = signal SIGSTOP
    frame #0: 0x00007fbf76ceb384 libpthread.so.0`__libc_read + 68
libpthread.so.0`__libc_read:
->  0x7fbf76ceb384 <+68>: cmpq   $-0x1000, %rax            ; imm = 0xF000 
    0x7fbf76ceb38a <+74>: ja     0x7fbf76ceb3c4            ; <+132>
    0x7fbf76ceb38c <+76>: movl   %r8d, %edi
    0x7fbf76ceb38f <+79>: movq   %rax, 0x8(%rsp)
  thread #2: tid = 16437, 0x00007fbf760c0839 libc.so.6`syscall + 25 at syscall.S:38, name = 'crash-ust', stop reason = signal SIGSTOP
    frame #0: 0x00007fbf760c0839 libc.so.6`syscall + 25 at syscall.S:38
  thread #3: tid = 16438, 0x00007fbf760c0839 libc.so.6`syscall + 25 at syscall.S:38, name = 'crash-ust', stop reason = signal SIGSTOP
    frame #0: 0x00007fbf760c0839 libc.so.6`syscall + 25 at syscall.S:38
  thread #4: tid = 16439, 0x00007fbf760b9bf9 libc.so.6`__GI___poll(fds=0x00007fbf72d5ade8, nfds=1, timeout=-1) + 73 at poll.c:29, name = 'crash', stop reason = signal SIGSTOP
    frame #0: 0x00007fbf760b9bf9 libc.so.6`__GI___poll(fds=0x00007fbf72d5ade8, nfds=1, timeout=-1) + 73 at poll.c:29
  thread #5: tid = 16440, 0x00007fbf76cebdae libpthread.so.0`__libc_open64 + 206, name = 'crash', stop reason = signal SIGSTOP
    frame #0: 0x00007fbf76cebdae libpthread.so.0`__libc_open64 + 206
libpthread.so.0`__libc_open64:
->  0x7fbf76cebdae <+206>: cmpq   $-0x1000, %rax            ; imm = 0xF000 
    0x7fbf76cebdb4 <+212>: ja     0x7fbf76cebde6            ; <+262>
    0x7fbf76cebdb6 <+214>: movl   %r8d, %edi
    0x7fbf76cebdb9 <+217>: movl   %eax, 0x8(%rsp)
  thread #6: tid = 16441, 0x00007fbf76ce79f3 libpthread.so.0`__pthread_cond_wait + 579, name = 'crash', stop reason = signal SIGSTOP
    frame #0: 0x00007fbf76ce79f3 libpthread.so.0`__pthread_cond_wait + 579
libpthread.so.0`__pthread_cond_wait:
->  0x7fbf76ce79f3 <+579>: cmpq   $-0x1000, %rax            ; imm = 0xF000 
    0x7fbf76ce79f9 <+585>: movq   0x30(%rsp), %r8
    0x7fbf76ce79fe <+590>: ja     0x7fbf76ce7b30            ; <+896>
    0x7fbf76ce7a04 <+596>: movl   %r9d, %edi
  thread #7: tid = 16442, 0x00007fbf76ce7ed9 libpthread.so.0`__pthread_cond_timedwait + 649, name = 'crash', stop reason = signal SIGSTOP
    frame #0: 0x00007fbf76ce7ed9 libpthread.so.0`__pthread_cond_timedwait + 649
libpthread.so.0`__pthread_cond_timedwait:
->  0x7fbf76ce7ed9 <+649>: cmpq   $-0x1000, %rax            ; imm = 0xF000 
    0x7fbf76ce7edf <+655>: ja     0x7fbf76ce81c0            ; <+1392>
    0x7fbf76ce7ee5 <+661>: movl   %r8d, %edi
    0x7fbf76ce7ee8 <+664>: callq  0x7fbf76ceb060            ; __pthread_disable_asynccancel
  thread #8: tid = 16443, 0x00007fbf76ceb384 libpthread.so.0`__libc_read + 68, name = 'crash', stop reason = signal SIGSTOP
    frame #0: 0x00007fbf76ceb384 libpthread.so.0`__libc_read + 68
libpthread.so.0`__libc_read:
->  0x7fbf76ceb384 <+68>: cmpq   $-0x1000, %rax            ; imm = 0xF000 
    0x7fbf76ceb38a <+74>: ja     0x7fbf76ceb3c4            ; <+132>
    0x7fbf76ceb38c <+76>: movl   %r8d, %edi
    0x7fbf76ceb38f <+79>: movq   %rax, 0x8(%rsp)

Executable module set to "/home/raf/crash/scd/crash".
Architecture set to: x86_64-pc-linux.
(lldb) plugin load ./libsosplugin.so
(lldb) setclrpath ./
(lldb) process handle -s false SIGUSR1 SIGUSR2
NAME         PASS   STOP   NOTIFY
===========  =====  =====  ======
SIGUSR1      true   false  true 
SIGUSR2      true   false  true

At this point the pe is issued and does not return any result (correct). But the clrstack results in a Sementation fault. 

(lldb) pe
(lldb) clrstack
(lldb) 
/home/raf/lldb.sh: line 5: 16459 Segmentation fault      sudo lldb-3.9 -o "process attach -p $pid" -o "plugin load ./libsosplugin.so" -o "setclrpath ./" -o "process handle -s false SIGUSR1 SIGUSR2"

After a number of other tests I believe the problem resides in problems loading the required libraries.

mikem8361 commented 5 years ago

Can you try the diagnostics repo sos/new instructions here?

mikem8361 commented 5 years ago

Does this still repro?