Attach failed with error - Interpreter doesn't handle one of the expression's opcodes

Arulanand commented 1 year ago

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior

I am trying to profile a Python application and it fails with the below error.

Executable module set to "/usr/local/bin/python3.9".
Architecture set to: x86_64-pc-linux-gnu.
(lldb) expr char $libpath[]="/home/airflow/.local/lib/python3.9/site-packages/memray/_inject.abi3.so"
error: Can't evaluate the expression without a running target due to: Interpreter doesn't handle one of the expression's opcodes

Failed to execute our lldb script.

Complete stack trace

root@90000861716b:/opt/airflow# memray attach 4644 --verbose
Debugger command line: lldb --batch -p 4644 --no-lldbinit -o 'expr char $libpath[]="/home/airflow/.local/lib/python3.9/site-packages/memray/_inject.abi3.so"' -o 'expr int $port=51813' -o 'expr void* $rtld_default=(void*)0' -o 'expr int $rtld_now=2' --source /home/airflow/.local/lib/python3.9/site-packages/memray/commands/_attach.lldb
debugger return code: 1
debugger output:
(lldb) process attach --pid 4644
warning: (x86_64) /home/airflow/.local/lib/python3.9/site-packages/numpy.libs/libgfortran-040039e1.so.5.0.0 No LZMA support found for reading .gnu_debugdata section
warning: (x86_64) /home/airflow/.local/lib/python3.9/site-packages/Pillow.libs/libXau-154567c4.so.6.0.0 No LZMA support found for reading .gnu_debugdata section
warning: (x86_64) /home/airflow/.local/lib/python3.9/site-packages/tokenizers.libs/libssl-cd1d6220.so.1.0.2k No LZMA support found for reading .gnu_debugdata section
warning: (x86_64) /home/airflow/.local/lib/python3.9/site-packages/tokenizers.libs/libcrypto-d3570994.so.1.0.2k No LZMA support found for reading .gnu_debugdata section
warning: (x86_64) /home/airflow/.local/lib/python3.9/site-packages/tokenizers.libs/libgssapi_krb5-497db0c6.so.2.2 No LZMA support found for reading .gnu_debugdata section
warning: (x86_64) /home/airflow/.local/lib/python3.9/site-packages/tokenizers.libs/libkrb5-fcafa220.so.3.3 No LZMA support found for reading .gnu_debugdata section
warning: (x86_64) /home/airflow/.local/lib/python3.9/site-packages/tokenizers.libs/libcom_err-2abe824b.so.2.1 No LZMA support found for reading .gnu_debugdata section
warning: (x86_64) /home/airflow/.local/lib/python3.9/site-packages/tokenizers.libs/libk5crypto-b1f99d5c.so.3.1 No LZMA support found for reading .gnu_debugdata section
warning: (x86_64) /home/airflow/.local/lib/python3.9/site-packages/tokenizers.libs/libkrb5support-d0bcff84.so.0.1 No LZMA support found for reading .gnu_debugdata section
warning: (x86_64) /home/airflow/.local/lib/python3.9/site-packages/tokenizers.libs/libkeyutils-dfe70bd6.so.1.5 No LZMA support found for reading .gnu_debugdata section
warning: (x86_64) /home/airflow/.local/lib/python3.9/site-packages/tokenizers.libs/libselinux-0922c95c.so.1 No LZMA support found for reading .gnu_debugdata section
warning: (x86_64) /home/airflow/.local/lib/python3.9/site-packages/tokenizers.libs/libpcre-9513aab5.so.1.2.0 No LZMA support found for reading .gnu_debugdata section
Process 4644 stopped
* thread #1, name = 'airflow task ru', stop reason = signal SIGSTOP
    frame #0: 0x00007fac9d52c70e libpython3.9.so.1.0`PyDict_SetItem + 110
libpython3.9.so.1.0`PyDict_SetItem:
->  0x7fac9d52c70e <+110>: leaq   0x18b10b(%rip), %rcx
    0x7fac9d52c715 <+117>: cmpq   %rcx, 0x20(%rbp)
    0x7fac9d52c719 <+121>: movq   (%rsp), %rsi
    0x7fac9d52c71d <+125>: movq   0x8(%rsp), %rcx
  thread #2, name = 'airflow task ru', stop reason = signal SIGSTOP
    frame #0: 0x00007fac9d2c6d2f libc.so.6
->  0x7fac9d2c6d2f:       cmpq   $-0x1000, %rax            ; imm = 0xF000
libc.so.6`__ulimit:
    0x7fac9d2c6d35 <+5>:  ja     0x7fac9d2c6d68            ; <+56> at ulimit.c:44:3
    0x7fac9d2c6d37 <+7>:  movl   %r8d, %edi
    0x7fac9d2c6d3a <+10>: movl   %eax, 0x8(%rsp)
  thread #3, name = 'airflow task ru', stop reason = signal SIGSTOP
    frame #0: 0x00007fac9d2c6d2f libc.so.6
->  0x7fac9d2c6d2f:       cmpq   $-0x1000, %rax            ; imm = 0xF000
libc.so.6`__ulimit:
    0x7fac9d2c6d35 <+5>:  ja     0x7fac9d2c6d68            ; <+56> at ulimit.c:44:3
    0x7fac9d2c6d37 <+7>:  movl   %r8d, %edi
    0x7fac9d2c6d3a <+10>: movl   %eax, 0x8(%rsp)
  thread #4, name = 'airflow task ru', stop reason = signal SIGSTOP
    frame #0: 0x00007fac9d2c2564 libc.so.6`__GI___getcwd(buf=<unavailable>, size=<unavailable>) at getcwd.c:111:7
  thread #5, name = 'airflow task ru', stop reason = signal SIGSTOP
    frame #0: 0x00007fac9d0647b2 libpthread.so.0`__pthread_cond_wait at futex-internal.h:186:13
  thread #6, name = '.NET Finalizer', stop reason = signal SIGSTOP
    frame #0: 0x00007fac9d064ad8 libpthread.so.0`__pthread_cond_timedwait at futex-internal.h:323:13

Executable module set to "/usr/local/bin/python3.9".
Architecture set to: x86_64-pc-linux-gnu.
(lldb) expr char $libpath[]="/home/airflow/.local/lib/python3.9/site-packages/memray/_inject.abi3.so"
error: Can't evaluate the expression without a running target due to: Interpreter doesn't handle one of the expression's opcodes

Failed to execute our lldb script.
Run with --verbose to debug the failure.

Expected Behavior

No response

Steps To Reproduce

Run memray attach as below.

Memray Version

1.8.1

Python Version

3.9

Operating System

Linux

Anything else?

No response

godlygeek commented 1 year ago

There isn't enough information here for us to be able to investigate this. It looks like it happened in a Docker container - are you able to share a Dockerfile that reproduces the issue? Or a set of steps that can be performed in order for us to reproduce the issue?

What version of lldb was in use? What version of Linux? Do things work properly if you install gdb?

godlygeek commented 1 year ago

It does sound like there's some issue here, but there's not enough information here for us to be able to track down what it might be. If you're able to reproduce this problem, we can use your help debugging this. Better yet, if you can give us a Dockerfile that reproduces the issue, we could debug this without any further help from you. But without either of those, there's just no way for us to track this down.

bloomberg / memray