llvm / llvm-project

The LLVM Project is a collection of modular and reusable compiler and toolchain technologies.
http://llvm.org
Other
29.32k stars 12.11k forks source link

lldb cannot properly run a program file on armv7-alpine-linux-musleabihf #75198

Open DavidSpickett opened 11 months ago

DavidSpickett commented 11 months ago

This was originally reported as https://discourse.llvm.org/t/no-debug-symbols-in-armv7-alpine-linux-musleabihf/75599, I'm going to summarise my findings here.

This is using Alpine 3.19 pulled from docker hub, and when building lldb from source I am first installing the packages listed in https://git.alpinelinux.org/aports/tree/community/lldb/APKBUILD.

I tried this with installed versions 17.0.5 and built from main around a9b30545448695c76ddb25a16fae613d641cfeb7 which is an 18.x.

On armv7, lldb seemingly fails to deduce the program's triple from the ELF file:

/build-llvm # lldb -b -o "log enable lldb all" -o "target create /tmp/test.o" -o "run"
(lldb) log enable lldb all
(lldb) target create /tmp/test.o
 Processing command: target create /tmp/test.o
 HandleCommand, cmd_obj : 'target create'
 HandleCommand, (revised) command_string: 'target create /tmp/test.o'
 HandleCommand, wants_raw_input:'False'
 HandleCommand, command line after removing command name(s): '/tmp/test.o'
 ObjectFileELF::GetModuleSpecifications file '/tmp/test.o' module OSABI: ELFOSABI_NONE
 ObjectFileELF::RefineModuleDetailsFromNote parsing note name='GNU', type=3
 ObjectFileELF::GetModuleSpecifications file '/tmp/test.o' module set to triple: arm---eabihf (architecture arm)
 force = false, arch=(arm, arm---eabihf)
 create = true
 0xeb58c200 Platform::Platform()
 0xeb58c7e0 Module::Module((arm) '/tmp/test.o')
 ObjectFileELF::GetModuleSpecifications file '/tmp/test.o' module OSABI: ELFOSABI_NONE
 ObjectFileELF::RefineModuleDetailsFromNote parsing note name='GNU', type=3
 ObjectFileELF::GetModuleSpecifications file '/tmp/test.o' module set to triple: arm---eabihf (architecture arm)
 0xeb58cf90 ObjectFile::ObjectFile() module = 0xeb58c7e0 (/tmp/test.o), file = /tmp/test.o, file_offset = 0x00000000, size = 10812

It's my assumption that not detecting some form of Linux here causes problems later as we get:

Target::Launch the platform doesn't know how to debug a process, getting a process plugin to do this for us.

Then we get an error saying the process couldn't be started.

It is possible to get the process started by specifying a slightly different triple as shown in the Discourse post, but this then loses symbols because it picks the wrong dynamic loader plugin. Again I think because we don't know this is Linux and it falls through to the Darwin loader plugin which of course finds nothing.

This does not happen on the arm64 version of the same container:

(lldb) target create /tmp/test.o
 Processing command: target create /tmp/test.o
 HandleCommand, cmd_obj : 'target create'
 HandleCommand, (revised) command_string: 'target create /tmp/test.o'
 HandleCommand, wants_raw_input:'False'
 HandleCommand, command line after removing command name(s): '/tmp/test.o'
 ObjectFileELF::GetModuleSpecifications file '/tmp/test.o' module OSABI: ELFOSABI_NONE
 ObjectFileELF::RefineModuleDetailsFromNote parsing note name='GNU', type=3
 ObjectFileELF::GetModuleSpecifications file '/tmp/test.o' module set to triple: aarch64-- (architecture aarch64)
 0xffff77861bd0 Module::Module((aarch64) '/tmp/test.o')
 ObjectFileELF::GetModuleSpecifications file '/tmp/test.o' module OSABI: ELFOSABI_NONE
 ObjectFileELF::RefineModuleDetailsFromNote parsing note name='GNU', type=3
 ObjectFileELF::GetModuleSpecifications file '/tmp/test.o' module set to triple: aarch64-- (architecture aarch64)
 0xffff778c2cf0 ObjectFile::ObjectFile() module = 0xffff77861bd0 (/tmp/test.o), file = /tmp/test.o, file_offset = 0x00000000, size = 73624
 ObjectFileELF::RefineModuleDetailsFromNote parsing note name='GNU', type=3
 0x0000FFFF7787B968 Broadcaster::Broadcaster("lldb.target")
(lldb) run
 Processing command: run
 HandleCommand, cmd_obj : 'process launch'
 HandleCommand, (revised) command_string: 'process launch -c/bin/ash --'
 HandleCommand, wants_raw_input:'False'
 Repeat command:
 HandleCommand, command line after removing command name(s): '-c/bin/ash --'
 Target::Launch() called for /tmp/test.o
 Target::Launch the process instance doesn't currently exist.
 have platform=true, platform_sp->IsHost()=true, default_to_use_pty=true
 at least one of stdin/stdout/stderr was not set, evaluating default handling
 target stdin='(empty)', target stdout='(empty)', stderr='(empty)'
 Generating a pty to use for stdin/out/err
 0xffff77891eb0 Listener::Listener('lldb.internal.Process.LaunchSynchronous.hijack')
 Target::Launch asking the platform to debug the process
 target 0x0000FFFF7787B720
 having target create process with gdb-remote plugin

Note:

 having target create process with gdb-remote plugin

So I think in the armv7 case the plugin is being asked if it can debug <some triple> and that triple is not correct.

LLDB does not have any specific references to musl in the source code, the most we have is a few fixes for building against it. So it's likely we're making some glibc specific choices if we're on Linux and/or treating this binary as a bare metal binary.

llvmbot commented 11 months ago

@llvm/issue-subscribers-lldb

Author: David Spickett (DavidSpickett)

This was originally reported as https://discourse.llvm.org/t/no-debug-symbols-in-armv7-alpine-linux-musleabihf/75599, I'm going to summarise my findings here. This is using Alpine 3.19 pulled from docker hub, and when building lldb from source I am first installing the packages listed in https://git.alpinelinux.org/aports/tree/community/lldb/APKBUILD. I tried this with installed versions 17.0.5 and built from `main` around `a9b30545448695c76ddb25a16fae613d641cfeb7` which is an 18.x. On armv7, lldb seemingly fails to deduce the program's triple from the ELF file: ``` /build-llvm # lldb -b -o "log enable lldb all" -o "target create /tmp/test.o" -o "run" (lldb) log enable lldb all (lldb) target create /tmp/test.o Processing command: target create /tmp/test.o HandleCommand, cmd_obj : 'target create' HandleCommand, (revised) command_string: 'target create /tmp/test.o' HandleCommand, wants_raw_input:'False' HandleCommand, command line after removing command name(s): '/tmp/test.o' ObjectFileELF::GetModuleSpecifications file '/tmp/test.o' module OSABI: ELFOSABI_NONE ObjectFileELF::RefineModuleDetailsFromNote parsing note name='GNU', type=3 ObjectFileELF::GetModuleSpecifications file '/tmp/test.o' module set to triple: arm---eabihf (architecture arm) force = false, arch=(arm, arm---eabihf) create = true 0xeb58c200 Platform::Platform() 0xeb58c7e0 Module::Module((arm) '/tmp/test.o') ObjectFileELF::GetModuleSpecifications file '/tmp/test.o' module OSABI: ELFOSABI_NONE ObjectFileELF::RefineModuleDetailsFromNote parsing note name='GNU', type=3 ObjectFileELF::GetModuleSpecifications file '/tmp/test.o' module set to triple: arm---eabihf (architecture arm) 0xeb58cf90 ObjectFile::ObjectFile() module = 0xeb58c7e0 (/tmp/test.o), file = /tmp/test.o, file_offset = 0x00000000, size = 10812 ``` It's my assumption that not detecting some form of Linux here causes problems later as we get: ``` Target::Launch the platform doesn't know how to debug a process, getting a process plugin to do this for us. ``` Then we get an error saying the process couldn't be started. It is possible to get the process started by specifying a slightly different triple as shown in the Discourse post, but this then loses symbols because it picks the wrong dynamic loader plugin. Again I think because we don't know this is Linux and it falls through to the Darwin loader plugin which of course finds nothing. This does not happen on the arm64 version of the same container: ``` (lldb) target create /tmp/test.o Processing command: target create /tmp/test.o HandleCommand, cmd_obj : 'target create' HandleCommand, (revised) command_string: 'target create /tmp/test.o' HandleCommand, wants_raw_input:'False' HandleCommand, command line after removing command name(s): '/tmp/test.o' ObjectFileELF::GetModuleSpecifications file '/tmp/test.o' module OSABI: ELFOSABI_NONE ObjectFileELF::RefineModuleDetailsFromNote parsing note name='GNU', type=3 ObjectFileELF::GetModuleSpecifications file '/tmp/test.o' module set to triple: aarch64-- (architecture aarch64) 0xffff77861bd0 Module::Module((aarch64) '/tmp/test.o') ObjectFileELF::GetModuleSpecifications file '/tmp/test.o' module OSABI: ELFOSABI_NONE ObjectFileELF::RefineModuleDetailsFromNote parsing note name='GNU', type=3 ObjectFileELF::GetModuleSpecifications file '/tmp/test.o' module set to triple: aarch64-- (architecture aarch64) 0xffff778c2cf0 ObjectFile::ObjectFile() module = 0xffff77861bd0 (/tmp/test.o), file = /tmp/test.o, file_offset = 0x00000000, size = 73624 ObjectFileELF::RefineModuleDetailsFromNote parsing note name='GNU', type=3 0x0000FFFF7787B968 Broadcaster::Broadcaster("lldb.target") ``` ``` (lldb) run Processing command: run HandleCommand, cmd_obj : 'process launch' HandleCommand, (revised) command_string: 'process launch -c/bin/ash --' HandleCommand, wants_raw_input:'False' Repeat command: HandleCommand, command line after removing command name(s): '-c/bin/ash --' Target::Launch() called for /tmp/test.o Target::Launch the process instance doesn't currently exist. have platform=true, platform_sp->IsHost()=true, default_to_use_pty=true at least one of stdin/stdout/stderr was not set, evaluating default handling target stdin='(empty)', target stdout='(empty)', stderr='(empty)' Generating a pty to use for stdin/out/err 0xffff77891eb0 Listener::Listener('lldb.internal.Process.LaunchSynchronous.hijack') Target::Launch asking the platform to debug the process target 0x0000FFFF7787B720 having target create process with gdb-remote plugin ``` Note: ``` having target create process with gdb-remote plugin ``` So I think in the armv7 case the plugin is being asked if it can debug `<some triple>` and that triple is not correct. LLDB does not have any specific references to musl in the source code, the most we have is a few fixes for building against it. So it's likely we're making some glibc specific choices if we're on Linux and/or treating this binary as a bare metal binary.
andypost commented 8 months ago

there's attempt to upgrade it but llvm18 on 32-bit ARM is broken for Alpine https://gitlab.alpinelinux.org/alpine/aports/-/merge_requests/61854

and there's failed tests https://github.com/llvm/llvm-project/issues/69455