mcjit / orc cannot load debug information in windows

[ZLangJIT]

#include <stdio.h>

int j() {
  printf("HELLO JIT\n");
  return 0;
}

clang -c jit_code.c -o jit_code.o -O0 -g3 -Xclang -triple -Xclang "x86_64-pc-windows-elf"

llvm-jitlink -- jit_code.o --entry=j --debugger-support llvm-jitlink.exe: Symbols not found: [ llvm_orc_registerJITLoaderGDBWrapper ]

Using -no-process-syms=true doesnt help

llvm 18.1.8

[llvmbot]

@llvm/issue-subscribers-jitlink

Author: None (ZLangJIT)

``` #include <stdio.h> int j() { printf("HELLO JIT\n"); return 0; } ``` clang -c jit_code.c -o jit_code.o -O0 -g3 -Xclang -triple -Xclang "x86_64-pc-windows-elf" llvm-jitlink -- jit_code.o --entry=j --debugger-support llvm-jitlink.exe: Symbols not found: [ llvm_orc_registerJITLoaderGDBWrapper ] Using `-no-process-syms=true` doesnt help llvm 18.1.8

[weliveindetail]

Interesting, the llvm-jitlink tool should export this symbol. Can you dump them (e.g. with llvm-nm) and see if llvm_orc_registerJITLoaderGDBWrapper is there? Otherwise, does it work with lli?

[ZLangJIT]

Hmm

+ llvm-nm llvm-18/bin/llvm-jitlink.exe
+ grep --color=always -i llvm_orc
llvm-18/bin/llvm-jitlink.exe: no symbols
+ true
+ llvm-nm llvm-18/bin/llvm-jitlink.exe -D
+ grep --color=always -i llvm_orc
llvm-18/bin/llvm-jitlink.exe: File format has no dynamic symbol table
+ true

[ZLangJIT]

ok so i searched the libs for any related gdb symbols and found this

testing llvm-nm "llvm-18/lib/LLVMOrcTargetProcess.lib"
         U llvm_orc_deregisterEHFrameSectionWrapper
         U llvm_orc_registerEHFrameSectionWrapper
00000000 T llvm_orc_deregisterEHFrameSectionWrapper
00000000 T llvm_orc_registerEHFrameSectionWrapper
         U llvm_orc_deregisterEHFrameSectionWrapper
         U llvm_orc_registerEHFrameSectionWrapper
00000000 T llvm_orc_registerJITLoaderPerfEnd
00000000 T llvm_orc_registerJITLoaderPerfImpl
00000000 T llvm_orc_registerJITLoaderPerfStart
ExecutionEngine/Orc/TargetProcess/CMakeFiles/LLVMOrcTargetProcess.dir/JITLoaderGDB.cpp.obj:
00000000 T llvm_orc_registerJITLoaderGDBAllocAction
00000000 T llvm_orc_registerJITLoaderGDBWrapper
contains symbols: "llvm-18/lib/LLVMOrcTargetProcess.lib"
testing llvm-nm -D "llvm-18/lib/LLVMOrcTargetProcess.lib"
D:\a\jit_debug_test\jit_debug_test\llvm-18\bin\llvm-nm.exe: error: ExecutionEngine/Orc/TargetProcess/CMakeFiles/LLVMOrcTargetProcess.dir/JITLoaderGDB.cpp.obj: File format has no dynamic symbol table
ExecutionEngine/Orc/TargetProcess/CMakeFiles/LLVMOrcTargetProcess.dir/JITLoaderGDB.cpp.obj:
contains symbols: "llvm-18/lib/LLVMOrcTargetProcess.lib"
testing llvm-objdump -t "llvm-18/lib/LLVMOrcTargetProcess.lib"
[596](sec  0)(fl 0x00)(ty   0)(scl   2) (nx 0) 0x00000000 llvm_orc_registerEHFrameSectionWrapper
[598](sec  0)(fl 0x00)(ty   0)(scl   2) (nx 0) 0x00000000 llvm_orc_deregisterEHFrameSectionWrapper
[18](sec  6)(fl 0x00)(ty  20)(scl   2) (nx 0) 0x00000000 llvm_orc_registerEHFrameSectionWrapper
[23](sec  7)(fl 0x00)(ty  20)(scl   2) (nx 0) 0x00000000 llvm_orc_deregisterEHFrameSectionWrapper
[343](sec  0)(fl 0x00)(ty   0)(scl   2) (nx 0) 0x00000000 llvm_orc_registerEHFrameSectionWrapper
[345](sec  0)(fl 0x00)(ty   0)(scl   2) (nx 0) 0x00000000 llvm_orc_deregisterEHFrameSectionWrapper
[ 8](sec  4)(fl 0x00)(ty  20)(scl   2) (nx 0) 0x00000000 llvm_orc_registerJITLoaderPerfImpl
[13](sec  5)(fl 0x00)(ty  20)(scl   2) (nx 0) 0x00000000 llvm_orc_registerJITLoaderPerfStart
[18](sec  6)(fl 0x00)(ty  20)(scl   2) (nx 0) 0x00000000 llvm_orc_registerJITLoaderPerfEnd
[11](sec  5)(fl 0x00)(ty  20)(scl   2) (nx 0) 0x00000000 llvm_orc_registerJITLoaderGDBAllocAction
[16](sec  6)(fl 0x00)(ty  20)(scl   2) (nx 0) 0x00000000 llvm_orc_registerJITLoaderGDBWrapper
AUX JITLoaderGDB.cpp
contains symbols: "llvm-18/lib/LLVMOrcTargetProcess.lib"

[ZLangJIT]

hmm

in the llvm source they are just extern "C" but not dllexport

https://github.com/llvm/llvm-project/blob/release/18.x/llvm/include/llvm/ExecutionEngine/Orc/TargetProcess/JITLoaderGDB.h

https://github.com/llvm/llvm-project/blob/release/18.x/llvm/lib/ExecutionEngine/Orc/TargetProcess/JITLoaderGDB.cpp

[ZLangJIT]

I tried compiling with this but it crashes when run

#ifdef _WIN32
#include <llvm/ExecutionEngine/Orc/Shared/WrapperFunctionUtils.h>
#endif

// GDB and LLDB support debugging of JIT-compiled code by observing calls to __jit_debug_register_code()
// by putting a breakpoint on it, and retrieving the debug info through __jit_debug_descriptor.
// On Linux it suffices for these symbols not to be stripped out, while for Windows a .pdb has to contain
// their information. LLVM defines them, but we don't want a huge .pdb with all LLVM source code's debug
// info. By forward-declaring them here it suffices to compile this file with /Zi.

// JIT_DLL_EXPORT requires complete types, no forward-declaring

extern "C" struct jit_descriptor;
extern "C" struct jit_descriptor __jit_debug_descriptor;

extern "C" JIT_DLL_EXPORT void __jit_debug_register_code();

#ifdef _WIN32

extern "C" JIT_DLL_EXPORT llvm::orc::shared::CWrapperFunctionResult
llvm_orc_registerJITLoaderGDBWrapper(const char *Data, uint64_t Size);

extern "C" JIT_DLL_EXPORT llvm::orc::shared::CWrapperFunctionResult
llvm_orc_registerJITLoaderGDBAllocAction(const char *Data, size_t Size);

#endif

#if true

// https://raw.githubusercontent.com/llvm/llvm-project/release/18.x/llvm/lib/ExecutionEngine/Orc/TargetProcess/JITLoaderGDB.cpp

//===- JITLoaderGDB.h - Register objects via GDB JIT interface -*- C++ -*-===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//

#include "llvm/ExecutionEngine/Orc/TargetProcess/JITLoaderGDB.h"

#include "llvm/ExecutionEngine/JITSymbol.h"
#include "llvm/Support/BinaryStreamReader.h"
#include "llvm/Support/FormatVariadic.h"

#include <cstdint>
#include <mutex>
#include <utility>

#define DEBUG_TYPE "orc"

// First version as landed in August 2009
static constexpr uint32_t JitDescriptorVersion = 1;

extern "C" {

// We put information about the JITed function in this global, which the
// debugger reads.  Make sure to specify the version statically, because the
// debugger checks the version before we can set it during runtime.
struct jit_descriptor __jit_debug_descriptor = {JitDescriptorVersion, 0,
                                                nullptr, nullptr};

// Debuggers that implement the GDB JIT interface put a special breakpoint in
// this function.
LLVM_ATTRIBUTE_NOINLINE void __jit_debug_register_code() {
  // The noinline and the asm prevent calls to this function from being
  // optimized out.
#if !defined(_MSC_VER)
  asm volatile("" ::: "memory");
#endif
}
}

using namespace llvm;
using namespace llvm::orc;

// Register debug object, return error message or null for success.
static void appendJITDebugDescriptor(const char *ObjAddr, size_t Size) {
  LLVM_DEBUG({
    dbgs() << "Adding debug object to GDB JIT interface "
           << formatv("([{0:x16} -- {1:x16}])",
                      reinterpret_cast<uintptr_t>(ObjAddr),
                      reinterpret_cast<uintptr_t>(ObjAddr + Size))
           << "\n";
  });

  jit_code_entry *E = new jit_code_entry;
  E->symfile_addr = ObjAddr;
  E->symfile_size = Size;
  E->prev_entry = nullptr;

  // Serialize rendezvous with the debugger as well as access to shared data.
  static std::mutex JITDebugLock;
  std::lock_guard<std::mutex> Lock(JITDebugLock);

  // Insert this entry at the head of the list.
  jit_code_entry *NextEntry = __jit_debug_descriptor.first_entry;
  E->next_entry = NextEntry;
  if (NextEntry) {
    NextEntry->prev_entry = E;
  }

  __jit_debug_descriptor.first_entry = E;
  __jit_debug_descriptor.relevant_entry = E;
  __jit_debug_descriptor.action_flag = JIT_REGISTER_FN;
}

extern "C" orc::shared::CWrapperFunctionResult
llvm_orc_registerJITLoaderGDBAllocAction(const char *Data, size_t Size) {
  using namespace orc::shared;
  return WrapperFunction<SPSError(SPSExecutorAddrRange, bool)>::handle(
             Data, Size,
             [](ExecutorAddrRange R, bool AutoRegisterCode) {
               appendJITDebugDescriptor(R.Start.toPtr<const char *>(),
                                        R.size());
               // Run into the rendezvous breakpoint.
               if (AutoRegisterCode)
                 __jit_debug_register_code();
               return Error::success();
             })
      .release();
}

extern "C" orc::shared::CWrapperFunctionResult
llvm_orc_registerJITLoaderGDBWrapper(const char *Data, uint64_t Size) {
  using namespace orc::shared;
  return WrapperFunction<SPSError(SPSExecutorAddrRange, bool)>::handle(
             Data, Size,
             [](ExecutorAddrRange R, bool AutoRegisterCode) {
               appendJITDebugDescriptor(R.Start.toPtr<const char *>(),
                                        R.size());
               // Run into the rendezvous breakpoint.
               if (AutoRegisterCode)
                 __jit_debug_register_code();
               return Error::success();
             })
      .release();
}
#endif

./ROOTFS_DEBUG/bin/jit
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.  Program arguments: D:\\a\\jit_debug_test\\jit_debug_test\\ROOTFS_DEBUG\\bin\\jit.exe
Exception Code: 0xC0000005
symbol address for main is 00007FF6EC999BE0
symbol address for __jit_debug_descriptor is 0000000000000000
symbol address for __jit_debug_register_code is 00007FF6EC999A30
symbol address for llvm_orc_registerJITLoaderGDBWrapper is 00007FF6EC999B10
symbol address for llvm_orc_registerJITLoaderGDBAllocAction is 00007FF6EC999A40
 #0 0x00007ff6edeb3f3d llvm::jitlink::JITLinkerBase::linkPhase1(class std::unique_ptr<class llvm::jitlink::JITLinkerBase, struct std::default_delete<class llvm::jitlink::JITLinkerBase>>) (D:\a\jit_debug_test\jit_debug_test\ROOTFS_DEBUG\bin\jit.exe+0x1973f3d)
 #1 0x00007ff6eddf54fc llvm::jitlink::link_ELF_x86_64(class std::unique_ptr<class llvm::jitlink::LinkGraph, struct std::default_delete<class llvm::jitlink::LinkGraph>>, class std::unique_ptr<class llvm::jitlink::JITLinkContext, struct std::default_delete<class llvm::jitlink::JITLinkContext>>) (D:\a\jit_debug_test\jit_debug_test\ROOTFS_DEBUG\bin\jit.exe+0x18b54fc)
 #2 0x00007ff6ed8b4c3b llvm::jitlink::link_ELF(class std::unique_ptr<class llvm::jitlink::LinkGraph, struct std::default_delete<class llvm::jitlink::LinkGraph>>, class std::unique_ptr<class llvm::jitlink::JITLinkContext, struct std::default_delete<class llvm::jitlink::JITLinkContext>>) (D:\a\jit_debug_test\jit_debug_test\ROOTFS_DEBUG\bin\jit.exe+0x1374c3b)
 #3 0x00007ff6ecefbb14 llvm::jitlink::link(class std::unique_ptr<class llvm::jitlink::LinkGraph, struct std::default_delete<class llvm::jitlink::LinkGraph>>, class std::unique_ptr<class llvm::jitlink::JITLinkContext, struct std::default_delete<class llvm::jitlink::JITLinkContext>>) (D:\a\jit_debug_test\jit_debug_test\ROOTFS_DEBUG\bin\jit.exe+0x9bbb14)
 #4 0x00007ff6ec9a7b1b llvm::orc::ObjectLinkingLayer::emit(class std::unique_ptr<class llvm::orc::MaterializationResponsibility, struct std::default_delete<class llvm::orc::MaterializationResponsibility>>, class std::unique_ptr<class llvm::MemoryBuffer, struct std::default_delete<class llvm::MemoryBuffer>>) (D:\a\jit_debug_test\jit_debug_test\ROOTFS_DEBUG\bin\jit.exe+0x467b1b)
 #5 0x00007ff6ecf118a1 llvm::orc::ObjectTransformLayer::emit(class std::unique_ptr<class llvm::orc::MaterializationResponsibility, struct std::default_delete<class llvm::orc::MaterializationResponsibility>>, class std::unique_ptr<class llvm::MemoryBuffer, struct std::default_delete<class llvm::MemoryBuffer>>) (D:\a\jit_debug_test\jit_debug_test\ROOTFS_DEBUG\bin\jit.exe+0x9d18a1)
 #6 0x00007ff6ecf11cba llvm::orc::IRCompileLayer::emit(class std::unique_ptr<class llvm::orc::MaterializationResponsibility, struct std::default_delete<class llvm::orc::MaterializationResponsibility>>, class llvm::orc::ThreadSafeModule) (D:\a\jit_debug_test\jit_debug_test\ROOTFS_DEBUG\bin\jit.exe+0x9d1cba)
 #7 0x00007ff6ecf11ef9 llvm::orc::IRTransformLayer::emit(class std::unique_ptr<class llvm::orc::MaterializationResponsibility, struct std::default_delete<class llvm::orc::MaterializationResponsibility>>, class llvm::orc::ThreadSafeModule) (D:\a\jit_debug_test\jit_debug_test\ROOTFS_DEBUG\bin\jit.exe+0x9d1ef9)
 #8 0x00007ff6ecf11ef9 llvm::orc::IRTransformLayer::emit(class std::unique_ptr<class llvm::orc::MaterializationResponsibility, struct std::default_delete<class llvm::orc::MaterializationResponsibility>>, class llvm::orc::ThreadSafeModule) (D:\a\jit_debug_test\jit_debug_test\ROOTFS_DEBUG\bin\jit.exe+0x9d1ef9)
 #9 0x00007ff6ecef3bad llvm::orc::BasicIRLayerMaterializationUnit::materialize(class std::unique_ptr<class llvm::orc::MaterializationResponsibility, struct std::default_delete<class llvm::orc::MaterializationResponsibility>>) (D:\a\jit_debug_test\jit_debug_test\ROOTFS_DEBUG\bin\jit.exe+0x9b3bad)
#10 0x00007ff6ec9d8605 llvm::orc::MaterializationTask::run(void) (D:\a\jit_debug_test\jit_debug_test\ROOTFS_DEBUG\bin\jit.exe+0x498605)
#11 0x00007ff6ec9d8841 llvm::orc::ExecutionSession::runOnCurrentThread(class std::unique_ptr<class llvm::orc::Task, struct std::default_delete<class llvm::orc::Task>>) (D:\a\jit_debug_test\jit_debug_test\ROOTFS_DEBUG\bin\jit.exe+0x498841)
#12 0x00007ff6ec9e7c5a llvm::detail::UniqueFunctionBase<void, class std::unique_ptr<class llvm::orc::Task, struct std::default_delete<class llvm::orc::Task>>>::CallImpl<void (__cdecl *)(class std::unique_ptr<class llvm::orc::Task, struct std::default_delete<class llvm::orc::Task>>)>(void *, class std::unique_ptr<class llvm::orc::Task, struct std::default_delete<class llvm::orc::Task>> &) (D:\a\jit_debug_test\jit_debug_test\ROOTFS_DEBUG\bin\jit.exe+0x4a7c5a)
#13 0x00007ff6ec9da49b llvm::orc::ExecutionSession::dispatchOutstandingMUs(void) (D:\a\jit_debug_test\jit_debug_test\ROOTFS_DEBUG\bin\jit.exe+0x49a49b)
#14 0x00007ff6ec9dddd3 llvm::orc::ExecutionSession::OL_completeLookup(class std::unique_ptr<class llvm::orc::InProgressLookupState, struct std::default_delete<class llvm::orc::InProgressLookupState>>, class std::shared_ptr<class llvm::orc::AsynchronousSymbolQuery>, class std::function<(class llvm::DenseMap<class llvm::orc::JITDylib *, class llvm::DenseSet<class llvm::orc::SymbolStringPtr, struct llvm::DenseMapInfo<class llvm::orc::SymbolStringPtr, void>>, struct llvm::DenseMapInfo<class llvm::orc::JITDylib *, void>, struct llvm::detail::DenseMapPair<class llvm::orc::JITDylib *, class llvm::DenseSet<class llvm::orc::SymbolStringPtr, struct llvm::DenseMapInfo<class llvm::orc::SymbolStringPtr, void>>>> const &)>) (D:\a\jit_debug_test\jit_debug_test\ROOTFS_DEBUG\bin\jit.exe+0x49ddd3)
#15 0x00007ff6ec9e9304 llvm::orc::InProgressFullLookupState::complete(class std::unique_ptr<class llvm::orc::InProgressLookupState, struct std::default_delete<class llvm::orc::InProgressLookupState>>) (D:\a\jit_debug_test\jit_debug_test\ROOTFS_DEBUG\bin\jit.exe+0x4a9304)
#16 0x00007ff6ec9ccc24 llvm::orc::ExecutionSession::OL_applyQueryPhase1(class std::unique_ptr<class llvm::orc::InProgressLookupState, struct std::default_delete<class llvm::orc::InProgressLookupState>>, class llvm::Error) (D:\a\jit_debug_test\jit_debug_test\ROOTFS_DEBUG\bin\jit.exe+0x48cc24)
#17 0x00007ff6ec9caeda llvm::orc::ExecutionSession::lookup(enum llvm::orc::LookupKind, class std::vector<struct std::pair<class llvm::orc::JITDylib *, enum llvm::orc::JITDylibLookupFlags>, class std::allocator<struct std::pair<class llvm::orc::JITDylib *, enum llvm::orc::JITDylibLookupFlags>>> const &, class llvm::orc::SymbolLookupSet, enum llvm::orc::SymbolState, class llvm::unique_function<(class llvm::Expected<class llvm::DenseMap<class llvm::orc::SymbolStringPtr, class llvm::orc::ExecutorSymbolDef, struct llvm::DenseMapInfo<class llvm::orc::SymbolStringPtr, void>, struct llvm::detail::DenseMapPair<class llvm::orc::SymbolStringPtr, class llvm::orc::ExecutorSymbolDef>>>)>, class std::function<(class llvm::DenseMap<class llvm::orc::JITDylib *, class llvm::DenseSet<class llvm::orc::SymbolStringPtr, struct llvm::DenseMapInfo<class llvm::orc::SymbolStringPtr, void>>, struct llvm::DenseMapInfo<class llvm::orc::JITDylib *, void>, struct llvm::detail::DenseMapPair<class llvm::orc::JITDylib *, class llvm::DenseSet<class llvm::orc::SymbolStringPtr, struct llvm::DenseMapInfo<class llvm::orc::SymbolStringPtr, void>>>> const &)>) (D:\a\jit_debug_test\jit_debug_test\ROOTFS_DEBUG\bin\jit.exe+0x48aeda)
#18 0x00007ff6ec9da85b llvm::orc::ExecutionSession::lookup(class std::vector<struct std::pair<class llvm::orc::JITDylib *, enum llvm::orc::JITDylibLookupFlags>, class std::allocator<struct std::pair<class llvm::orc::JITDylib *, enum llvm::orc::JITDylibLookupFlags>>> const &, class llvm::orc::SymbolLookupSet, enum llvm::orc::LookupKind, enum llvm::orc::SymbolState, class std::function<(class llvm::DenseMap<class llvm::orc::JITDylib *, class llvm::DenseSet<class llvm::orc::SymbolStringPtr, struct llvm::DenseMapInfo<class llvm::orc::SymbolStringPtr, void>>, struct llvm::DenseMapInfo<class llvm::orc::JITDylib *, void>, struct llvm::detail::DenseMapPair<class llvm::orc::JITDylib *, class llvm::DenseSet<class llvm::orc::SymbolStringPtr, struct llvm::DenseMapInfo<class llvm::orc::SymbolStringPtr, void>>>> const &)>) (D:\a\jit_debug_test\jit_debug_test\ROOTFS_DEBUG\bin\jit.exe+0x49a85b)
#19 0x00007ff6ec9dacc5 llvm::orc::ExecutionSession::lookup(class std::vector<struct std::pair<class llvm::orc::JITDylib *, enum llvm::orc::JITDylibLookupFlags>, class std::allocator<struct std::pair<class llvm::orc::JITDylib *, enum llvm::orc::JITDylibLookupFlags>>> const &, class llvm::orc::SymbolStringPtr, enum llvm::orc::SymbolState) (D:\a\jit_debug_test\jit_debug_test\ROOTFS_DEBUG\bin\jit.exe+0x49acc5)
#20 0x00007ff6ec9b7a60 llvm::orc::LLJIT::lookupLinkerMangled(class llvm::orc::JITDylib &, class llvm::orc::SymbolStringPtr) (D:\a\jit_debug_test\jit_debug_test\ROOTFS_DEBUG\bin\jit.exe+0x477a60)
#21 0x00007ff6ec99230e llvm::orc::LLJIT::lookupLinkerMangled(class llvm::orc::JITDylib &, class llvm::StringRef) D:\a\jit_debug_test\jit_debug_test\llvm-18\include\llvm\ExecutionEngine\Orc\LLJIT.h:164:0
#22 0x00007ff6ec99223a llvm::orc::LLJIT::lookup(class llvm::orc::JITDylib &, class llvm::StringRef) D:\a\jit_debug_test\jit_debug_test\llvm-18\include\llvm\ExecutionEngine\Orc\LLJIT.h:176:0
#23 0x00007ff6ec98cebf llvm::orc::LLJIT::lookup(class llvm::StringRef) D:\a\jit_debug_test\jit_debug_test\llvm-18\include\llvm\ExecutionEngine\Orc\LLJIT.h:181:0
#24 0x00007ff6ec98a04c JIT::lookup(class llvm::StringRef) D:\a\jit_debug_test\jit_debug_test\jit.cpp:196:0
#25 0x00007ff6ec99a4bc JIT::lookup_as_pointer<(void)>(class llvm::StringRef) D:\a\jit_debug_test\jit_debug_test\jit.h:67:0
#26 0x00007ff6ec999ca5 main D:\a\jit_debug_test\jit_debug_test\main.cpp:158:0
#27 0x00007ff6ee49ed64 invoke_main D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:78:0
#28 0x00007ff6ee49ed64 __scrt_common_main_seh D:\a\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288:0
#29 0x00007ffd509b4cb0 (C:\Windows\System32\KERNEL32.DLL+0x14cb0)
#30 0x00007ffd514beceb (C:\Windows\SYSTEM32\ntdll.dll+0x7eceb)
D:\a\_temp\6f20a241-fd1e-4d83-ad6e-f801711ad65d.sh: line 4:   447 Segmentation fault      ./ROOTFS_DEBUG/bin/jit

lldb ROOTFS_DEBUG/bin/jit.exe -s j.lldb --source-on-crash j.lldberr
(lldb) target create "ROOTFS_DEBUG/bin/jit.exe"
Current executable set to 'D:\a\jit_debug_test\jit_debug_test\ROOTFS_DEBUG\bin\jit.exe' (x86_64).
(lldb) command source -s 0 'j.lldb'
Executing commands in 'D:\a\jit_debug_test\jit_debug_test\j.lldb'.
(lldb) b j
Breakpoint 1: no locations (pending).
WARNING:  Unable to resolve breakpoint to any actual locations.
(lldb) r
Process 4632 launched: 'D:\a\jit_debug_test\jit_debug_test\ROOTFS_DEBUG\bin\jit.exe' (x86_64)
Process 4632 stopped
* thread #1, stop reason = Exception 0xc0000005 encountered at address 0x7ff6edeb3f3d: Access violation reading location 0xffffffffffffffff
    frame #0: 0x00007ff6edeb3f3d jit.exe`main + 22127453
jit.exe`main:
->  0x7ff6edeb3f3d <+22127453>: callq  *0x8(%r10)
    0x7ff6edeb3f41 <+22127457>: jmp    0x7ff6edeb3fe4            ; <+22127620>
    0x7ff6edeb3f46 <+22127462>: movq   0x8(%rdi), %rcx
    0x7ff6edeb3f4a <+22127466>: leaq   0x38(%rsp), %rdx
(lldb) bt
error: jit.exe :: Class 'std::pair<llvm::StringRef,llvm::StringRef>' has a member 'first' of type 'llvm::StringRef' which does not have a complete definition.
(lldb) error: jit.exe :: Class 'llvm::ErrorSuccess' has a base class 'llvm::Error' which does not have a complete definition.
* thread #1, stop reason = Exception 0xc0000005 encountered at address 0x7ff6edeb3f3d: Access violation reading location 0xffffffffffffffff
  * frame #0: 0x00007ff6edeb3f3d jit.exe`main + 22127453
    frame #1: 0x00007ff6eddf54fc jit.exe`main + 21346588
    frame #2: 0x00007ff6ed8b4c3b jit.exe`main + 15839323
    frame #3: 0x00007ff6ecefbb14 jit.exe`main + 5644084
    frame #4: 0x00007ff6ec9a7b1b jit.exe`main + 57147
    frame #5: 0x00007ff6ecf118a1 jit.exe`main + 5733569
    frame #6: 0x00007ff6ecf11cba jit.exe`main + 5734618
    frame #7: 0x00007ff6ecf11ef9 jit.exe`main + 5735193
    frame #8: 0x00007ff6ecf11ef9 jit.exe`main + 5735193
    frame #9: 0x00007ff6ecef3bad jit.exe`main + 5611469
    frame #10: 0x00007ff6ec9d8605 jit.exe`main + 256549
    frame #11: 0x00007ff6ec9d8841 jit.exe`main + 257121
    frame #12: 0x00007ff6ec9e7c5a jit.exe`main + 319610
    frame #13: 0x00007ff6ec9da49b jit.exe`main + 264379
    frame #14: 0x00007ff6ec9dddd3 jit.exe`main + 279027
    frame #15: 0x00007ff6ec9e9304 jit.exe`main + 325412
    frame #16: 0x00007ff6ec9ccc24 jit.exe`main + 208964
    frame #17: 0x00007ff6ec9caeda jit.exe`main + 201466
    frame #18: 0x00007ff6ec9da85b jit.exe`main + 265339
    frame #19: 0x00007ff6ec9dacc5 jit.exe`main + 266469
    frame #20: 0x00007ff6ec9b7a60 jit.exe`main + 122496
    frame #21: 0x00007ff6ec99230e jit.exe`class llvm::Expected<class llvm::orc::ExecutorAddr> llvm::orc::LLJIT::lookupLinkerMangled(class llvm::orc::JITDylib &, class llvm::StringRef) at LLJIT.h:164
    frame #22: 0x00007ff6ec99223a jit.exe`class llvm::Expected<class llvm::orc::ExecutorAddr> llvm::orc::LLJIT::lookup(class llvm::orc::JITDylib &, class llvm::StringRef) at LLJIT.h:176
    frame #23: 0x00007ff6ec98cebf jit.exe`public: class llvm::Expected<class llvm::orc::ExecutorAddr> __cdecl llvm::orc::LLJIT::lookup(this=<unavailable>, UnmangledName=0x00000007c21ef7a0) at LLJIT.h:181
    frame #24: 0x00007ff6ec98a04c jit.exe`public: class llvm::Expected<class llvm::orc::ExecutorAddr> __cdecl JIT::lookup(this=0x00000007c21ef928, symbol=0x00000007c21ef840) at jit.cpp:196
    frame #25: 0x00007ff6ec99a4bc jit.exe` * JIT::lookup_as_pointer<int ()>(this=0x00000007c21ef928, symbol=0x00000007c21ef900) at jit.h:67
    frame #26: 0x00007ff6ec999ca5 jit.exe`main(argc=1, argv=0x000001f42dc545b0) at main.cpp:158
    frame #27: 0x00007ff6ee49ed64 jit.exe`static int __scrt_common_main_seh() at exe_common.inl:288
    frame #28: 0x00007ffd509b4cb0 kernel32.dll`BaseThreadInitThunk + 16
    frame #29: 0x00007ffd514beceb ntdll.dll`RtlUserThreadStart + 43

[ZLangJIT]

i tried using lli but it doesn't seem to read any debug info

          clang -fexceptions -c test_lli.c -o test_lli.bc -emit-llvm -O0 -g3 -Xclang -triple -Xclang "x86_64-pc-windows-elf"

          lldb -s j.lldb --source-on-crash j.lldberr lli -- --jit-kind=mcjit test_lli.bc

          lldb -s j.lldb --source-on-crash j.lldberr lli -- --jit-kind=orc --jit-linker=rtdyld test_lli.bc

          lldb -s j.lldb --source-on-crash j.lldberr lli -- --jit-kind=orc --jit-linker=jitlink test_lli.bc

lldb shows assembly for all 3

(lldb) r
Process 1132 launched: 'D:\a\jit_debug_test\jit_debug_test\llvm-18\bin\lli.exe' (x86_64)
Process 1132 stopped
* thread #1, stop reason = Exception 0xc0000005 encountered at address 0x1b9e32f0166: Access violation reading location 0x00000000
    frame #0: 0x000001b9e32f0166
->  0x1b9e32f0166: movl   (%rax), %edx
    0x1b9e32f0168: movabsq $0x1b9e3300000, %rcx      ; imm = 0x1B9E3300000 
    0x1b9e32f0172: movabsq $0x7ffba14dbdc0, %rax     ; imm = 0x7FFBA14DBDC0 
    0x1b9e32f017c: callq  *%rax
(lldb) bt
* thread #1, stop reason = Exception 0xc0000005 encountered at address 0x1b9e32f0166: Access violation reading location 0x00000000
  * frame #0: 0x000001b9e32f0166
(lldb) 

for this https://github.com/alexjordan/LLVM-TMS320C64X/blob/master/docs/DebuggingJITedCode.html

[ZLangJIT]

Interesting, the llvm-jitlink tool should export this symbol. Can you dump them (e.g. with llvm-nm) and see if llvm_orc_registerJITLoaderGDBWrapper is there? Otherwise, does it work with lli?

See the above comment

[llvmbot]

@llvm/issue-subscribers-debuginfo

Author: None (ZLangJIT)

``` #include <stdio.h> int j() { printf("HELLO JIT\n"); return 0; } ``` clang -c jit_code.c -o jit_code.o -O0 -g3 -Xclang -triple -Xclang "x86_64-pc-windows-elf" llvm-jitlink -- jit_code.o --entry=j --debugger-support llvm-jitlink.exe: Symbols not found: [ llvm_orc_registerJITLoaderGDBWrapper ] Using `-no-process-syms=true` doesnt help llvm 18.1.8

[ZLangJIT]

Im not sure what else to do

[weliveindetail]

@ZLangJIT with this patch the function is resolved successfully

diff --git a/llvm/include/llvm/ExecutionEngine/Orc/TargetProcess/JITLoaderGDB.h b/llvm/include/llvm/ExecutionEngine/Orc/TargetProcess/JITLoaderGDB.h
index 9f91a64e95ce..ead33ae2c332 100644
--- a/llvm/include/llvm/ExecutionEngine/Orc/TargetProcess/JITLoaderGDB.h
+++ b/llvm/include/llvm/ExecutionEngine/Orc/TargetProcess/JITLoaderGDB.h
@@ -42,10 +42,10 @@ struct jit_descriptor {
 };
 }

-extern "C" llvm::orc::shared::CWrapperFunctionResult
+extern "C" __declspec(dllexport) llvm::orc::shared::CWrapperFunctionResult
 llvm_orc_registerJITLoaderGDBWrapper(const char *Data, uint64_t Size);

-extern "C" llvm::orc::shared::CWrapperFunctionResult
+extern "C" __declspec(dllexport) llvm::orc::shared::CWrapperFunctionResult
 llvm_orc_registerJITLoaderGDBAllocAction(const char *Data, size_t Size);

 #endif // LLVM_EXECUTIONENGINE_ORC_TARGETPROCESS_JITLOADERGDB_H

However, I think direct use of __declspec(dllexport) is discouraged in LLVM. Instead, exports should be handled in the build system.

[lhames]

@weliveindetail Tangentially related: https://discourse.llvm.org/t/rfc-move-orc-executor-support-into-top-level-project/81049 . ;)

I think we'll need to keep some version of this in LLVM for MCJIT compat, but we can special-case it (e.g. add an absolute symbol for these functions by default). The ORC RT project should always export these functions and we can bake that into the build system unconditionally.

[weliveindetail]

Tangentially related

Thanks for the pointer! I will have a look later and follow-up over there.

The ORC RT project should always export these functions and we can bake that into the build system unconditionally.

Yes, agree and all relevant executables do export them today. Except on Windows! Since MSVC limits the number of exported symbols (https://learn.microsoft.com/en-us/cpp/error-messages/tool-errors/linker-tools-error-lnk1189), the CMake implementation in LLVM simply skips auto-export: https://github.com/llvm/llvm-project/blob/release/19.x/llvm/cmake/modules/AddLLVM.cmake#L1357-L1359

I guess it has always been like that and we aren't aware, because no-one really takes care of Windows support in the JIT upstream? clang-repl has manual exports for MSVC (https://github.com/llvm/llvm-project/blob/release/19.x/clang/tools/clang-repl/CMakeLists.txt), but I guess in MinGW it would fail as described in this ticket.

[weliveindetail]

@ZLangJIT The above patch fixes the symbol resolution error, but debugging still doesn't work. I tested with LLDB and GDB and I don't get frames or symbols for JITed code:

Thread 1 hit Breakpoint 2.2, 0x00007ffd21bdc8aa in printf () from C:\WINDOWS\System32\msvcrt.dll
(gdb) bt
#0  0x00007ffd21bdc8aa in printf () from C:\WINDOWS\System32\msvcrt.dll
#1  0x00000287fdb31158 in ?? ()
#2  0x00000287fdb30000 in ?? ()
#3  0x00000287fda57b10 in ?? ()
#4  0x00000098c0dff3c0 in ?? ()
#5  0x00000098c0dff3c0 in ?? ()
#6  0x00000000c0dff3c0 in ?? ()
#7  0x00007ff6a86a58f4 in llvm::orc::runAsMain (Main=0x287fdb31140, Args=...,
    ProgramName=std::optional [no contained value])
    at S:/LLVM/llvm-project/llvm/lib/ExecutionEngine/Orc/TargetProcess/TargetExecutionUtils.cpp:39
#8  0x00007ff6a869f815 in llvm::orc::SelfExecutorProcessControl::runAsMain (this=0x287fda4b260,
    MainFnAddr=..., Args=...)
    at S:/LLVM/llvm-project/llvm/lib/ExecutionEngine/Orc/ExecutorProcessControl.cpp:120
#9  0x00007ff6a83faabb in runWithoutRuntime (S=..., EntryPointAddr=...)
    at S:/LLVM/llvm-project/llvm/tools/llvm-jitlink/llvm-jitlink.cpp:2333
#10 0x00007ff6a83f9139 in main (argc=2, argv=0x287fc288fc0)
    at S:/LLVM/llvm-project/llvm/tools/llvm-jitlink/llvm-jitlink.cpp:2413

This isn't surprising. You can PM me in case it's an important feature for you.

[mstorsjo]

However, I think direct use of __declspec(dllexport) is discouraged in LLVM. Instead, exports should be handled in the build system.

Yes - on MinGW builds with LLVM_LINK_LLVM_DYLIB enabled, using explicit dllexports interfere a little bit.

If there are no explicit dllexports - which is normally the case - then all symbols are exported. But if there's a single explicit dllexport, then only those explicit dllexports are exported, and nothing else. We had to work around this for dllexports in clang-repl in 592e935e115ffb451eb9b782376711dab6558fe0, by passing an explicit target_link_options(clang-cpp PRIVATE LINKER:--export-all-symbols) for those cases.

[weliveindetail]

Thanks for the note @mstorsjo! IIUC we could use that approach to implement to exporting in ORC directly and propagate the required linker flag to all targets that use them. (And that's why CMakeLists.txt in clang-repl has no special-case for MinGW!) Yes, we could consider that.

[mstorsjo]

Thanks for the note @mstorsjo! IIUC we could use that approach to implement to exporting in ORC directly and propagate the required linker flag to all targets that use them. (And that's why CMakeLists.txt in clang-repl has no special-case for MinGW!) Yes, we could consider that.

Possibly, yes. Libraries containing dllexported symbols can be problematic and surprising for the users who link it in though, where you may need to add the --export-all-symbols to make things work as it used to before (in this case, in particular libLLVM-*.dll for LLVM_LINK_LLVM_DYLIB), but if these libraries aren't usually linked in outside of the main scope where the user actively uses them, it might be ok.

[weliveindetail]

clang-repl has no special-case for MinGW

In fact, we hit the same symbol export problem in MSVC builds. My fix works there as well, so I can generalize it to WIN32.

but if these libraries aren't usually linked in outside of the main scope where the user actively uses them, it might be ok

Hm, that sounds like it might bring up issues that I don't want to tackle :) And we really don't need all symbols, so I'd stick with my existing fix. It's simple and it's local.

[ZLangJIT]

@ZLangJIT The above patch fixes the symbol resolution error, but debugging still doesn't work. I tested with LLDB and GDB and I don't get frames or symbols for JITed code:

Thread 1 hit Breakpoint 2.2, 0x00007ffd21bdc8aa in printf () from C:\WINDOWS\System32\msvcrt.dll
(gdb) bt
#0  0x00007ffd21bdc8aa in printf () from C:\WINDOWS\System32\msvcrt.dll
#1  0x00000287fdb31158 in ?? ()
#2  0x00000287fdb30000 in ?? ()
#3  0x00000287fda57b10 in ?? ()
#4  0x00000098c0dff3c0 in ?? ()
#5  0x00000098c0dff3c0 in ?? ()
#6  0x00000000c0dff3c0 in ?? ()
#7  0x00007ff6a86a58f4 in llvm::orc::runAsMain (Main=0x287fdb31140, Args=...,
    ProgramName=std::optional [no contained value])
    at S:/LLVM/llvm-project/llvm/lib/ExecutionEngine/Orc/TargetProcess/TargetExecutionUtils.cpp:39
#8  0x00007ff6a869f815 in llvm::orc::SelfExecutorProcessControl::runAsMain (this=0x287fda4b260,
    MainFnAddr=..., Args=...)
    at S:/LLVM/llvm-project/llvm/lib/ExecutionEngine/Orc/ExecutorProcessControl.cpp:120
#9  0x00007ff6a83faabb in runWithoutRuntime (S=..., EntryPointAddr=...)
    at S:/LLVM/llvm-project/llvm/tools/llvm-jitlink/llvm-jitlink.cpp:2333
#10 0x00007ff6a83f9139 in main (argc=2, argv=0x287fc288fc0)
    at S:/LLVM/llvm-project/llvm/tools/llvm-jitlink/llvm-jitlink.cpp:2413

This isn't surprising. You can PM me in case it's an important feature for you.

just a reminder that even when the gdb debug interface symbols are found, there is still the problem of actually getting the debug information to be loaded

[ZLangJIT]

Any progress ?