search

NationalSecurityAgency / ghidra

Ghidra is a software reverse engineering (SRE) framework
https://www.nsa.gov/ghidra
Apache License 2.0
51.75k stars 5.89k forks source link

MacOS Ghidra start debugger issue #5517

Closed DAMisener closed 4 months ago

DAMisener commented 1 year ago

MacOS Ghidra start debugger failure

Failure sequence:

Is there a missing dependencies or configuration issue regarding the lldb java interconnect? Any suggestion on how to address this would be greatly appreciated.

Environment:

d-millar commented 1 year ago

I would start with the InstructionsForBuildingLLDBInterface in Ghidra/Debug/Debugger-swig-lldb - let us know if you get stuck.

DAMisener commented 1 year ago

Sorry for the long response time. My java/gradle foo is almost non-existent so I'm plodding slowly.

I have hopefully installed the necessary prerequisites and tried to follow the instructions in:

/usr/local/Caskroom/ghidra/10.3.1-20230614/ghidra_10.3.1_PUBLIC/Ghidra/Debug/Debugger-swig-lldb/InstructionsForBuildingLLDBInterface.txt

When I execute the folllowing instructions:

cd "/usr/local/Caskroom/ghidra/10.3.1-20230614/ghidra_10.3.1_PUBLIC/Ghidra/Debug/Debugger-swig-lldb"
gradle --version
lldb --version
export GHIDRA_INSTALL_DIR="/Volumes/Macintosh HD 1/usr/local/Caskroom/ghidra/10.3.1-20230614/ghidra_10.3.1_PUBLIC"

./macos_debugger_lldb_build_from_brew.sh

The following output is generated:

 cd "/usr/local/Caskroom/ghidra/10.3.1-20230614/ghidra_10.3.1_PUBLIC/Ghidra/Debug/Debugger-swig-lldb"
gradle --version
lldb --version
export GHIDRA_INSTALL_DIR="/Volumes/Macintosh HD 1/usr/local/Caskroom/ghidra/10.3.1-20230614/ghidra_10.3.1_PUBLIC"
./macos_debugger_lldb_build_from_brew.sh

------------------------------------------------------------
Gradle 8.2
------------------------------------------------------------

Build time:   2023-06-30 18:02:30 UTC
Revision:     5f4a070a62a31a17438ac998c2b849f4f6892877

Kotlin:       1.8.20
Groovy:       3.0.17
Ant:          Apache Ant(TM) version 1.10.13 compiled on January 4 2023
JVM:          20.0.1 (Homebrew 20.0.1)
OS:           Mac OS X 13.4.1 x86_64

lldb-1403.0.17.67
Apple Swift version 5.8.1 (swiftlang-5.8.0.124.5 clang-1403.0.22.11.100)
+ '[' -z '/Volumes/Macintosh HD 1/usr/local/Caskroom/ghidra/10.3.1-20230614/ghidra_10.3.1_PUBLIC' ']'
+ '[' '!' -z '/Volumes/Macintosh HD 1/usr/local/Caskroom/ghidra/10.3.1-20230614/ghidra_10.3.1_PUBLIC' ']'
+ pushd '/Volumes/Macintosh HD 1/usr/local/Caskroom/ghidra/10.3.1-20230614/ghidra_10.3.1_PUBLIC/Ghidra/Debug/Debugger-swig-lldb'
/Volumes/Macintosh HD 1/usr/local/Caskroom/ghidra/10.3.1-20230614/ghidra_10.3.1_PUBLIC/Ghidra/Debug/Debugger-swig-lldb /usr/local/Caskroom/ghidra/10.3.1-20230614/ghidra_10.3.1_PUBLIC/Ghidra/Debug/Debugger-swig-lldb
+ LLVM_VERSION=16
+ brew install llvm@16
==> Downloading https://formulae.brew.sh/api/formula.jws.json
######################################################################################################################################################################################################################### 100.0%
==> Downloading https://formulae.brew.sh/api/cask.jws.json
######################################################################################################################################################################################################################### 100.0%
Warning: llvm 16.0.6 is already installed and up-to-date.
To reinstall 16.0.6, run:
  brew reinstall llvm
++ mktemp -d
+ LLVM_TEMP_DIR=/var/folders/8l/vhg5vggj04g77ny91mlgzn680000gn/T/tmp.n16stYnE
+ brew unpack --patch --destdir /var/folders/8l/vhg5vggj04g77ny91mlgzn680000gn/T/tmp.n16stYnE llvm@16
==> Unpacking llvm to: /var/folders/8l/vhg5vggj04g77ny91mlgzn680000gn/T/tmp.n16stYnE/llvm-16.0.6
==> Downloading https://github.com/llvm/llvm-project/releases/download/llvmorg-16.0.6/llvm-project-16.0.6.src.tar.xz
Already downloaded: /Users/DM/Library/Caches/Homebrew/downloads/8cbb1836057b964f8f1bea2eee120bd27a78aa698ba2d5263996f3071234b47e--llvm-project-16.0.6.src.tar.xz
++ echo '/var/folders/8l/vhg5vggj04g77ny91mlgzn680000gn/T/tmp.n16stYnE/llvm@16-*'
+ export 'LLVM_HOME=/var/folders/8l/vhg5vggj04g77ny91mlgzn680000gn/T/tmp.n16stYnE/llvm@16-*'
+ LLVM_HOME='/var/folders/8l/vhg5vggj04g77ny91mlgzn680000gn/T/tmp.n16stYnE/llvm@16-*'
++ brew --prefix llvm@16
+ BREW_LLVM=/usr/local/opt/llvm
+ export 'LDFLAGS=-L/usr/local/opt/llvm/lib/c++ -Wl,-rpath,/usr/local/opt/llvm/lib/c++,-L/usr/local/opt/llvm/lib'
+ LDFLAGS='-L/usr/local/opt/llvm/lib/c++ -Wl,-rpath,/usr/local/opt/llvm/lib/c++,-L/usr/local/opt/llvm/lib'
+ export 'PATH=/usr/local/opt/llvm/bin:/Users/DM/.ops/bin:/usr/local/sbin:/Users/DM/.cargo/bin:/Users/DM/Library/Application Support/GoodSync:/Applications/MacVim.app/Contents/bin:/Applications/Araxis Merge.app/Contents/Utilities:/Users/DM/.rbenv/shims:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/Little Snitch.app/Contents/Components:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Apple/usr/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Users/DM/.fig/bin:/usr/local/opt/fzf/bin'
+ PATH='/usr/local/opt/llvm/bin:/Users/DM/.ops/bin:/usr/local/sbin:/Users/DM/.cargo/bin:/Users/DM/Library/Application Support/GoodSync:/Applications/MacVim.app/Contents/bin:/Applications/Araxis Merge.app/Contents/Utilities:/Users/DM/.rbenv/shims:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/Little Snitch.app/Contents/Components:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Apple/usr/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Users/DM/.fig/bin:/usr/local/opt/fzf/bin'
+ export CPPFLAGS=-I/usr/local/opt/llvm/include
+ CPPFLAGS=-I/usr/local/opt/llvm/include
++ echo /usr/local/opt/llvm
+ export LLVM_BUILD=/usr/local/opt/llvm
+ LLVM_BUILD=/usr/local/opt/llvm
+ gradle buildNatives

> Configure project :Debugger-swig-lldb
OS: Mac OS X
Arch: x86_64 (gradle: x86-64)
Using platform: mac_x86_64

> Configure project :Decompiler
OS: Mac OS X
Arch: x86_64 (gradle: x86-64)
Using platform: mac_x86_64

> Configure project :PDB
OS: Mac OS X
Arch: x86_64 (gradle: x86-64)
Using platform: mac_x86_64

> Task :Debugger-swig-lldb:compileMainMac_x86_64SharedLibraryMainCpp FAILED
/usr/local/Caskroom/ghidra/10.3.1-20230614/ghidra_10.3.1_PUBLIC/Ghidra/Debug/Debugger-swig-lldb/src/main/cpp/LLDBWrapJava.cpp:266:10: fatal error: 'lldb/lldb-public.h' file not found
#include "lldb/lldb-public.h"
         ^~~~~~~~~~~~~~~~~~~~
1 error generated.

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':Debugger-swig-lldb:compileMainMac_x86_64SharedLibraryMainCpp'.
> A build operation failed.
      C++ compiler failed while compiling LLDBWrapJava.cpp.
  See the complete log at: file:///usr/local/Caskroom/ghidra/10.3.1-20230614/ghidra_10.3.1_PUBLIC/Ghidra/Debug/Debugger-swig-lldb/build/tmp/compileMainMac_x86_64SharedLibraryMainCpp/output.txt
   > C++ compiler failed while compiling LLDBWrapJava.cpp.

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
> Get more help at https://help.gradle.org.

BUILD FAILED in 2s
2 actionable tasks: 2 executed
➜  Debugger-swig-lldb

It looked like some the required include file isn't where it should be...but I do have:

/Volumes/Macintosh HD 1/usr/local/Cellar/llvm/16.0.6/include/lldb/lldb-public.h

which contains:

//===-- lldb-public.h -------------------------------------------*- C++ -*-===//
//
// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
// See https://llvm.org/LICENSE.txt for license information.
// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
//
//===----------------------------------------------------------------------===//

#ifndef LLDB_LLDB_PUBLIC_H
#define LLDB_LLDB_PUBLIC_H

#include "lldb/lldb-defines.h"
#include "lldb/lldb-enumerations.h"
#include "lldb/lldb-forward.h"
#include "lldb/lldb-types.h"

#endif // LLDB_LLDB_PUBLIC_H

Any ideas on how to correct this issue... i.e. change a path somewhere or copy the above file to someplace else??

d-millar commented 1 year ago

@DAMisener Looks like the script isn’t quite accounting for the fact that you already had llvm installed. Not quite sure why there isn’t a copy of lldb-public.h in /usr/local/opt/llvm/include/lldb as a result of the scripted download. Am traveling right now, but can look into that when I get back.

In the meantime, I would suggest two possibilities - (1) copy all of the .h files beginning with lldb- from /usr/local/Cellar/llvm/16.0.6/include/lldb into /usr/local/opt/llvm/include/lldb and re-run the script (might work), or (2) skip the script and follow the remaining instructions in InstructionsForBuilding with LLVM_HOME and LLVM_BUILD set appropriately (more complicated but should work if you point things to the right level within the most complete downloads you have).

Realize this is a pain (we’re working on a simpler solution) but, without LLVM support for an out-of-the-box JNI wrapper via SWIG, it’s the best solution we have at the moment.

DAMisener commented 1 year ago

Thanks...

I was going to try (1) above but it looks like the files are already there!

pwd

/usr/local/opt/llvm/include/lldb

ls - l

drwxr-xr-x  26 DM  staff   832B Jul  9 17:54 .
drwxr-xr-x  21 DM  staff   672B Jul  9 17:54 ..
-rw-r--r--@  1 DM  staff   6.0K Jul  9 17:54 .DS_Store
drwxr-xr-x  73 DM  staff   2.3K Jun 10 19:58 API
drwxr-xr-x  27 DM  staff   864B Jun 10 19:58 Breakpoint
drwxr-xr-x  65 DM  staff   2.0K Jun 10 19:58 Core
drwxr-xr-x  20 DM  staff   640B Jun 10 19:58 DataFormatters
drwxr-xr-x  20 DM  staff   640B Jun 10 19:58 Expression
drwxr-xr-x  51 DM  staff   1.6K Jun 10 19:58 Host
drwxr-xr-x   5 DM  staff   160B Jun 10 19:58 Initialization
drwxr-xr-x  55 DM  staff   1.7K Jun 10 19:58 Interpreter
drwxr-xr-x  38 DM  staff   1.2K Jun 10 19:58 Symbol
drwxr-xr-x  80 DM  staff   2.5K Jun 10 19:58 Target
drwxr-xr-x  63 DM  staff   2.0K Jun 10 19:58 Utility
drwxr-xr-x   3 DM  staff    96B Jun 10 19:58 Version
-rw-r--r--   1 DM  staff   4.9K Jun 10 19:58 lldb-defines.h
-rw-r--r--   1 DM  staff    47K Jun 10 19:58 lldb-enumerations.h
-rw-r--r--   1 DM  staff    16K Jun 10 19:58 lldb-forward.h
-rw-r--r--   1 DM  staff   8.5K Jun 10 19:58 lldb-private-enumerations.h
-rw-r--r--   1 DM  staff   656B Jun 10 19:58 lldb-private-forward.h
-rw-r--r--   1 DM  staff   6.8K Jun 10 19:58 lldb-private-interfaces.h
-rw-r--r--   1 DM  staff   4.4K Jun 10 19:58 lldb-private-types.h
-rw-r--r--   1 DM  staff   669B Jun 10 19:58 lldb-private.h
-rw-r--r--   1 DM  staff   582B Jun 10 19:58 lldb-public.h
-rw-r--r--   1 DM  staff   3.3K Jun 10 19:58 lldb-types.h
-rw-r--r--   1 DM  staff    50K Jun 10 19:58 lldb-versioning.h

Am I missing some ENVIRONMENT variable or directory in a PATH perhap?

echo \$PATH

/Users/DM/.ops/bin:/usr/local/sbin:/Users/DM/.cargo/bin:/Users/DM/Library/Application Support/GoodSync:/Applications/MacVim.app/Contents/bin:/Applications/Araxis Merge.app/Contents/Utilities:/Users/DM/.rbenv/shims:/opt/local/bin:/opt/local/sbin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/Little Snitch.app/Contents/Components:/usr/local/share/dotnet:~/.dotnet/tools:/Library/Apple/usr/bin:/Library/Frameworks/Mono.framework/Versions/Current/Commands:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/Users/DM/.fig/bin:/usr/local/opt/fzf/bin

BTW:

Apple clang version 14.0.3 (clang-1403.0.22.14.1)
Target: x86_64-apple-darwin22.5.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
d-millar commented 1 year ago

Hmmmm, the “gradle buildNatives” task should have picked up the relevant .h files from CPPFLAGS, which appears to be set correctly. Possible the buildNatives r ram got modified, but….any luck with option two?

d-millar commented 1 year ago

OK, I see it - the problem is the ordering of the include dirs differs for different installations, sadly. If you look at the buildNatives.gradle file in Ghidra/Debug/Debugger-swig-lldb, you will find several -I tags with a path to the include dir, e.g. b.cppCompiler.args "-I$llvm_dir/lldb/include". llvm_dir equals the environment variable LLVM_HOME, so you’ll need to modify those lines so that the path used contains “lldb/lldb-public.h”. Then run “gradle buildNatives” (or re-run the script - maybe). Yell if you get stuck.

DAMisener commented 1 year ago

Thanks for your last post... most enlighting.

I didn't understand what you meant by ordering of the include dirs differs for different installations but I poked around a little more and I think I found 2 contributing issues.

The version of LLVM_VERSION in my environment (see above) is llvm-16.0.6 not llvm@16.

It looks like the supplied script doesn't handle point versions.

Also the trailing sed needs quotes to handle possible blanks in the LAUNCH_PROPERTIES (i.e. /Volumes/Macintosh HD/...)

Executing a modified copy (diff below) now seems to run to completion without error :-)

diff ~/work/trunk/ghidra.sh macos_debugger_lldb_build_from_brew.sh
1,8c1,21
32,34c45
< # export LLVM_HOME="$(echo ${LLVM_TEMP_DIR}/llvm@${LLVM_VERSION}-*)"
< export LLVM_HOME="$(echo ${LLVM_TEMP_DIR}/llvm-16.0.6)"
---
> export LLVM_HOME="$(echo ${LLVM_TEMP_DIR}/llvm@${LLVM_VERSION}-*)"
61c72
< sed -i '' /llvm/d "${LAUNCH_PROPERTIES}"
---
> sed -i '' /llvm/d ${LAUNCH_PROPERTIES}

I can now invoke the resulting debugger via right clicking on test Open with debugger.

Ghidra continues the disassembly (chewing heavily on the CPU > 100%!!) and updates the bottom line with status lines like Decompiling 100015ca0, code error at 10001c000: Disassembly not permitted within uninitialized memory block0

The resulting application.log file contains:

2023-07-11 23:24:25 INFO  (LoggingInitialization) Using log config file: jar:file:/usr/local/Caskroom/ghidra/10.3.1-20230614/ghidra_10.3.1_PUBLIC/Ghidra/Framework/Generic/lib/Generic.jar!/generic.log4j.xml  
2023-07-11 23:24:25 INFO  (LoggingInitialization) Using log file: /Users/DM/.ghidra/.ghidra_10.3.1_PUBLIC/application.log  
2023-07-11 23:24:25 INFO  (Preferences) Loading user preferences: /Users/DM/.ghidra/.ghidra_10.3.1_PUBLIC/preferences  
2023-07-11 23:24:26 INFO  (ClassSearcher) Searching for classes...  
2023-07-11 23:24:27 INFO  (ClassSearcher) Class search complete (1030 ms)  
2023-07-11 23:24:27 INFO  (SSLContextInitializer) Initializing SSL Context  
2023-07-11 23:24:27 INFO  (SecureRandomFactory) Initializing Random Number Generator...  
2023-07-11 23:24:27 INFO  (SecureRandomFactory) Random Number Generator initialization complete: NativePRNGNonBlocking  
2023-07-11 23:24:27 INFO  (ApplicationTrustManagerFactory) Trust manager disabled, cacerts have not been set  
2023-07-11 23:24:27 INFO  (GhidraRun) User DM started Ghidra.  
2023-07-11 23:24:28 DEBUG (RecoverySnapshotMgrPlugin) Recovery snapshot timer set to 5 minute(s)  
2023-07-11 23:24:29 INFO  (DefaultProject) Opening project: /Users/DM/work/trunk/ghidra/test  
2023-07-11 23:24:31 INFO  (PackedDatabaseCache) Packed database cache: /var/folders/8l/vhg5vggj04g77ny91mlgzn680000gn/T/DM-Ghidra/packed-db-cache  
2023-07-11 23:24:31 DEBUG (PackedDatabaseCache) Using cached packed database: /usr/local/Caskroom/ghidra/10.3.1-20230614/ghidra_10.3.1_PUBLIC/Ghidra/Features/Base/data/typeinfo/mac_10.9/mac_osx.gdt  
2023-07-11 23:24:33 DEBUG (ToolTaskManager) Background processing started...  
2023-07-11 23:24:33 DEBUG (ToolTaskManager) Exec Task Map modules to open programs  
2023-07-11 23:24:33 DEBUG (ToolTaskManager) Map modules to open programs task finish (0.015 secs)  
2023-07-11 23:24:33 DEBUG (ToolTaskManager) Map modules to open programs task complete (0.21 secs)  
2023-07-11 23:24:33 DEBUG (ToolTaskManager) Background processing complete (0.21 secs)  
2023-07-11 23:24:48 DEBUG (ToolTaskManager) Background processing started...  
2023-07-11 23:24:48 DEBUG (ToolTaskManager) Exec Task Auto Analysis  
2023-07-11 23:24:49 ERROR (DWARFFunctionImporter) Error creating data object at 1000392f0 ghidra.program.model.util.CodeUnitInsertionException: Could not create Data at address 1000392f0
    at ghidra.program.model.data.DataUtilities.getData(DataUtilities.java:249)
    at ghidra.program.model.data.DataUtilities.createData(DataUtilities.java:164)
    at ghidra.program.model.data.DataUtilities.createData(DataUtilities.java:144)
    at ghidra.app.util.bin.format.dwarf4.next.DWARFFunctionImporter.outputGlobal(DWARFFunctionImporter.java:425)
    at ghidra.app.util.bin.format.dwarf4.next.DWARFFunctionImporter.importFunctions(DWARFFunctionImporter.java:127)
    at ghidra.app.util.bin.format.dwarf4.next.DWARFParser.parse(DWARFParser.java:202)
    at ghidra.app.plugin.core.analysis.DWARFAnalyzer.added(DWARFAnalyzer.java:199)
    at ghidra.app.plugin.core.analysis.AnalysisScheduler.runAnalyzer(AnalysisScheduler.java:186)
    at ghidra.app.plugin.core.analysis.AnalysisTask.applyTo(AnalysisTask.java:39)
    at ghidra.app.plugin.core.analysis.AutoAnalysisManager$AnalysisTaskWrapper.run(AutoAnalysisManager.java:690)
    at ghidra.app.plugin.core.analysis.AutoAnalysisManager.startAnalysis(AutoAnalysisManager.java:790)
    at ghidra.app.plugin.core.analysis.AutoAnalysisManager.startAnalysis(AutoAnalysisManager.java:669)
    at ghidra.app.plugin.core.analysis.AutoAnalysisManager.startAnalysis(AutoAnalysisManager.java:634)
    at ghidra.app.plugin.core.analysis.AnalysisBackgroundCommand.applyTo(AnalysisBackgroundCommand.java:58)
    at ghidra.framework.plugintool.mgr.BackgroundCommandTask.run(BackgroundCommandTask.java:102)
    at ghidra.framework.plugintool.mgr.ToolTaskManager.run(ToolTaskManager.java:336)
    at java.base/java.lang.Thread.run(Thread.java:1623)

2023-07-11 23:24:49 ERROR (DWARFFunctionImporter) Error creating data object at 1000393b8 ghidra.program.model.util.CodeUnitInsertionException: Could not create Data at address 1000393b8
    at ghidra.program.model.data.DataUtilities.getData(DataUtilities.java:249)
    at ghidra.program.model.data.DataUtilities.createData(DataUtilities.java:164)
    at ghidra.program.model.data.DataUtilities.createData(DataUtilities.java:144)
    at ghidra.app.util.bin.format.dwarf4.next.DWARFFunctionImporter.outputGlobal(DWARFFunctionImporter.java:425)
    at ghidra.app.util.bin.format.dwarf4.next.DWARFFunctionImporter.importFunctions(DWARFFunctionImporter.java:127)
    at ghidra.app.util.bin.format.dwarf4.next.DWARFParser.parse(DWARFParser.java:202)
    at ghidra.app.plugin.core.analysis.DWARFAnalyzer.added(DWARFAnalyzer.java:199)
    at ghidra.app.plugin.core.analysis.AnalysisScheduler.runAnalyzer(AnalysisScheduler.java:186)
    at ghidra.app.plugin.core.analysis.AnalysisTask.applyTo(AnalysisTask.java:39)
    at ghidra.app.plugin.core.analysis.AutoAnalysisManager$AnalysisTaskWrapper.run(AutoAnalysisManager.java:690)
    at ghidra.app.plugin.core.analysis.AutoAnalysisManager.startAnalysis(AutoAnalysisManager.java:790)
    at ghidra.app.plugin.core.analysis.AutoAnalysisManager.startAnalysis(AutoAnalysisManager.java:669)
    at ghidra.app.plugin.core.analysis.AutoAnalysisManager.startAnalysis(AutoAnalysisManager.java:634)
    at ghidra.app.plugin.core.analysis.AnalysisBackgroundCommand.applyTo(AnalysisBackgroundCommand.java:58)
    at ghidra.framework.plugintool.mgr.BackgroundCommandTask.run(BackgroundCommandTask.java:102)
    at ghidra.framework.plugintool.mgr.ToolTaskManager.run(ToolTaskManager.java:336)
    at java.base/java.lang.Thread.run(Thread.java:1623)

...

2023-07-11 23:29:19 INFO  (AutoAnalysisManager) -----------------------------------------------------
    ASCII Strings                              0.400 secs
    Apply Data Archives                        0.334 secs
    Call Convention ID                       264.471 secs
    Call-Fixup Installer                       0.013 secs
    Create Address Tables                      0.037 secs
    Create Address Tables - One Time           0.015 secs
    Create Function                            0.099 secs
    DWARF                                      1.347 secs
    Data Reference                             0.014 secs
    Decompiler Switch Analysis                 0.027 secs
    Decompiler Switch Analysis - One Time      0.085 secs
    Demangler GNU                              0.066 secs
    Disassemble                                0.071 secs
    Disassemble Entry Points                   0.711 secs
    Disassemble Entry Points - One Time        0.003 secs
    Embedded Media                             0.009 secs
    External Entry References                  0.001 secs
    External Symbol Resolver                   0.007 secs
    Function ID                                0.384 secs
    Function Start Search                      0.064 secs
    Function Start Search After Code           0.027 secs
    Function Start Search After Data           0.015 secs
    Mach-O Function Starts                     0.011 secs
    Non-Returning Functions - Discovered       0.768 secs
    Non-Returning Functions - Known            0.020 secs
    Reference                                  0.022 secs
    Scalar Operand References                  0.193 secs
    Shared Return Calls                        0.061 secs
    Stack                                      0.890 secs
    Subroutine References                      0.025 secs
    x86 Constant Reference Analyzer            0.736 secs
-----------------------------------------------------
     Total Time   270 secs
-----------------------------------------------------

2023-07-11 23:29:19 DEBUG (ToolTaskManager) Auto Analysis task finish (271.322 secs)  
2023-07-11 23:29:19 DEBUG (ToolTaskManager) Queue - Auto Analysis  
2023-07-11 23:29:19 DEBUG (ToolTaskManager) (0.0 secs)  
2023-07-11 23:29:19 DEBUG (ToolTaskManager) Auto Analysis task complete (271.329 secs)  
2023-07-11 23:29:19 DEBUG (ToolTaskManager) Background processing complete (271.336 secs)  
2023-07-11 23:29:48 INFO  (RecoveryMgr) Tue Jul 11 23:29:48 ADT 2023 Recovery snapshot created: /Users/DM/work/trunk/ghidra/test.rep/idata/00/~00000000.db/snapshotA.grf

I'm not sure if this is OK or not. I suspect not... because when I set a breakpoint on the my entrypoint and start debugging.... it just hangs :-(.

Any suggestion would be appreciated if I haven't overstayed my welcome .

d-millar commented 1 year ago

Ah, good catch re sed+spaces and the minor release issues. The script was contributed by another user, and I think I haven’t quite grokked some of its subtleties. I will fix the sed error per your suggestion. May need to think about a generic fix for the minor version issue.

With regard to the error messages you’re seeing, all of those relate to importing the test file, i.e. you would have seen identical messages had you dragged the file into the usual code viewer tool. From that perspective, they have essentially nothing to do with the debugger and aren’t related to your current problem.

How are you starting the debugger? and are you placing the breakpoint from the static listing, the dynamic listing, or the interpreter CLI? Also, once the debugger is started, are you getting entries in the Modules View?

(and, nope, you haven’t overstayed your welcome - you’ll have to work much harder to get there, lol!)

d-millar commented 1 year ago

@DAMisener Well, I've added some code to the script to handle the "brew unpack" issue which causes 16.0.6 to go unrecognized. Surprisingly, the sed issue is actually more complicated. I can make the mods you've suggested and get the path with spaces injected into launch.properties, but that actually just creates a new problem. It appears java.library.path cannot handle spaces in the value, and I cannot for the life of me figure out a workaround. Maybe a stupid question, but do you really have to include "/Volumes/Macintosh HD 1" in GHIDRA_INSTALL_DIR? I would have thought /usr/local/Caskroom/ghidra... would have been sufficient.

cyberkaida commented 1 year ago

@d-millar @ryanmkurtz Please let me know if you would like me to look at the script again. Please feel free to tag me if there are issues with the script you would like help with fixing. I am happy to help maintain it.

d-millar commented 1 year ago

@cyberkaida Much appreciated - I think we're in reasonable shape right now. I made a few changes to change the default version to 16 and to handle some brew oddities, but this most recent issue (above) is really a problem with Java, not the script. Thanks, though! Will keep you posted if I get stuck!

strayge commented 1 year ago

Got the same error file not found "lldb/lldb-public.h" with latest patch https://github.com/NationalSecurityAgency/ghidra/commit/07b664bbbffd028969ac28c0945ee80c8249eef1 (ghidra installed via brew)

For me LLVM_HOME was set to /var/folders/b3/7f03t1zn0j39_x_9gzc7scpm0000gn/T/tmp.mkqw4cxwXF/llvm@16-*. While tmp.mkqw4cxwXF directory contains only llvm-16.0.6.

Seems like new check

if [ -z "${LLVM_HOME}" ]; then

should be changed to

if [ ! -d "${LLVM_HOME}" ]; then

as variable will never be empty, even when directory not exists.

d-millar commented 1 year ago

Yep, I think you're correct - my bash shell skills are apparently a little sketchy. Thanks!

marknelsonengineer-student commented 7 months ago

I can confirm that this is still an issue with:

After reading through the thread, recompiling Ghidra is too far out of reach for me.

Thank you all for your work on this issue. I'd love to get this working someday.

Alex-Vasile commented 7 months ago

After much hassle, I have gotten this running:

  1. Modify "macos_debugger_lldb_build_from_brew" to use LLVM@17. Current ghidra source requires a file that’s not present in LLVM@16.
  2. (Optional) Modify the script to use a permanent folder instead of a temp folder so that subsequent runs are fast
  3. Install ghidra, LLVM@17, gradle@7, and openjdk@17 (NOT temurin as brew suggests since that's Java21) from brew. Make sure these are the versions that your shell will find if you have multiple installed.
  4. DO NOT modify the buildNatives.gradle. The paths should be $llvm/lldb/include
  5. Run the script.
  6. Copy \path_to_ghidra_build\Ghidra\Debug\Debugger-swig-lldb\build\os\mac_arm_64 into \User\name\Library\Java\Extension
  7. Launch the brew-installed ghidra, you’ll have to do sudo ghidraRun to get the debugger agent to start or to connect (and re-create your project since it'll be run by a different user).
d-millar commented 7 months ago

@Alex-Vasile Apologies for not updating the script to 17 - my fault entirely. I upgraded the SWIG and forgot about the script. On the good new front, we're making changes that should make the debugger story for MacOS considerably simpler - no JNI, no SWIG. I hope you all will try the new versions out, and let us know what you think.

Alex-Vasile commented 7 months ago

@d-millar that sounds great! I will check it out, is there a PR or release for me to keep an eye out for?

Thanks for the work you're doing on this, I was frustrated with getting it running but I wasn't aiming any of it at you or whoever maintains that functionality.

d-millar commented 7 months ago

Should be in the next release - @nsadeveloper789's working on the documentation as we speak.

cyberkaida commented 6 months ago

I am excited to see the new changes, really looking forward to no JNI and SWIG!

I'll try it as soon as it is available! 🤩

nsadeveloper789 commented 6 months ago

FWIW, the master branch changed over to the new "Trace RMI" system yesterday (5a5b8eb3d771413fcc50a2c39d90a6a60f8a3c7a). There are some known issues I've encountered while updating the Tutorial, but you're welcome to give it a try by building from source.

marknelsonengineer-student commented 4 months ago

Aloha... first, Mahalo (thank you in Hawaiian) to the Ghidra team for working on this issue.

Second, this issue is closed as OBE, but I'd like to confirm with the team that this will be resolved in the next release. Is that correct?

Thank you all. --Mark

d-millar commented 4 months ago

@marknelsonengineer-student The new versions were released with 11.1. Try them out - let us know what you think!