Open jstucke opened 3 years ago
Could you please provide the full log output from the analyzeHeadless. Could you also please share your binary to speed things up for us. I tried one of our binaries and could not reproduce the problem.
Could you please provide the full log output from the analyzeHeadless.
Here is the log for -processor ARM:LE:32:v5t
:
> analyzeHeadless . test_project -deleteProject -import hello_world.elf -postScript test.py -processor ARM:LE:32:v5t
OpenJDK 64-Bit Server VM warning: Archived non-system classes are disabled because the java.system.class.loader property is specified (value = "ghidra.GhidraClassLoader"). To use archived non-system classes, this property must not be set
openjdk version "14.0.1" 2020-04-14
OpenJDK Runtime Environment (build 14.0.1+7-Ubuntu-1ubuntu1)
OpenJDK 64-Bit Server VM (build 14.0.1+7-Ubuntu-1ubuntu1, mixed mode, sharing)
INFO Using log config file: jar:file:/opt/ghidra_9.1.2_PUBLIC/Ghidra/Framework/Generic/lib/Generic.jar!/generic.log4j.xml (LoggingInitialization)
INFO Using log file: /home/jstucke/.ghidra/.ghidra_9.1.2_PUBLIC/application.log (LoggingInitialization)
INFO Loading user preferences: /home/jstucke/.ghidra/.ghidra_9.1.2_PUBLIC/preferences (Preferences)
INFO Class search complete (510 ms) (ClassSearcher)
INFO Initializing SSL Context (SSLContextInitializer)
INFO Initializing Random Number Generator... (SecureRandomFactory)
INFO Random Number Generator initialization complete: NativePRNGNonBlocking (SecureRandomFactory)
INFO Trust manager disabled, cacerts have not been set (ApplicationTrustManagerFactory)
INFO HEADLESS Script Paths:
/home/jstucke/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Features/Base/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Features/BytePatterns/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Features/Decompiler/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Features/FileFormats/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Features/FunctionID/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Features/GnuDemangler/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Features/Python/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Features/VersionTracking/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Processors/8051/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Processors/DATA/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Processors/PIC/ghidra_scripts (HeadlessAnalyzer)
INFO HEADLESS: execution starts (HeadlessAnalyzer)
INFO Creating temporary project: /home/jstucke/test/ghidra/./test_project (HeadlessAnalyzer)
INFO Creating project: /home/jstucke/test/ghidra/./test_project (DefaultProject)
INFO REPORT: Processing input files: (HeadlessAnalyzer)
INFO project: /home/jstucke/test/ghidra/./test_project (HeadlessAnalyzer)
INFO IMPORTING: /home/jstucke/test/ghidra/hello_world.elf (HeadlessAnalyzer)
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by net.sf.cglib.core.ReflectUtils$2 (file:/opt/ghidra_9.1.2_PUBLIC/Ghidra/Framework/Generic/lib/cglib-nodep-2.2.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of net.sf.cglib.core.ReflectUtils$2
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
INFO REPORT: Import succeeded with language "ARM:LE:32:v5t" and cspec "default" for file: /home/jstucke/test/ghidra/hello_world.elf (HeadlessAnalyzer)
INFO ANALYZING all memory and code: /home/jstucke/test/ghidra/hello_world.elf (HeadlessAnalyzer)
INFO Packed database cache: /tmp/jstucke-Ghidra/packed-db-cache (PackedDatabaseCache)
INFO -----------------------------------------------------
ARM Constant Reference Analyzer 0.019 secs
ARM Pre-Pattern Analyzer 0.003 secs
ARM Symbol 0.001 secs
ASCII Strings 0.318 secs
Apply Data Archives 0.291 secs
Call Convention Identification 0.015 secs
Call-Fixup Installer 0.001 secs
Create Address Tables 0.007 secs
Create Function 0.019 secs
Data Reference 0.001 secs
Decompiler Switch Analysis 0.093 secs
Demangler 0.016 secs
Disassemble 0.009 secs
Disassemble Entry Points 0.044 secs
Disassemble Entry Points - One Time 0.010 secs
ELF Header Annotation 0.140 secs
Embedded Media 0.010 secs
External Entry References 0.000 secs
Function Start Search 0.047 secs
Function Start Search After Code 0.000 secs
Function Start Search After Data 0.000 secs
Function Start Search delayed - One Time 0.000 secs
Non-Returning Functions - Discovered 0.007 secs
Reference 0.001 secs
Stack 0.005 secs
Subroutine References 0.002 secs
-----------------------------------------------------
Total Time 1 secs
-----------------------------------------------------
(AutoAnalysisManager)
INFO REPORT: Analysis succeeded for file: /home/jstucke/test/ghidra/hello_world.elf (HeadlessAnalyzer)
INFO SCRIPT: /home/jstucke/test/ghidra/test.py (HeadlessAnalyzer)
Name: _DT_INIT
Name: FUN_000002ec
Name: FUN_00000304
Name: FUN_0000034c
Name: _DT_FINI
INFO ANALYZING changes made by post scripts: /home/jstucke/test/ghidra/hello_world.elf (HeadlessAnalyzer)
INFO REPORT: Post-analysis succeeded for file: /home/jstucke/test/ghidra/hello_world.elf (HeadlessAnalyzer)
INFO REPORT: Save succeeded for file: /hello_world.elf (HeadlessAnalyzer)
And here is the output for -processor ARM:LE:32:v8
:
analyzeHeadless . test_project -deleteProject -import hello_world.elf -postScript test.py -processor ARM:LE:32:v8
OpenJDK 64-Bit Server VM warning: Archived non-system classes are disabled because the java.system.class.loader property is specified (value = "ghidra.GhidraClassLoader"). To use archived non-system classes, this property must not be set
openjdk version "14.0.1" 2020-04-14
OpenJDK Runtime Environment (build 14.0.1+7-Ubuntu-1ubuntu1)
OpenJDK 64-Bit Server VM (build 14.0.1+7-Ubuntu-1ubuntu1, mixed mode, sharing)
INFO Using log config file: jar:file:/opt/ghidra_9.1.2_PUBLIC/Ghidra/Framework/Generic/lib/Generic.jar!/generic.log4j.xml (LoggingInitialization)
INFO Using log file: /home/jstucke/.ghidra/.ghidra_9.1.2_PUBLIC/application.log (LoggingInitialization)
INFO Loading user preferences: /home/jstucke/.ghidra/.ghidra_9.1.2_PUBLIC/preferences (Preferences)
INFO Class search complete (442 ms) (ClassSearcher)
INFO Initializing SSL Context (SSLContextInitializer)
INFO Initializing Random Number Generator... (SecureRandomFactory)
INFO Random Number Generator initialization complete: NativePRNGNonBlocking (SecureRandomFactory)
INFO Trust manager disabled, cacerts have not been set (ApplicationTrustManagerFactory)
INFO HEADLESS Script Paths:
/home/jstucke/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Features/Base/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Features/BytePatterns/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Features/Decompiler/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Features/FileFormats/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Features/FunctionID/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Features/GnuDemangler/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Features/Python/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Features/VersionTracking/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Processors/8051/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Processors/DATA/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Processors/PIC/ghidra_scripts (HeadlessAnalyzer)
INFO HEADLESS: execution starts (HeadlessAnalyzer)
INFO Creating temporary project: /home/jstucke/test/ghidra/./test_project (HeadlessAnalyzer)
INFO Creating project: /home/jstucke/test/ghidra/./test_project (DefaultProject)
INFO REPORT: Processing input files: (HeadlessAnalyzer)
INFO project: /home/jstucke/test/ghidra/./test_project (HeadlessAnalyzer)
INFO IMPORTING: /home/jstucke/test/ghidra/hello_world.elf (HeadlessAnalyzer)
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by net.sf.cglib.core.ReflectUtils$2 (file:/opt/ghidra_9.1.2_PUBLIC/Ghidra/Framework/Generic/lib/cglib-nodep-2.2.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of net.sf.cglib.core.ReflectUtils$2
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
INFO REPORT: Import succeeded with language "ARM:LE:32:v8" and cspec "default" for file: /home/jstucke/test/ghidra/hello_world.elf (HeadlessAnalyzer)
INFO ANALYZING all memory and code: /home/jstucke/test/ghidra/hello_world.elf (HeadlessAnalyzer)
INFO Packed database cache: /tmp/jstucke-Ghidra/packed-db-cache (PackedDatabaseCache)
INFO REPORT: Analysis succeeded for file: /home/jstucke/test/ghidra/hello_world.elf (HeadlessAnalyzer)
INFO SCRIPT: /home/jstucke/test/ghidra/test.py (HeadlessAnalyzer)
Name: _init
Name: printf
Name: __libc_start_main
Name: abort
Name: _start
Name: call_weak_fn
Name: deregister_tm_clones
Name: register_tm_clones
Name: __do_global_dtors_aux
Name: frame_dummy
Name: main
Name: __libc_csu_init
Name: __libc_csu_fini
Name: _fini
Name: __gmon_start__
Name: printf
Name: abort
Name: __libc_start_main
INFO ANALYZING changes made by post scripts: /home/jstucke/test/ghidra/hello_world.elf (HeadlessAnalyzer)
INFO REPORT: Post-analysis succeeded for file: /home/jstucke/test/ghidra/hello_world.elf (HeadlessAnalyzer)
INFO REPORT: Save succeeded for file: /hello_world.elf (HeadlessAnalyzer)
Could you also please share your binary to speed things up for us.
It is curious that the v8 import does not dump the analysis results. Was this removed from the v8 log output? Not sure why it would be missing.
It is curious that the v8 import does not dump the analysis results. Was this removed from the v8 log output? Not sure why it would be missing.
I was also wondering about that. But analyses like function search shouldn't be necessary, because the binary is not stripped. I did not omit anything from the log. It is the same behaviour as without the -processor
parameter:
> analyzeHeadless . test_project -deleteProject -import hello_world.elf -postScript test.py
OpenJDK 64-Bit Server VM warning: Archived non-system classes are disabled because the java.system.class.loader property is specified (value = "ghidra.GhidraClassLoader"). To use archived non-system classes, this property must not be set
openjdk version "14.0.1" 2020-04-14
OpenJDK Runtime Environment (build 14.0.1+7-Ubuntu-1ubuntu1)
OpenJDK 64-Bit Server VM (build 14.0.1+7-Ubuntu-1ubuntu1, mixed mode, sharing)
INFO Using log config file: jar:file:/opt/ghidra_9.1.2_PUBLIC/Ghidra/Framework/Generic/lib/Generic.jar!/generic.log4j.xml (LoggingInitialization)
INFO Using log file: /home/jstucke/.ghidra/.ghidra_9.1.2_PUBLIC/application.log (LoggingInitialization)
INFO Loading user preferences: /home/jstucke/.ghidra/.ghidra_9.1.2_PUBLIC/preferences (Preferences)
INFO Class search complete (434 ms) (ClassSearcher)
INFO Initializing SSL Context (SSLContextInitializer)
INFO Initializing Random Number Generator... (SecureRandomFactory)
INFO Random Number Generator initialization complete: NativePRNGNonBlocking (SecureRandomFactory)
INFO Trust manager disabled, cacerts have not been set (ApplicationTrustManagerFactory)
INFO HEADLESS Script Paths:
/home/jstucke/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Features/Base/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Features/BytePatterns/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Features/Decompiler/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Features/FileFormats/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Features/FunctionID/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Features/GnuDemangler/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Features/Python/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Features/VersionTracking/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Processors/8051/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Processors/DATA/ghidra_scripts
/opt/ghidra_9.1.2_PUBLIC/Ghidra/Processors/PIC/ghidra_scripts (HeadlessAnalyzer)
INFO HEADLESS: execution starts (HeadlessAnalyzer)
INFO Creating temporary project: /home/jstucke/test/ghidra/./test_project (HeadlessAnalyzer)
INFO Creating project: /home/jstucke/test/ghidra/./test_project (DefaultProject)
INFO REPORT: Processing input files: (HeadlessAnalyzer)
INFO project: /home/jstucke/test/ghidra/./test_project (HeadlessAnalyzer)
INFO IMPORTING: /home/jstucke/test/ghidra/hello_world.elf (HeadlessAnalyzer)
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by net.sf.cglib.core.ReflectUtils$2 (file:/opt/ghidra_9.1.2_PUBLIC/Ghidra/Framework/Generic/lib/cglib-nodep-2.2.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)
WARNING: Please consider reporting this to the maintainers of net.sf.cglib.core.ReflectUtils$2
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
INFO REPORT: Import succeeded with language "ARM:LE:32:v8" and cspec "default" for file: /home/jstucke/test/ghidra/hello_world.elf (HeadlessAnalyzer)
INFO ANALYZING all memory and code: /home/jstucke/test/ghidra/hello_world.elf (HeadlessAnalyzer)
INFO Packed database cache: /tmp/jstucke-Ghidra/packed-db-cache (PackedDatabaseCache)
INFO REPORT: Analysis succeeded for file: /home/jstucke/test/ghidra/hello_world.elf (HeadlessAnalyzer)
INFO SCRIPT: /home/jstucke/test/ghidra/test.py (HeadlessAnalyzer)
Name: _init
Name: printf
Name: __libc_start_main
Name: abort
Name: _start
Name: call_weak_fn
Name: deregister_tm_clones
Name: register_tm_clones
Name: __do_global_dtors_aux
Name: frame_dummy
Name: main
Name: __libc_csu_init
Name: __libc_csu_fini
Name: _fini
Name: __gmon_start__
Name: printf
Name: abort
Name: __libc_start_main
INFO ANALYZING changes made by post scripts: /home/jstucke/test/ghidra/hello_world.elf (HeadlessAnalyzer)
INFO REPORT: Post-analysis succeeded for file: /home/jstucke/test/ghidra/hello_world.elf (HeadlessAnalyzer)
INFO REPORT: Save succeeded for file: /hello_world.elf (HeadlessAnalyzer)
Describe the bug
When running scripts on ARM binaries in headless mode and explicitly stating the processor with
-processor ARM:LE:32:v8
, Ghidra treats the binary as if it was stripped if any other ARM version than "v8" is used (tested with v4t, v5t, v7 and "Cortex"). This effect does not occur when running the script manually through the GUI (regardless of the selected processor/architecture).To Reproduce
1. Compile Minimal Demo Binary
Create hello_world.c with content
(binary is compiled for armv5t and not stripped)
2. Create Minimal Demo Script
Create test.py with content:
3. Run Ghidra Headless with Script
Expected Behavior
The script should print the function names (like it does with
-processor ARM:LE:32:v8
):Actual Behavior
When passing
-processor ARM:LE:32:v5t
Ghidra seems to behave as if the binary was stripped (not all functions were found and the symbols are gone) -- or maybe I'm just doing it wrong:Environment