search

NationalSecurityAgency / ghidra

Ghidra is a software reverse engineering (SRE) framework
https://www.nsa.gov/ghidra
Apache License 2.0
52.25k stars 5.92k forks source link

Ghidra Headless mode freezing up #2145

Open sarahcxh opened 4 years ago

sarahcxh commented 4 years ago

When doing batch processing (with around 100 files) with post-script analysis for every binary, my Headless Analyzer freezes up. It freezes immediately after the post-script is completed and does not continue running after that even after waiting for 10 minutes or so. However, when I execute Ghidra's headless mode in a different terminal and just process that one file only, it will work and will not freeze up.

dragonmacher commented 4 years ago

Please provide a thread listing using java's jstack tool when in the bad state.

From a terminal, execute: jps to get the pid jstack <pid>

ryanmkurtz commented 3 years ago

Closing due to inactivity.

moyix commented 2 years ago

I think I'm seeing the same issue running Ghidra inside a singularity container to process multiple files in headless mode. After processing the last one, it will sometimes (~5% of the time) hang. Here is a stack trace captured using jstack:

[bd52@cm006 ~]$ singularity exec --bind /scratch/bd52/nn_comments_bins_stripped:/fastdata/nn_comments_bins /scratch/bd52/ghdock_greene.sif bash
Singularity> jps
2439332 Jps
2437626 Ghidra
Singularity> jstack 2437626
2022-03-27 13:31:39
Full thread dump OpenJDK 64-Bit Server VM (11.0.14+9-post-Debian-1 mixed mode):

Threads class SMR info:
_java_thread_list=0x00001463c4000c30, length=16, elements={
0x000014640c772800, 0x000014640c776800, 0x000014640c78f800, 0x000014640c795800,
0x000014640c797800, 0x000014640c79a000, 0x000014640c79c000, 0x000014640c847000,
0x000014640d0c6800, 0x000014640dde0000, 0x000014640de53800, 0x000014640de5e000,
0x000014636c02c000, 0x0000146370002000, 0x000014640c019800, 0x00001463c4001000
}

"Reference Handler" #2 daemon prio=10 os_prio=0 cpu=67.95ms elapsed=593.63s tid=0x000014640c772800 nid=0x25320c waiting on condition  [0x0000146410ec5000]
   java.lang.Thread.State: RUNNABLE
        at java.lang.ref.Reference.waitForReferencePendingList(java.base@11.0.14/Native Method)
        at java.lang.ref.Reference.processPendingReferences(java.base@11.0.14/Reference.java:241)
        at java.lang.ref.Reference$ReferenceHandler.run(java.base@11.0.14/Reference.java:213)

"Finalizer" #3 daemon prio=8 os_prio=0 cpu=30.04ms elapsed=593.63s tid=0x000014640c776800 nid=0x25320d in Object.wait()  [0x0000146410dc4000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(java.base@11.0.14/Native Method)
        - waiting on <no object reference available>
        at java.lang.ref.ReferenceQueue.remove(java.base@11.0.14/ReferenceQueue.java:155)
        - waiting to re-lock in wait() <0x0000000080a733c8> (a java.lang.ref.ReferenceQueue$Lock)
        at java.lang.ref.ReferenceQueue.remove(java.base@11.0.14/ReferenceQueue.java:176)
        at java.lang.ref.Finalizer$FinalizerThread.run(java.base@11.0.14/Finalizer.java:170)

"Signal Dispatcher" #4 daemon prio=9 os_prio=0 cpu=0.25ms elapsed=593.61s tid=0x000014640c78f800 nid=0x253218 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Service Thread" #5 daemon prio=9 os_prio=0 cpu=0.05ms elapsed=593.61s tid=0x000014640c795800 nid=0x253219 runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"C2 CompilerThread0" #6 daemon prio=9 os_prio=0 cpu=62187.66ms elapsed=593.61s tid=0x000014640c797800 nid=0x25321a waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"C1 CompilerThread0" #7 daemon prio=9 os_prio=0 cpu=12125.82ms elapsed=593.61s tid=0x000014640c79a000 nid=0x25321b waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE
   No compile task

"Sweeper thread" #8 daemon prio=9 os_prio=0 cpu=360.86ms elapsed=593.61s tid=0x000014640c79c000 nid=0x25321c runnable  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Common-Cleaner" #9 daemon prio=8 os_prio=0 cpu=3.37ms elapsed=593.58s tid=0x000014640c847000 nid=0x25321f in Object.wait()  [0x00001464107be000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(java.base@11.0.14/Native Method)
        - waiting on <no object reference available>
        at java.lang.ref.ReferenceQueue.remove(java.base@11.0.14/ReferenceQueue.java:155)
        - waiting to re-lock in wait() <0x000000008050d3b8> (a java.lang.ref.ReferenceQueue$Lock)
        at jdk.internal.ref.CleanerImpl.run(java.base@11.0.14/CleanerImpl.java:148)
        at java.lang.Thread.run(java.base@11.0.14/Thread.java:829)
        at jdk.internal.misc.InnocuousThread.run(java.base@11.0.14/InnocuousThread.java:161)

"Log4j2-TF-2-Scheduled-1" #12 daemon prio=5 os_prio=0 cpu=3.26ms elapsed=592.12s tid=0x000014640d0c6800 nid=0x253226 waiting on condition  [0x00001463f463b000]
   java.lang.Thread.State: TIMED_WAITING (parking)
        at jdk.internal.misc.Unsafe.park(java.base@11.0.14/Native Method)
        - parking to wait for  <0x0000000080755158> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.parkNanos(java.base@11.0.14/LockSupport.java:234)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(java.base@11.0.14/AbstractQueuedSynchronizer.java:2123)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.14/ScheduledThreadPoolExecutor.java:1182)
        at java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(java.base@11.0.14/ScheduledThreadPoolExecutor.java:899)
        at java.util.concurrent.ThreadPoolExecutor.getTask(java.base@11.0.14/ThreadPoolExecutor.java:1054)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.14/ThreadPoolExecutor.java:1114)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.14/ThreadPoolExecutor.java:628)
        at java.lang.Thread.run(java.base@11.0.14/Thread.java:829)

"FelixDispatchQueue" #14 prio=5 os_prio=0 cpu=0.12ms elapsed=590.26s tid=0x000014640dde0000 nid=0x25322d in Object.wait()  [0x00001463e07e5000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(java.base@11.0.14/Native Method)
        - waiting on <0x00000000814f46d8> (a java.util.ArrayList)
        at java.lang.Object.wait(java.base@11.0.14/Object.java:328)
        at org.apache.felix.framework.EventDispatcher.run(EventDispatcher.java:1122)
        - waiting to re-lock in wait() <0x00000000814f46d8> (a java.util.ArrayList)
        at org.apache.felix.framework.EventDispatcher.access$000(EventDispatcher.java:54)
        at org.apache.felix.framework.EventDispatcher$1.run(EventDispatcher.java:102)
        at java.lang.Thread.run(java.base@11.0.14/Thread.java:829)

"GhidraSwinglessTimer" #16 daemon prio=5 os_prio=0 cpu=52.20ms elapsed=590.06s tid=0x000014640de53800 nid=0x25323b in Object.wait()  [0x00001463e05e3000]
   java.lang.Thread.State: TIMED_WAITING (on object monitor)
        at java.lang.Object.wait(java.base@11.0.14/Native Method)
        - waiting on <no object reference available>
        at java.util.TimerThread.mainLoop(java.base@11.0.14/Timer.java:553)
        - waiting to re-lock in wait() <0x00000000814a1e50> (a java.util.TaskQueue)
        at java.util.TimerThread.run(java.base@11.0.14/Timer.java:506)

"File System Listener" #17 daemon prio=5 os_prio=0 cpu=0.11ms elapsed=590.06s tid=0x000014640de5e000 nid=0x25323d waiting on condition  [0x00001463e04e2000]
   java.lang.Thread.State: WAITING (parking)
        at jdk.internal.misc.Unsafe.park(java.base@11.0.14/Native Method)
        - parking to wait for  <0x00000000814a2e68> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
        at java.util.concurrent.locks.LockSupport.park(java.base@11.0.14/LockSupport.java:194)
        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.14/AbstractQueuedSynchronizer.java:2081)
        at java.util.concurrent.LinkedBlockingQueue.take(java.base@11.0.14/LinkedBlockingQueue.java:433)
        at ghidra.framework.store.FileSystemEventManager$FileSystemEventProcessingThread.run(FileSystemEventManager.java:187)

"GTimer" #31 daemon prio=5 os_prio=0 cpu=0.95ms elapsed=587.25s tid=0x000014636c02c000 nid=0x25326c in Object.wait()  [0x00001463929f4000]
   java.lang.Thread.State: WAITING (on object monitor)
        at java.lang.Object.wait(java.base@11.0.14/Native Method)
        - waiting on <no object reference available>
        at java.lang.Object.wait(java.base@11.0.14/Object.java:328)
        at java.util.TimerThread.mainLoop(java.base@11.0.14/Timer.java:527)
        - waiting to re-lock in wait() <0x0000000082b23c28> (a java.util.TaskQueue)
        at java.util.TimerThread.run(java.base@11.0.14/Timer.java:506)

"process reaper" #33 daemon prio=10 os_prio=0 cpu=0.43ms elapsed=587.23s tid=0x0000146370002000 nid=0x253270 runnable  [0x00001463f4038000]
   java.lang.Thread.State: RUNNABLE
        at java.lang.ProcessHandleImpl.waitForProcessExit0(java.base@11.0.14/Native Method)
        at java.lang.ProcessHandleImpl$1.run(java.base@11.0.14/ProcessHandleImpl.java:138)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(java.base@11.0.14/ThreadPoolExecutor.java:1128)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(java.base@11.0.14/ThreadPoolExecutor.java:628)
        at java.lang.Thread.run(java.base@11.0.14/Thread.java:829)

"DestroyJavaVM" #268 prio=5 os_prio=0 cpu=54214.49ms elapsed=498.60s tid=0x000014640c019800 nid=0x2531fb waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"Attach Listener" #269 daemon prio=9 os_prio=0 cpu=0.46ms elapsed=0.10s tid=0x00001463c4001000 nid=0x2538d1 waiting on condition  [0x0000000000000000]
   java.lang.Thread.State: RUNNABLE

"VM Thread" os_prio=0 cpu=272.65ms elapsed=593.63s tid=0x000014640c76a800 nid=0x25320b runnable

"GC Thread#0" os_prio=0 cpu=3923.61ms elapsed=593.66s tid=0x000014640c031800 nid=0x253200 runnable

"GC Thread#1" os_prio=0 cpu=4015.35ms elapsed=591.93s tid=0x00001463d8001000 nid=0x253229 runnable

"G1 Main Marker" os_prio=0 cpu=5.82ms elapsed=593.66s tid=0x000014640c064000 nid=0x253201 runnable

"G1 Conc#0" os_prio=0 cpu=1912.43ms elapsed=593.66s tid=0x000014640c065800 nid=0x253202 runnable

"G1 Refine#0" os_prio=0 cpu=49.49ms elapsed=593.64s tid=0x000014640c708000 nid=0x253209 runnable

"G1 Refine#1" os_prio=0 cpu=7.34ms elapsed=579.80s tid=0x00001463dc003000 nid=0x2532b1 runnable

"G1 Young RemSet Sampling" os_prio=0 cpu=563.92ms elapsed=593.64s tid=0x000014640c709800 nid=0x25320a runnable
"VM Periodic Task Thread" os_prio=0 cpu=72.48ms elapsed=593.51s tid=0x000014640c87e800 nid=0x253220 waiting on condition

JNI global refs: 17, weak refs: 45

Could this be reopened?