binsync / libbs

A library for writing plugins in any decompiler: includes API lifting, common data formatting, and GUI abstraction!
BSD 2-Clause "Simplified" License
63 stars 4 forks source link

Support Python3 in Ghidra, drop ghidra_bridge #82

Closed mahaloz closed 2 months ago

mahaloz commented 3 months ago

Closes #73 and bumps to the next major version.

mahaloz commented 3 months ago

Two separate issues may have killed this attempt:

No Subclassing Java Classes

  1. https://github.com/jpype-project/jpype/issues/420

This affects the following line(s): https://github.com/binsync/libbs/blob/40bc932e32d8b3e904c0faf451f727bd5d98f8f8/libbs/decompilers/ghidra/hooks.py#L185

This would mean libbs can no longer be used at all to make GUI callback classes unless we can find a workaround.

Implementing Java Classes is Thread Dependent (?)

  1. https://github.com/dod-cyber-crime-center/pyhidra/issues/41

It's possible there is a workaround for this. If not, then we no longer can do this: https://github.com/binsync/libbs/blob/40bc932e32d8b3e904c0faf451f727bd5d98f8f8/libbs/decompilers/ghidra/hooks.py#L17

That would mean we can no longer detect when any Artifact is changed, which means no caching or special change handling. This also makes the approach dead for BinSync proper.


I'll wait for a response from the Pyhidra dev's to see if this PR is dead in the water.

mahaloz commented 3 months ago

Based on the current commit, the following works in UI mode:

from libbs.api import DecompilerInterface
deci = DecompilerInterface.discover()
print(deci.functions.values())

It currently takes about 2 seconds to collect all the functions in light mode!

mahaloz commented 3 months ago

Headless mode now works:

from libbs.api import DecompilerInterface
deci = DecompilerInterface.discover(headless=True, force_decompiler="ghidra", binary_path="./posix_syscall")
for addr, light_func in deci.functions.items():
    print(light_func)
    print(deci.functions[addr]) # decompilation!

After headless is initiated, it takes only 0.7340998649597168 seconds to iterate and decompile every function!

mahaloz commented 2 months ago

Headless mode now works with project loading:

from libbs.api import DecompilerInterface
# load from a project should be 5x faster on this binary after loading once. 
deci = DecompilerInterface.discover(
    headless=True, force_decompiler="ghidra", binary_path="./posix_syscall"
    project_location=".", project_name="posix_syscall_ghidra", analyze=False
)
for addr, light_func in deci.functions.items():
    print(light_func)
    print(deci.functions[addr]) # decompilation!

This is also on CI: https://github.com/binsync/libbs/blob/104cd93f6ce63b1c42bdfbbdbca477b266df7f65/tests/test_decompilers.py#L154

mahaloz commented 2 months ago

Summary of PR

This pull request was closed and replaced by #84 since we were unable to fully replace ghidra_bridge backend with pyhidra. The following problems killed the full replacement:

  1. Inconsistent thread artifact changer callbacks, which was related to https://github.com/binsync/libbs/pull/82#issuecomment-2155323490. This manifested as inconsistent events being generated when changing something like a function name.
  2. Inability to use PySide6 inside pyhidra's interpreter. This problem took a lot of time to debug, and I never fully found the reason. It only manifested on Mac and could be generated with the following code in Pyhidra:
    from PySide6.QtWidgets import QApplication
    QApplication()

    This caused a segfault.

Since libbs is used by other downstream plugins for decompilers, which rely on Qt being available, #2 was the real killer.

I will leave this branch alive since, in its current state, it is usable inside the UI, just without QT and those callbacks being consistent.