search

radareorg / r2ghidra

Native Ghidra Decompiler for r2
https://www.radare.org/
GNU Lesser General Public License v3.0
340 stars 43 forks source link

Linux build errors v5.7.8 #100

Closed phodina closed 1 year ago

phodina commented 1 year ago

Description

Building r2ghidra v.5.7.8

During build there are multiple errors on Linux x86_64

../r2ghidra-5.7.8/src/ArchMap.cpp: In function ‘std::string       SleighIdFromCore(RCore*)’:
    ../r2ghidra-5.7.8/src/ArchMap.cpp:204:45: error: ‘static          void SleighArchitecture::collectSpecFiles(std::ostream&)’ is      protected within this context
      204 |  R2Architecture::collectSpecFiles (std::cerr);
          |                                             ^
    In file included from ../r2ghidra-5.7.8/src/ArchMap.h:6,
                     from ../r2ghidra-5.7.8/src/ArchMap.cpp:3:
    ../r2ghidra-5.7.8/ghidra-native/src/decompiler/sleigh_arch.       hh:100:15: note: declared protected here
      100 |   static void collectSpecFiles(ostream &errs);  ///<      Gather specification files in normal locations
          |               ^~~~~~~~~~~~~~~~
    ../r2ghidra-5.7.8/src/ArchMap.cpp:205:31: error:                  ‘getLanguageDescriptions’ is not a member of ‘R2Architecture’
      205 |  auto langs = R2Architecture::                            getLanguageDescriptions ();
1453       |                               ^~~~~~~~~~~~~~~~~~~~~~~

../r2ghidra-5.7.8/src/SleighInstruction.cpp: In member            function ‘void SleighParserContext::                              setPrototype(SleighInstructionPrototype*)’:
1506 ../r2ghidra-5.7.8/src/SleighInstruction.cpp:35:4: error:          ‘getBaseState’ was not declared in this scope; did you mean       ‘getParserState’?
       35 |   *getBaseState() = &prototype->rootState;
          |    ^~~~~~~~~~~~
          |    getParserState
../r2ghidra-5.7.8/src/SleighInstruction.cpp: In member            function ‘SleighParserContext* R2Sleigh::                         newSleighParserContext(Address&, SleighInstructionPrototype*      )’:
    ../r2ghidra-5.7.8/src/SleighInstruction.cpp:79:53: error:         ‘getContextCache’ was not declared in this scope; did you         mean ‘ContextCache’?
       79 |  SleighParserContext *pos = new                           SleighParserContext(getContextCache());
          |                                                           ^~~~~~~~~~~~~~~
          |                                                           ContextCache

I can upload the whole build log, but don't know where to attach it.

trufae commented 1 year ago

Looks like its trying to compile an unpatched ghidra decompiler

can you give me the commands you used to repro? It works well with meson and make in the ci on all tested OSs. But im not testing release snapshots

phodina commented 1 year ago

Yes @trufae, here are the definitions for the packages. The radare packaged is old so I updated the version as well. I'll upstream it once it builds and works

(define-module (ghidra)
  #:use-module (guix gexp)
  #:use-module (guix packages)
  #:use-module (guix download)
  #:use-module (gnu packages)
  #:use-module (gnu packages digest)
  #:use-module (gnu packages compression)
  #:use-module (gnu packages libevent)
  #:use-module (gnu packages tls)
  #:use-module (gnu packages pkg-config)
  #:use-module (gnu packages engineering)
  #:use-module (gnu packages xml)
  #:use-module (guix build-system gnu)
  #:use-module (guix build-system meson)
  #:use-module (guix build-system cmake)
  #:use-module ((guix licenses) #:prefix license:)
  #:use-module (guix git-download))

(define-public radare2-new
  (package
    (name "radare2")
    (version "5.7.8")
    (source (origin
              (method git-fetch)
              (uri (git-reference
                    (url "https://github.com/radareorg/radare2")
                    (commit version)))
              (patches
(parameterize
        ((%patch-path
           (map (lambda (directory)
                  (string-append directory "/expanse/packages/patches"))
                %load-path)))
     (search-patches
              "radare2-remove-git-clone.patch"
)))
              (sha256
               (base32
                "16j7kcy6b718zw6iy1kmqy4hbx6yngg89q83ccd2a37fh9w55zzv"))
              (file-name (git-file-name name version))))
    (build-system gnu-build-system)
    (arguments
     (list #:tests? #f                      ; tests require git and network access
       #:configure-flags
       #~(list "--with-openssl"
             "--with-rpath"
             "--with-syscapstone"
             "--with-sysmagic"
             "--with-syszip"
             "--with-sysxxhash")
       #:make-flags
       #~(list "CC=gcc")
    #:phases
    #~(modify-phases %standard-phases
    (delete 'validate-runpath)
         (add-before 'configure 'mklibdir
           (lambda* (#:key inputs outputs #:allow-other-keys)
             (mkdir-p (string-append #$output "/lib"))))
    (add-after 'unpack 'git-clone
    (lambda* _
    (copy-recursively #$(origin
              (method git-fetch)
              (uri (git-reference
              (url "https://github.com/radareorg/vector35-arch-arm64")
              (commit "9ab2b0bedde459dc86e079718333de4a63bbbacb")))

              (sha256
               (base32
                "10n6hx7jqraw12ir0x745xdyqmi72sgrjj4gjr0k07xa027fv8nq")))
 "libr/asm/arch/arm/v35arm64/arch-arm64")
    (copy-recursively #$(origin
              (method git-fetch)
              (uri (git-reference
              (url "https://github.com/radareorg/vector35-arch-armv7")
              (commit "dde39f69ffea19fc37e681874b12cb4707bc4f30")))

              (sha256
               (base32
                "0mbnc2a6j3s40620nvmzd9hjmwsyjpl7yww7himwyw2j8xrr0xbf")))
 "libr/asm/arch/arm/v35arm64/arch-armv7"))))))
    ;; TODO: Add gmp and libzip and make the build system actually find them.
    (inputs
     (list capstone libuv openssl zip))
    (native-inputs
     (list pkg-config))
    (propagated-inputs
     ;; In the Libs: section of r_hash.pc.
     (list xxhash))
    (home-page "https://radare.org/")
    (synopsis "Reverse engineering framework")
    (description
     "Radare2 is a complete framework for reverse-engineering, debugging, and
analyzing binaries.  It is composed of a set of small utilities that can be
used together or independently from the command line.

Radare2 is built around a scriptable disassembler and hexadecimal editor that
support a variety of executable formats for different processors and operating
systems, through multiple back ends for local and remote files and disk
images.

It can also compare (@dfn{diff}) binaries with graphs and extract information
like relocation symbols.  It is able to deal with malformed binaries, making
it suitable for security research and analysis.")
    (license license:lgpl3)))

(define-public r2ghidra
  (package
    (name "r2ghidra")
    (version "5.7.8")
    (source (origin
              (method url-fetch)
              (uri (string-append
              "https://github.com/radareorg/r2ghidra/archive/refs/tags/"
              version ".tar.gz"))
              (patches
(parameterize
        ((%patch-path
           (map (lambda (directory)
                  (string-append directory "/expanse/packages/patches"))
                %load-path)))
     (search-patches
     "r2ghidra-pugixml.patch"
     "r2ghidra-remove-git-dependency.patch"
)))
              (sha256
               (base32
                "1a8smjbpxj39bpqra2xxxpjlk86j4df8y3jbfgbz8daaxcw98yn8"))))
    (build-system meson-build-system)
    (arguments
    (list #:phases
    #~(modify-phases %standard-phases
    (add-after 'unpack 'git-clone
    (lambda* _
    (copy-recursively #$(origin
              (method git-fetch)
              (uri (git-reference
              (url "https://github.com/radareorg/ghidra-native")
              (commit "0.2.5")))

              (sha256
               (base32
                "15p4cnxjixf7fh0mg2hk476mdlci97y6ws710b7pc18r7lbhw9vp")))
 "ghidra-native"))))))
    (native-inputs (list pkg-config))
    (inputs (list pugixml openssl radare2-new))
    (home-page "https://www.radare.org/")
    (synopsis "Native Ghidra Decompiler for r2")
    (description "")
    (license license:lgpl3)))

And the patches to remove bundled dependencies:

From 5e7413ff4899ce44b9ddf0acbb61ad04f14894fa Mon Sep 17 00:00:00 2001
From: Petr Hodina <phodina@protonmail.com>
Date: Sat, 5 Nov 2022 23:58:01 +0100
Subject: [PATCH] pugixml dep

diff --git a/meson.build b/meson.build
index a36dff7..db68426 100644
--- a/meson.build
+++ b/meson.build
@@ -7,10 +7,6 @@ version : '5.7.6',
 default_options : ['c_std=c11', 'cpp_std=c++11']
 )

-pugixml_sources = [
-  'third-party/pugixml/src/pugixml.cpp'
-]
-
 r2ghidra_sources = [
   'src/ArchMap.cpp',
   'src/CodeXMLParse.cpp',
@@ -29,7 +25,6 @@ r2ghidra_sources = [

 incdirs = [
   'src',
-  'third-party/pugixml/src/',
   'ghidra-native/src/decompiler/',
 ]

@@ -168,7 +163,6 @@ ghidra_decompiler_sources = [

 r2ghidra_core_sources = [
   r2ghidra_sources,
-  pugixml_sources,
   ghidra_decompiler_sources,
   'src/anal_ghidra_plugin.c',
   'src/anal_ghidra.cpp',
@@ -178,7 +172,6 @@ r2ghidra_core_sources = [

 sleighc_sources = [
   r2ghidra_sources,
-  pugixml_sources,
   'ghidra-native/src/decompiler/slgh_compile.cc',
   'ghidra-native/src/decompiler/slghparse.cc',
   'ghidra-native/src/decompiler/slghscan.cc',
--
2.37.2
From 5e2f52b37fab2bec479ad453e45210f23aad3250 Mon Sep 17 00:00:00 2001
From: Petr Hodina <phodina@protonmail.com>
Date: Sat, 5 Nov 2022 23:37:17 +0100
Subject: [PATCH] remove git dependency

diff --git a/Makefile b/Makefile
index 9fbbb32..0bf6ff5 100644
--- a/Makefile
+++ b/Makefile
@@ -48,8 +48,7 @@ install uninstall user-install user-uninstall:
    $(MAKE) -C ghidra $@

 ghidra-native:
-   git clone https://github.com/radareorg/ghidra-native
-   cd ghidra-native && git checkout $(GHIDRA_NATIVE_COMMIT)
+   cd ghidra-native
    $(MAKE) -C ghidra-native patch

 mrproper: clean
diff --git a/configure b/configure
index 9ccfeb2..269a32d 100755
--- a/configure
+++ b/configure
@@ -287,7 +287,7 @@ parse_options "$1"
 shift
 done

-ENVWORDS="MANDIR INFODIR LIBDIR INCLUDEDIR LOCALSTATEDIR ETCDIR SYSCONFDIR DATADIR DOCDIR LIBEXECDIR SBINDIR BINDIR EPREFIX PREFIX SPREFIX TARGET HOST BUILD INSTALL INSTALL_LIB INSTALL_MAN INSTALL_PROGRAM INSTALL_PROGRAM_STRIP INSTALL_DIR INSTALL_SCRIPT INSTALL_DATA HOST_OS HOST_CPU BUILD_OS BUILD_CPU TARGET_OS TARGET_CPU VERSION VERSION_MAJOR VERSION_MINOR VERSION_PATCH VERSION_NUMBER PKGNAME VPATH CONTACT CONTACT_NAME CONTACT_MAIL CXX CXXFLAGS LDFLAGS HAVE_LANG_CXX CPP HAVE_GIT GIT PKGCONFIG R2_CFLAGS R2_LDFLAGS HAVE_PKGCFG_R_CORE R2ASM_CFLAGS R2ASM_LDFLAGS HAVE_PKGCFG_R_ASM R2ANAL_CFLAGS R2ANAL_LDFLAGS HAVE_PKGCFG_R_ANAL HAVE_R2 R2 HAVE_BISON BISON HAVE_FLEX FLEX R2_PREFIX R2_USER_PLUGINS R2_LIBR_PLUGINS"
+ENVWORDS="MANDIR INFODIR LIBDIR INCLUDEDIR LOCALSTATEDIR ETCDIR SYSCONFDIR DATADIR DOCDIR LIBEXECDIR SBINDIR BINDIR EPREFIX PREFIX SPREFIX TARGET HOST BUILD INSTALL INSTALL_LIB INSTALL_MAN INSTALL_PROGRAM INSTALL_PROGRAM_STRIP INSTALL_DIR INSTALL_SCRIPT INSTALL_DATA HOST_OS HOST_CPU BUILD_OS BUILD_CPU TARGET_OS TARGET_CPU VERSION VERSION_MAJOR VERSION_MINOR VERSION_PATCH VERSION_NUMBER PKGNAME VPATH CONTACT CONTACT_NAME CONTACT_MAIL CXX CXXFLAGS LDFLAGS HAVE_LANG_CXX CPP PKGCONFIG R2_CFLAGS R2_LDFLAGS HAVE_PKGCFG_R_CORE R2ASM_CFLAGS R2ASM_LDFLAGS HAVE_PKGCFG_R_ASM R2ANAL_CFLAGS R2ANAL_LDFLAGS HAVE_PKGCFG_R_ANAL HAVE_R2 R2 HAVE_BISON BISON HAVE_FLEX FLEX R2_PREFIX R2_USER_PLUGINS R2_LIBR_PLUGINS"

 create_environ

@@ -330,34 +330,6 @@ exit 1
    (echo "int main(int ac, char **av){return 0;}" | ${CPP}  >/dev/null 2>&1)
    if [ 0 = 0 ]; then echo ${CPP}; else
    echo "ERROR: ${CPP} cannot parse sources"; fi
-ochof "checking for git... "
-if [ -x "${GIT}" ]; then
-   FIND=${GIT}
-else
-   FIND=""
-   IFS=:
-   for A in ${PATH} ; do
-       if [ -x "${A}/git" ]; then
-           FIND="${A}/git"
-           break;
-       fi
-   done
-   unset IFS
-fi
-if [ -n "${FIND}" ]; then
-   ocho ${FIND};
-HAVE_GIT=1
-   GIT=${FIND}
-else
-   HAVE_GIT=0
-   GIT=git
-if [ "" = 1 ]; then
-echo "Cannot find git" >&2
-else
-echo no
-fi
-   echo "error: This program is required." >&2
-   exit 1 ; fi
 ochof "checking for pkg-config... "
 if [ -n "$PKGCONFIG" ]; then
    if [ -x "$PKGCONFIG" ]; then
--
2.37.2

The pugixml is packages within Guix. The philosophy is to add is as package dependency.

Git clone during package build fails as the build is done in container without network and all dependencies have to be declared first - therefore the repos are dowloaded in special phase.

Btw Guix is similar to Nix which also doesn't have this package yet.

trufae commented 1 year ago

if you run the preconfigure script you can do configure+make or meson without needing connectivity. about pugixml, the plan is to remove the dependency completely and use the xml parser that is shipped in r2.

Would you like to help on any of this? can you confirm if the preconfigure script solves the problem? about submodules/external/repos/dependencies i want to use meson wraps, because acr supports them too and its easier to maintain

phodina commented 1 year ago

I can run the preconfigure script and if it fails ask for help :-) Thanks so far for the guidance!

trufae commented 1 year ago

thanks, let me know if there's anything else? can this ticket be closed?

phodina commented 1 year ago

@trufae Sorry for late response. It turns out it's my fault as I forgot to apply the patches that come with ghidra-native.

It now builds and works correctly.

Just out of the curiosity why isn't the ghidra-native part of the git submodules? And why aren't the patches applied to some branch?

trufae commented 1 year ago

git submodules are problematic for several reasons:

The reason i maintain ghidra-native is to reduce techdebt. This is, ghidra repository is HUGE (500MB vs the only 24MB of ghidra-native that contains only the c++ code). but still I need to patch the code and i need to sync with upstream changes, which i didnt handle it recently because there are several breaking commits that make r2ghidra not to work at all. So, having the patch files separately it makes maintaining and updating the codebase from ghidra's source much easier.

Those patches should be applied automatically by the build system. applying the patches to a branch will make them harder to maintain and rebase because as i said, last ghidra master has changed many things that make the patches not applicable.

Ideally I think that the best way to go would be to make ghidra-native a standalone project, something that provides a pkg-config file and a library that r2ghidra can link to it. Not sure how this will work because the C++ mess, extending classes and all the global variables their code uses can result in more problems. So i'm still open to suggestions and of course i would be happy if someone would contribute or help to maintain any of this.

trufae commented 1 year ago

let's close the ticket :) feel free to keep talking here or reach me out in telegram/matrix/discord. happy to hear that it's working for you too

milahu commented 1 year ago

im getting similar errors with the latest version ...

``` [1/221] Compiling C++ object libcore_r2ghidra.so.p/src_CodeXMLParse.cpp.o In file included from ../src/CodeXMLParse.cpp:3: In file included from ../src/CodeXMLParse.h:6: In file included from /nix/store/2iw029qbnipa97blnx435526m40f65hs-radare2-5.7.2/include/libr/r_anal.h:10: In file included from /nix/store/2iw029qbnipa97blnx435526m40f65hs-radare2-5.7.2/include/libr/r_types.h:8: In file included from /nix/store/2iw029qbnipa97blnx435526m40f65hs-radare2-5.7.2/include/libr/r_types_base.h:8: In file included from /nix/store/rxsp481311w6d2ci16q2c0mx9wv46bjw-glibc-2.35-163-dev/include/ctype.h:25: /nix/store/rxsp481311w6d2ci16q2c0mx9wv46bjw-glibc-2.35-163-dev/include/features.h:412:4: warning: _FORTIFY_SOURCE requires compiling with optimization (-O) [-W#warnings] # warning _FORTIFY_SOURCE requires compiling with optimization (-O) ^ 1 warning generated. [2/221] Compiling C++ object libcore_r2ghidra.so.p/src_ArchMap.cpp.o FAILED: libcore_r2ghidra.so.p/src_ArchMap.cpp.o clang++ -Ilibcore_r2ghidra.so.p -I. -I.. -I../src -I../third-party/pugixml/src -I../ghidra-native/src/decompiler -I/nix/store/2iw029qbnipa97blnx435526m40f65hs-radare2-5.7.2/include/libr -I/nix/store/5knahpac8by9yd35snrqicavyvp044cc-openssl-3.0.7-dev/include -I/nix/store/w287qy6v8yr75iwvk5wn6nc3v8n7li0y-capstone-4.0.2/include/capstone -I/nix/store/2iw029qbnipa97blnx435526m40f65hs-radare2-5.7.2/include/libr/sdb -fcolor-diagnostics -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -Wnon-virtual-dtor -std=c++11 -O0 -fPIC -MD -MQ libcore_r2ghidra.so.p/src_ArchMap.cpp.o -MF libcore_r2ghidra.so.p/src_ArchMap.cpp.o.d -o libcore_r2ghidra.so.p/src_ArchMap.cpp.o -c ../src/ArchMap.cpp In file included from ../src/ArchMap.cpp:3: In file included from ../src/ArchMap.h:6: In file included from ../ghidra-native/src/decompiler/sleigh_arch.hh:22: In file included from ../ghidra-native/src/decompiler/filemanage.hh:21: In file included from /nix/store/8xmmwmb6r6hc6f3cdh64mbi138p96vpg-gcc-11.3.0/include/c++/11.3.0/vector:60: In file included from /nix/store/8xmmwmb6r6hc6f3cdh64mbi138p96vpg-gcc-11.3.0/include/c++/11.3.0/bits/stl_algobase.h:59: In file included from /nix/store/8xmmwmb6r6hc6f3cdh64mbi138p96vpg-gcc-11.3.0/include/c++/11.3.0/x86_64-unknown-linux-gnu/bits/c++config.h:586: In file included from /nix/store/8xmmwmb6r6hc6f3cdh64mbi138p96vpg-gcc-11.3.0/include/c++/11.3.0/x86_64-unknown-linux-gnu/bits/os_defines.h:39: /nix/store/rxsp481311w6d2ci16q2c0mx9wv46bjw-glibc-2.35-163-dev/include/features.h:412:4: warning: _FORTIFY_SOURCE requires compiling with optimization (-O) [-W#warnings] # warning _FORTIFY_SOURCE requires compiling with optimization (-O) ^ ../src/ArchMap.cpp:59:6: warning: field 'bits' will be initialized after field 'big_endian' [-Wreorder-ctor] , bits (bits) ^ ../src/ArchMap.cpp:204:18: error: 'collectSpecFiles' is a protected member of 'SleighArchitecture' R2Architecture::collectSpecFiles (std::cerr); ^ ../ghidra-native/src/decompiler/sleigh_arch.hh:100:15: note: declared protected here static void collectSpecFiles(ostream &errs); ///< Gather specification files in normal locations ^ ../src/ArchMap.cpp:205:31: error: no member named 'getLanguageDescriptions' in 'R2Architecture' auto langs = R2Architecture::getLanguageDescriptions (); ~~~~~~~~~~~~~~~~^ ../src/ArchMap.cpp:205:31: error: 'loadLanguageDescription' is a private member of 'SleighArchitecture' ../ghidra-native/src/decompiler/sleigh_arch.hh:95:15: note: implicitly declared private here static void loadLanguageDescription(const string &specfile,ostream &errs); ^ ../src/ArchMap.cpp:259:20: warning: unused variable 'am' [-Wunused-variable] const ArchMapper *am = &arch_it->second; ^ 3 warnings and 3 errors generated. ```

run the preconfigure script

It turns out it's my fault as I forgot to apply the patches that come with ghidra-native.

yes, fixed by applying the patches in ghidra-native/patches per ghidra-native/Makefile

https://github.com/radareorg/r2ghidra/blob/0799403b8ae3c2a09a176f1312825619912185e3/preconfigure#L6

https://github.com/radareorg/r2ghidra/blob/0799403b8ae3c2a09a176f1312825619912185e3/ghidra/Makefile#L25-L28

https://github.com/radareorg/ghidra-native/blob/6ce2dc2fee1ec1262bd0622c879ded8cb74e4ead/Makefile#L12-L16

patch: patch.done

patch.done:
    for a in $(shell ls patches/*.patch | sort -n) ; do echo "patch -p1 < $$a" ; patch -p1 < $$a ; done
    touch patch.done
trufae commented 1 year ago

The patches should be applied on a clean clone

milahu commented 1 year ago

@phodina did you have success in loading the r2ghidra plugin from a separate package? or did you simply create a single package with radare2 and the r2ghidra plugin?

https://github.com/NixOS/nixpkgs/issues/86448#issuecomment-622337145

$ r2 -H R2_LIBR_PLUGINS
/nix/store/fgiv9xh85yj26vyq4f8lp51fnbm6a7n0-radare2-4.3.1/lib/radare2/4.3.1

Ideally radare2 would accept environment variables to load plugins from.

trufae commented 1 year ago

There's the e dir.plugins eval var for this. but those R2_* env vars are not honored from r2.

is nix still shipping a 3yo version of r2?!?

milahu commented 1 year ago

the R2_LIBR_PLUGINS env should be used per https://github.com/radareorg/radare2/issues/21300

nixpkgs/pkgs/development/tools/analysis/radare2 → version 5.8.2