FernandoDoming / r2diaphora

Port of the binary diffing library, diaphora, for radare2 and mariadb
GNU Affero General Public License v3.0
49 stars 1 forks source link

KeyError and exceptions during binary analysis on Kali Linux #3

Closed SlidingWindow closed 1 year ago

SlidingWindow commented 1 year ago

I installed r2Diaphora using PIP3 on my Kali machine and getting errors/exceptions.

  1. Exception while trying to read function at 0x%x in sample %s
  2. "KeyError: 'instrs'" : It looks like this key is not present in the dictionary

/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py

`flow = log_exec_r2_cmdj(f"afbj @ {f}")

for block in flow: nodes += 1 block.update({"start": block["addr"], "end": block["addr"] + block["size"] }) instructions_data = [] block_ea = block["start"] - image_base idx = len(bb_topological) bb_topological[idx] = [] bb_topo_num[block_ea] = idx for x in block["instrs"]: _, ins = diaphora_decode(x) mnem = ins["mnemonic"] disasm = ins["disasm"]` Kali Machine: -------------- `PRETTY_NAME="Kali GNU/Linux Rolling" NAME="Kali GNU/Linux" VERSION="2023.1" VERSION_ID="2023.1" VERSION_CODENAME="kali-rolling" ID=kali ID_LIKE=debian HOME_URL="https://www.kali.org/" SUPPORT_URL="https://forums.kali.org/" BUG_REPORT_URL="https://bugs.kali.org/" ANSI_COLOR="1;31"` PIP3 version ------------- `pip 23.0.1 from /usr/lib/python3/dist-packages/pip (python 3.11)` Here's the stack trace ------------------------ `2023-05-28 02:43:22,587 [INFO] - Generating database 3f31d166a6decf13ce3a684c1e5ce9f8aff47e3da1c57ab63842e838c10e2ea8 for file [x] Analyze all flags starting with sym. and entry0 (aa) [x] Analyze function calls (aac) [x] Analyze len bytes of instructions for references (aar) [x] Finding and parsing C++ vtables (avrr) [x] Type matching analysis for all functions (aaft) [x] Propagate noreturn information (aanr) [x] Finding function preludes [x] Enable constraint types analysis for variables --- Logging error --- Traceback (most recent call last): File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 347, in read_function_with_timeout ret = self.read_function(f) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 449, in read_function for x in block["instrs"]: ~~~~~^^^^^^^^^^ KeyError: 'instrs' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.11/logging/__init__.py", line 1110, in emit msg = self.format(record) ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/logging/__init__.py", line 953, in format return fmt.format(record) ^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/logging/__init__.py", line 687, in format record.message = record.getMessage() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/logging/__init__.py", line 377, in getMessage msg = msg % self.args ~~~~^~~~~~~~~~~ TypeError: %x format: an integer is required, not str Call stack: File "/usr/local/bin/r2diaphora", line 8, in sys.exit(main()) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 1103, in main generate_db_for_file( File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 1004, in generate_db_for_file _gen_diaphora_db(filepath, dbname, function_filter=function_filter) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 985, in _gen_diaphora_db _diff_or_export(function_filter, dbname=out_db, userdata=userdata) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 925, in _diff_or_export bd.export(function_filter, userdata) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 134, in export self.do_export(function_filter, userdata) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 94, in do_export props = self.read_function_with_timeout(func, timeout = 60) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 356, in read_function_with_timeout log.exception( Message: 'Exception while trying to read function at 0x%x in sample %s' Arguments: ('134527204', '') 2023-05-28 02:43:23,081 [INFO] - Skipping uninteresting function sym.imp.__libc_start_main 2023-05-28 02:43:23,081 [INFO] - Skipping uninteresting function section..fini --- Logging error --- Traceback (most recent call last): File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 347, in read_function_with_timeout ret = self.read_function(f) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 449, in read_function for x in block["instrs"]: ~~~~~^^^^^^^^^^ KeyError: 'instrs' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.11/logging/__init__.py", line 1110, in emit msg = self.format(record) ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/logging/__init__.py", line 953, in format return fmt.format(record) ^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/logging/__init__.py", line 687, in format record.message = record.getMessage() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/logging/__init__.py", line 377, in getMessage msg = msg % self.args ~~~~^~~~~~~~~~~ TypeError: %x format: an integer is required, not str Call stack: File "/usr/local/bin/r2diaphora", line 8, in sys.exit(main()) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 1103, in main generate_db_for_file( File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 1004, in generate_db_for_file _gen_diaphora_db(filepath, dbname, function_filter=function_filter) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 985, in _gen_diaphora_db _diff_or_export(function_filter, dbname=out_db, userdata=userdata) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 925, in _diff_or_export bd.export(function_filter, userdata) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 134, in export self.do_export(function_filter, userdata) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 94, in do_export props = self.read_function_with_timeout(func, timeout = 60) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 356, in read_function_with_timeout log.exception( Message: 'Exception while trying to read function at 0x%x in sample %s' Arguments: ('134527248', '') 2023-05-28 02:43:23,083 [INFO] - Skipping uninteresting function section..init --- Logging error --- Traceback (most recent call last): File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 347, in read_function_with_timeout ret = self.read_function(f) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 449, in read_function for x in block["instrs"]: ~~~~~^^^^^^^^^^ KeyError: 'instrs' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.11/logging/__init__.py", line 1110, in emit msg = self.format(record) ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/logging/__init__.py", line 953, in format return fmt.format(record) ^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/logging/__init__.py", line 687, in format record.message = record.getMessage() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/logging/__init__.py", line 377, in getMessage msg = msg % self.args ~~~~^~~~~~~~~~~ TypeError: %x format: an integer is required, not str Call stack: File "/usr/local/bin/r2diaphora", line 8, in sys.exit(main()) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 1103, in main generate_db_for_file( File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 1004, in generate_db_for_file _gen_diaphora_db(filepath, dbname, function_filter=function_filter) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 985, in _gen_diaphora_db _diff_or_export(function_filter, dbname=out_db, userdata=userdata) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 925, in _diff_or_export bd.export(function_filter, userdata) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 134, in export self.do_export(function_filter, userdata) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 94, in do_export props = self.read_function_with_timeout(func, timeout = 60) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 356, in read_function_with_timeout log.exception( Message: 'Exception while trying to read function at 0x%x in sample %s' Arguments: ('134519668', '') --- Logging error --- Traceback (most recent call last): File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 347, in read_function_with_timeout ret = self.read_function(f) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 449, in read_function for x in block["instrs"]: ~~~~~^^^^^^^^^^ KeyError: 'instrs' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.11/logging/__init__.py", line 1110, in emit msg = self.format(record) ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/logging/__init__.py", line 953, in format return fmt.format(record) ^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/logging/__init__.py", line 687, in format record.message = record.getMessage() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/logging/__init__.py", line 377, in getMessage msg = msg % self.args ~~~~^~~~~~~~~~~ TypeError: %x format: an integer is required, not str Call stack: File "/usr/local/bin/r2diaphora", line 8, in sys.exit(main()) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 1103, in main generate_db_for_file( File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 1004, in generate_db_for_file _gen_diaphora_db(filepath, dbname, function_filter=function_filter) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 985, in _gen_diaphora_db _diff_or_export(function_filter, dbname=out_db, userdata=userdata) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 925, in _diff_or_export bd.export(function_filter, userdata) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 134, in export self.do_export(function_filter, userdata) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 94, in do_export props = self.read_function_with_timeout(func, timeout = 60) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 356, in read_function_with_timeout log.exception( Message: 'Exception while trying to read function at 0x%x in sample %s' Arguments: ('134527344', '') --- Logging error --- Traceback (most recent call last): File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 347, in read_function_with_timeout ret = self.read_function(f) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 449, in read_function for x in block["instrs"]: ~~~~~^^^^^^^^^^ KeyError: 'instrs' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/lib/python3.11/logging/__init__.py", line 1110, in emit msg = self.format(record) ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/logging/__init__.py", line 953, in format return fmt.format(record) ^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/logging/__init__.py", line 687, in format record.message = record.getMessage() ^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.11/logging/__init__.py", line 377, in getMessage msg = msg % self.args ~~~~^~~~~~~~~~~ TypeError: %x format: an integer is required, not str Call stack: File "/usr/local/bin/r2diaphora", line 8, in sys.exit(main()) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 1103, in main generate_db_for_file( File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 1004, in generate_db_for_file _gen_diaphora_db(filepath, dbname, function_filter=function_filter) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 985, in _gen_diaphora_db _diff_or_export(function_filter, dbname=out_db, userdata=userdata) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 925, in _diff_or_export bd.export(function_filter, userdata) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 134, in export self.do_export(function_filter, userdata) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 94, in do_export props = self.read_function_with_timeout(func, timeout = 60) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 356, in read_function_with_timeout log.exception( Message: 'Exception while trying to read function at 0x%x in sample %s' TypeError: %x format: an integer is required, not str Call stack: File "/usr/local/bin/r2diaphora", line 8, in sys.exit(main()) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 1103, in main generate_db_for_file( File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 1004, in generate_db_for_file _gen_diaphora_db(filepath, dbname, function_filter=function_filter) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 985, in _gen_diaphora_db _diff_or_export(function_filter, dbname=out_db, userdata=userdata) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 925, in _diff_or_export bd.export(function_filter, userdata) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 134, in export self.do_export(function_filter, userdata) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 94, in do_export props = self.read_function_with_timeout(func, timeout = 60) File "/usr/local/lib/python3.11/dist-packages/r2diaphora/diaphora_r2.py", line 356, in read_function_with_timeout log.exception( Message: 'Exception while trying to read function at 0x%x in sample %s' Arguments: ('134530992', '') fef753ce71602a2d4aed64cee700d4b0 2023-05-28 02:50:52,990 [INFO] - Database exported: 3f31d166a6decf13ce3a684c1e5ce9f8aff47e3da1c57ab63842e838c10e2ea8` Please let me know if I missed something. Thanks!
FernandoDoming commented 1 year ago

Hmm maybe the radare2 version is outdated, can you do a r2 -v and post the result?

SlidingWindow commented 1 year ago

@FernandoDoming,

Looks like it is. I see that the 5.8.6 is the latest one.

┌──(kali㉿kali)-[~] └─$ r2 -v
radare2 5.5.0 0 @ linux-x86-64 git.5.5.0 commit: unknown build:

FernandoDoming commented 1 year ago

Does it work with the latest r2?

Ret2c7 commented 1 year ago

I have the same problem

FernandoDoming commented 1 year ago

@Ret2c7 Please post the radare2 and r2diaphora versions you are using and attempt to update to latest version of both if not already on them.

Ret2c7 commented 1 year ago

@FernandoDoming radare2 5.8.7 30844 @ linux-x86-64 birth: git.5.8.6-172-gc9469af1fd 2023-06-07__11:23:24 commit: c9469af1fdc29315f424ef1aba7e898809bdf05e options: gpl -O? cs:5 cl:2 make

FernandoDoming commented 1 year ago

That looks new enough. To be able to reproduce and debug I'd need the binary file you are analyzing. The hash is enough if the file is in VirusTotal or OTX.

FernandoDoming commented 1 year ago

Closing as not reproducible. Also added safeguards for potential empty blocks reported by radare.