A recent update broke down ELF scans

[chimelab]

After updating to latest code (on branch master), ELF scans was broken down. Bang prompts many exceptions. Not sure which one is the root cause. Then I switched to an early version (on branch littlefs_kaitai), and it works well. Btw, I'm on ubuntu 22.04 and tried several files. The issue can be reproduced by scanning "file" located in /usr/bin/, or any other .so files.

[armijnhemel]

It is a little bit hard to debug without seeing the error.

Question: did you rebuild the Kaitai Struct files? The definition for ELF was changed recently so you would need to rerun "make". I will clarify this in the README.

[armijnhemel]

Possibly these are errors printed by pwntools. Perhaps something like this?

[ERROR] Address 0x3f is not contained in ELF('/tmp/bang/root/rel/bin/bash')

If so, then that's a completely harmless error from pwntools that I still need to squash and that you can ignore.

[chimelab]

Yes, I did rebuild (and also tried "make clean" before it). It seems like a common bug for any elf files. Two log files are attached here: usr-bin-file-with-master-latest.log -- bad one with latest code from branch master; usr-bin-file-with-littlefs_kaitai.log -- good one with latest code from branch littlefs_kaitai.

usr-bin-file-with-littlefs_kaitai.log usr-bin-file-with-master-latest.log

[chimelab]

Possibly these are errors printed by pwntools. Perhaps something like this?

[ERROR] Address 0x3f is not contained in ELF('/tmp/bang/root/rel/bin/bash')

If so, then that's a completely harmless error from pwntools that I still need to squash and that you can ignore.

It's a critical issue. Bang can't tell any metadata, such as strings, functions, varnames, etc, with the latest code from branch master. Only hashes are provided.

[armijnhemel]

Possibly these are errors printed by pwntools. Perhaps something like this?

[ERROR] Address 0x3f is not contained in ELF('/tmp/bang/root/rel/bin/bash')

If so, then that's a completely harmless error from pwntools that I still need to squash and that you can ignore.

It's a critical issue. Bang can't tell any metadata, such as strings, functions, varnames, etc, with the latest code from branch master. Only hashes are provided.

Yeah, it's not from pwntools. Something else is going on, I will investigate.

[armijnhemel]

Yes, I did rebuild (and also tried "make clean" before it). It seems like a common bug for any elf files. Two log files are attached here: usr-bin-file-with-master-latest.log -- bad one with latest code from branch master; usr-bin-file-with-littlefs_kaitai.log -- good one with latest code from branch littlefs_kaitai.

usr-bin-file-with-littlefs_kaitai.log usr-bin-file-with-master-latest.log

I can see that the ELF parser is not being loaded (in build_automaton). That's very strange. In the coming few days I will spin up a VM and test.

[chimelab]

Yes, I did rebuild (and also tried "make clean" before it). It seems like a common bug for any elf files. Two log files are attached here: usr-bin-file-with-master-latest.log -- bad one with latest code from branch master; usr-bin-file-with-littlefs_kaitai.log -- good one with latest code from branch littlefs_kaitai. usr-bin-file-with-littlefs_kaitai.log usr-bin-file-with-master-latest.log

I can see that the ELF parser is not being loaded (in build_automaton). That's very strange. In the coming few days I will spin up a VM and test.

Thanks for the reminding. The problem was fixed. It's my fault, because I forgot to install additional packages (pefile, xxhash, ...).

[chimelab]

Two more suggestions: 1, Add below lines in build_automation/_get_unpackers_recursive:

         except ModuleNotFoundError as e:
                if not e.msg.endswith(".UnpackParser'"):
                    traceback.print_exc() # added by hhao020
                pass

2, Add alpha tags (e.g. alpha-2023MMDD) regularly, per 1~2 weeks. This might help us (fans of BANG) choose a proper base whenever we get in troubles, to understand what were changed.