search

pr0v3rbs / FirmAE

Towards Large-Scale Emulation of IoT Firmware for Dynamic Analysis
MIT License
642 stars 119 forks source link

Binwalk Extractor failed #33

Closed manihalvifred closed 2 years ago

manihalvifred commented 3 years ago

Just tested this on 2 freshly installed Ubuntu machines. When trying to run FirmAE, binwalk seems to throw some errors related to its running uid.

sudo ./run.sh -c dlink ../Downloads/dataset/dlink/DIR822B1_FW200KRb06.bin 
[*] ../Downloads/dataset/dlink/DIR822B1_FW200KRb06.bin emulation start!!!

Extractor Exception: Binwalk extraction uses many third party utilities, which may not be secure. If you wish to have extraction utilities executed as the current user, use '--run-as=root' (binwalk itself must be run as root).
----------------------------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/dist-packages/binwalk/core/module.py", line 258, in __init__
    self.load()
  File "/usr/local/lib/python3.9/dist-packages/binwalk/modules/extractor.py", line 147, in load
    raise ModuleException("Binwalk extraction uses many third party utilities, which may not be secure. If you wish to have extraction utilities executed as the current user, use '--run-as=%s' (binwalk itself must be run as root)." % user_info.pw_name)
binwalk.core.exceptions.ModuleException: Binwalk extraction uses many third party utilities, which may not be secure. If you wish to have extraction utilities executed as the current user, use '--run-as=root' (binwalk itself must be run as root).
pr0v3rbs commented 3 years ago

Thank you for your issue! Running binwalk with the root user is may unsafe, because of the third-party utilities. However, the run.sh script is working with the root user. Thus, the --run-as=root command was added in the extractor.py.

As described in the binwalk script, be careful of third-party utilities.

YB1-cyber commented 3 years ago

Im afraid it now raises a new issue ... # ./run.sh -d tplink /root/Desktop/firmware/DIR880A1_FW107WWb08.bin [*] /root/Desktop/firmware/DIR880A1_FW107WWb08.bin emulation start!!!

General Error: Cannot open file --run-as=root (CWD: /tmp/tmps7l2ay4u) : [Errno 2] No such file or directory: '--run-as=root'

pr0v3rbs commented 3 years ago

@YB1-cyber It still needs to run with the root user. Try this sudo ./run.sh -d tplink /root/Desktop/firmware/DIR880A1_FW107WWb08.bin command.

YB1-cyber commented 3 years ago

@pr0v3rbs Im Already root user that's why sudo is missing in the command. (anyway i checked also with sudo before i commented.)

# sudo ./run.sh -d tplink /root/Desktop/firmware/DIR880A1_FW107WWb08.bin [*] /root/Desktop/firmware/DIR880A1_FW107WWb08.bin emulation start!!!

General Error: Cannot open file --run-as=root (CWD: /tmp/tmpgm2pp_2h) : [Errno 2] No such file or directory: '--run-as=root'

Traceback (most recent call last): File "/root/Desktop/FirmAE/./sources/extractor/extractor.py", line 445, in extract for module in binwalk.scan(self.item, "--run-as=root", "-e", "-r", "-C", self.temp, File "/usr/lib/python3/dist-packages/binwalk/init.py", line 10, in scan objs = m.execute() File "/usr/lib/python3/dist-packages/binwalk/core/module.py", line 783, in execute obj = self.run(module) File "/usr/lib/python3/dist-packages/binwalk/core/module.py", line 802, in run obj = self.load(module, kwargs) File "/usr/lib/python3/dist-packages/binwalk/core/module.py", line 833, in load argv.update(self.dependencies(module, argv['enabled'])) File "/usr/lib/python3/dist-packages/binwalk/core/module.py", line 865, in dependencies raise ModuleException("Failed to load " + dependency.name + " module") binwalk.core.exceptions.ModuleException: Failed to load General module

thanks

pr0v3rbs commented 3 years ago

@YB1-cyber Can you try after reinstalling the binwalk? Check this command python3 -m pip install git+https://github.com/ReFirmLabs/binwalk.

YB1-cyber commented 3 years ago

same error... # python3 -m pip install git+https://github.com/ReFirmLabs/binwalk Collecting git+https://github.com/ReFirmLabs/binwalk Cloning https://github.com/ReFirmLabs/binwalk to /tmp/pip-req-build-fh9ig121 Running command git clone -q https://github.com/ReFirmLabs/binwalk /tmp/pip-req-build-fh9ig121 Building wheels for collected packages: binwalk Building wheel for binwalk (setup.py) ... done Created wheel for binwalk: filename=binwalk-2.3.3+fa0c0bd-py3-none-any.whl size=135413 sha256=45af1106e0f68ba61ba68d36d588e6d9b3a45b1ebc374047a7a18b5fedeaa562 Stored in directory: /tmp/pip-ephem-wheel-cache-4bd365is/wheels/c0/a5/3b/542feb60450a355008411f9ce955c66df12475959305a35faa Successfully built binwalk Installing collected packages: binwalk Attempting uninstall: binwalk Found existing installation: binwalk 2.3.2 Not uninstalling binwalk at /usr/lib/python3/dist-packages, outside environment /usr Can't uninstall 'binwalk'. No files were found to uninstall. Successfully installed binwalk-2.3.3+fa0c0bd

# uname -a Linux kali 5.10.0-kali9-amd64 #1 SMP Debian 5.10.46-4kali1 (2021-08-09) x86_64 GNU/Linux

pr0v3rbs commented 3 years ago

It seems the recent binwalk is not compatible with the root user (But, sudo executing looks to work well).

I'm checking now. Thank you for your issue!

YB1-cyber commented 3 years ago

@pr0v3rbs thanks, i'll wait pls ping me when there're good news

pr0v3rbs commented 3 years ago

@YB1-cyber

There are multiple issues on binwalk 2.3.3 such as extracting symbolic link files about the root (/) directory and root user permission.

I currently set the binwalk version 2.3.1. (Check install.sh) The command changed with python3 -m pip install git+https://github.com/ReFirmLabs/binwalk@772f271.

Please reinstall with the 2.3.1 version binwalk and try again! I will check the compatibility with the latest binwalk.

Thank you!

YB1-cyber commented 3 years ago

after a new clean install ... # binwalk Binwalk v2.3.2

# ./run.sh -d tplink /root/Desktop/firmware/DIR880A1_FW107WWb08.bin [] /root/Desktop/firmware/DIR880A1_FW107WWb08.bin emulation start!!! [] extract done!!! [*] get architecture done!!! mke2fs 1.46.4 (18-Aug-2021) e2fsck 1.46.4 (18-Aug-2021) e2fsck: Cannot continue, aborting.

(although volume is mounted on /dev/loop14p1)

pr0v3rbs commented 3 years ago

@YB1-cyber

It seems the problem of the binwalk was solved.

About the mounted error issue, please check this issue #31.

Thank you!

loverics commented 2 years ago

In FirmAE/sources/extractor/extractor.py on line 445 add --run-as=root to binwalk-scan. This will allow binwalk to run as expected.

for module in binwalk.scan(self.item, "--run-as=root", "-e", "-r", "-C", self.temp,
                                       signature=True, quiet=True):
loverics commented 2 years ago

I added PR #36 to address this issue.

pr0v3rbs commented 2 years ago

@loverics Check this https://github.com/pr0v3rbs/FirmAE/pull/36#issuecomment-962585681.

M4tir commented 2 years ago

@loverics Thank you!

pr0v3rbs commented 2 years ago

This issue was fixed in 6911f56

mittybeer commented 2 years ago

Great!

Thanks a lot. I Will be testing it with my students next week.

Kind regards,

Mick Beer

Op zo 31 jul. 2022 om 11:18 schreef Mingeun Kim @.***>

This issue was fixed in 6911f56 https://github.com/pr0v3rbs/FirmAE/commit/6911f56d92937b0d30bb58ab36d622c31a97a1ee

— Reply to this email directly, view it on GitHub https://github.com/pr0v3rbs/FirmAE/issues/33#issuecomment-1200385427, or unsubscribe https://github.com/notifications/unsubscribe-auth/ATNYJ77YT72JVPAY2JFAMX3VWZAGNANCNFSM5E5BHUDA . You are receiving this because you are subscribed to this thread.Message ID: @.***>