e-m-b-a / emba

EMBA - The firmware security analyzer
https://www.securefirmware.de
GNU General Public License v3.0
2.73k stars 236 forks source link

Extraction with Unblob Fails #368

Closed wowitsjack closed 2 years ago

wowitsjack commented 2 years ago

Issue:

When performing a default analysis as per the wiki, unblob will fail to parse the input file.

To Reproduce Steps to reproduce the behavior:

  1. EMBA installation (dev-mode/default mode)

  2. Use the firmware available here: https://static.tp-link.com/upload/firmware/2022/202201/20220111/Archer%20C1200(US)_V3_211219.zip | https://www.tp-link.com/us/support/download/archer-c1200/

  3. Start EMBA with the following parameters:EMBA="." FIRMWARE=/absolute/path/to/firmware LOG=/home/n/firmware_log/ docker-compose run emba

  4. See error

Expected behavior Expecting a clean unblob extraction.

Desktop:

==> Analyze binary firmware blob with unblob
-----------------------------------------------------------------
[*] Extracting firmware to directory /logs/p61_unblob_eval/unblob_extracted
2022-10-26 13:45.47 [info     ] Start processing file          file=/firmware pid=6019
2022-10-26 13:45.47 [error    ] Unhandled exception during unblob pid=6019
Traceback (most recent call last):
  File "/external/unblob/unblob/cli.py", line 241, in main
    reports = cli.invoke(ctx)
  File "/root/.cache/pypoetry/virtualenvs/unblob-CA0PO7D--py3.10/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/root/.cache/pypoetry/virtualenvs/unblob-CA0PO7D--py3.10/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/external/unblob/unblob/cli_options.py", line 15, in decorator
    return func(*args, **kwargs)
  File "/external/unblob/unblob/cli.py", line 204, in cli
    results = process_file(config, file, report_file)
  File "/external/unblob/unblob/signals.py", line 37, in decorator
    return func(*args, **kwargs)
  File "/external/unblob/unblob/processing.py", line 115, in process_file
    process_result = _process_task(config, task)
  File "/external/unblob/unblob/processing.py", line 132, in _process_task
    pool = make_pool(
  File "/external/unblob/unblob/pool.py", line 151, in make_pool
    return MultiPool(
  File "/external/unblob/unblob/pool.py", line 74, in __init__
    self._input = Queue(ctx=mp.get_context())
  File "/usr/lib/python3.10/multiprocessing/queues.py", line 297, in __init__
    Queue.__init__(self, maxsize, ctx=ctx)
  File "/usr/lib/python3.10/multiprocessing/queues.py", line 43, in __init__
    self._rlock = ctx.Lock()
  File "/usr/lib/python3.10/multiprocessing/context.py", line 68, in Lock
    return Lock(ctx=self.get_context())
  File "/usr/lib/python3.10/multiprocessing/synchronize.py", line 162, in __init__
    SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)
  File "/usr/lib/python3.10/multiprocessing/synchronize.py", line 57, in __init__
    sl = self._semlock = _multiprocessing.SemLock(
FileNotFoundError: [Errno 2] No such file or directory
m-1-k-3 commented 2 years ago

Works here. Probably you should start EMBA with the provided emba.sh script:

└─$ sudo ./emba.sh -f ~/Downloads/c1200v3-us-up-ver3-0-8-P1\[20211219-rel66975\]_2021-12-19_19.11.58.bin  -l ~/firmware-stuff/emba_logs_archer_c1200 -S -p ./scan-profiles/default-scan-emulation.emba -y -j

image

m-1-k-3 commented 2 years ago

Can we get an update on this issue?

wowitsjack commented 2 years ago

Still failing. Unable to get working under WSL2 in any form.

Unsure how to proceed.

[!] INFO: This is an evaluation module for the extractor unblob - https://unblob.org/.
[!] INFO: The results are currently not further used in the EMBA firmware analysis process (this will probably change in the future).

==> Analyze binary firmware blob with unblob
-----------------------------------------------------------------
[*] Extracting firmware to directory /logs/p61_unblob_eval/unblob_extracted
2022-11-04 13:15.07 [info     ] Start processing file          file=/firmware pid=6228
2022-11-04 13:15.07 [error    ] Unhandled exception during unblob pid=6228
Traceback (most recent call last):
  File "/external/unblob/unblob/cli.py", line 241, in main
    reports = cli.invoke(ctx)
  File "/root/.cache/pypoetry/virtualenvs/unblob-CA0PO7D--py3.10/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/root/.cache/pypoetry/virtualenvs/unblob-CA0PO7D--py3.10/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/external/unblob/unblob/cli_options.py", line 15, in decorator
    return func(*args, **kwargs)
  File "/external/unblob/unblob/cli.py", line 204, in cli
    results = process_file(config, file, report_file)
  File "/external/unblob/unblob/signals.py", line 37, in decorator
    return func(*args, **kwargs)
  File "/external/unblob/unblob/processing.py", line 115, in process_file
    process_result = _process_task(config, task)
  File "/external/unblob/unblob/processing.py", line 132, in _process_task
    pool = make_pool(
  File "/external/unblob/unblob/pool.py", line 151, in make_pool
    return MultiPool(
  File "/external/unblob/unblob/pool.py", line 74, in __init__
    self._input = Queue(ctx=mp.get_context())
  File "/usr/lib/python3.10/multiprocessing/queues.py", line 297, in __init__
    Queue.__init__(self, maxsize, ctx=ctx)
  File "/usr/lib/python3.10/multiprocessing/queues.py", line 43, in __init__
    self._rlock = ctx.Lock()
  File "/usr/lib/python3.10/multiprocessing/context.py", line 68, in Lock
    return Lock(ctx=self.get_context())
  File "/usr/lib/python3.10/multiprocessing/synchronize.py", line 162, in __init__
    SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)
  File "/usr/lib/python3.10/multiprocessing/synchronize.py", line 57, in __init__
    sl = self._semlock = _multiprocessing.SemLock(
FileNotFoundError: [Errno 2] No such file or directory
------------------------------------------------------------------------------------------------
Error detected - status code 1
Command:  echo -e "Command:  $ORANGE$BASH_COMMAND$NC"
Location:  ./modules/P61_unblob_eval.sh, line 125
Stack Trace:
    [1] unblobber(): ./modules/P61_unblob_eval.sh, line 125 -> unblobber /firmware /logs/p61_unblob_eval/unblob_extracted
    [2] P61_unblob_eval(): ./modules/P61_unblob_eval.sh, line 61 -> P61_unblob_eval
    [3] run_modules(): ./emba.sh, line 179 -> run_modules P 1 0
    [4] main(): ./emba.sh, line 928 -> main
    [5] main(): ./emba.sh, line 1064 -> main

Important: Consider filling out a bug report at https://github.com/e-m-b-a/emba/issues
wowitsjack commented 2 years ago

I'm kinda at my wits end trying to get this running under any capacity. Is there an email I could reach you at or something? I will literally pay money to watch you set this up, so I can see what all 14 of us who've tried to get this running must be missing. Because we're at a complete loss.

m-1-k-3 commented 2 years ago

What is your command starting up EMBA.

As WSL is in a very early experimental state I recommend using a virtual Ubuntu/Kali for a first try. If you really want using WSL I recommend reading this wiki entry first: https://github.com/e-m-b-a/emba/wiki/Experimental-WSL-support

wowitsjack commented 2 years ago

If I start it with EMBA="." FIRMWARE=/home/user/firmware/ LOG=/home/user/firmware_logs/ docker-compose run emba I get this: 2022-11-04 23_40_12

If I run it without the docker/env command as above, I get this: 2022-11-04 23_42_24

2022-11-04 23_47_01

Full trace at https://paste2.org/k7N5kjDs

m-1-k-3 commented 2 years ago

Could you please start EMBA as described here https://github.com/e-m-b-a/emba/issues/368#issuecomment-1293077261 or here https://github.com/e-m-b-a/emba#quick-start-with-default-scan-profile

If you try to start it the way you are doing you need to prepare a lot of things manually ...

wowitsjack commented 2 years ago

I have followed #368 and opened the Ubuntu terminal, entered the folder, and ran the command specified - https://paste2.org/MPUb0HtA

If I run the docker launch command first - https://paste2.org/Zjp0yJVx

I'm sorry if I'm sounding stupid, or missing something. None of us in our research group can figure this out, and we're a bit confused. We've tried upwards of 5 times each on multiple systems to get EMBA running, but following the instructions specified of:

is not working. At this point I am imploring anybody to record how they are setting this up, or using it, so we can maybe see what 17 graduate researchers couldn't.

m-1-k-3 commented 2 years ago
  • clone repo
  • run installer
  • run docker env command
  • run scan inside shell

I don't know from where you have these instructinos. I have posted the instructions now multiple times:

Please do not post non-serious offerings in this environment!

m-1-k-3 commented 2 years ago

I have followed #368 and opened the Ubuntu terminal, entered the folder, and ran the command specified - https://paste2.org/MPUb0HtA

looks not too bad. Try disabling unblob as described and check if can move on:

disable it in emba.sh setting the UNBLOB variable to 0

wowitsjack commented 2 years ago

Edit: Upon being accused of making 'non serious' offerings, and having my perfectly valid comments deleted by the author here, I'm going to say this project is probably not for us. You can close this issue, delete this comment too if you want, it's no longer my concern.

I'm afraid I can't recommend or move forwards with presenting work done with a tool of which the author attacks and deletes commenters.

Take care, and thank you.