e-m-b-a / embark

EMBArk - The firmware security scanning environment
https://www.securefirmware.de
MIT License
313 stars 46 forks source link

Not possible to install embark due to a dependency issue with unblob. #162

Closed Moneysac90 closed 10 months ago

Moneysac90 commented 10 months ago

There is a dependency issue when installing embark on Ubuntu 22.04.03 LTS Server.

OS-Version:

uname -a
Linux embark-vm 5.15.0-88-generic #98-Ubuntu SMP Mon Oct 2 15:18:56 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

I tried to install embark as described in the manual:

git clone https://github.com/e-m-b-a/embark.git
cd embark
sudo ./installer.sh -d

The installer shows the following error:

./helpers/helpers_emba_dependency_check.sh: line 476: [: 0000000000
0000000000
0000000000
0000000000
0000000000
0000000000
0000000000
0000000000
0000000000
0000000000
0000000000
0000000000
0000000000
0000000000: integer expression expected
    Unblob version Traceback (most recent call last):
  File "/external/emba_venv/bin/unblob", line 3, in <module>
    from unblob.cli import main
  File "/external/unblob/unblob/cli.py", line 20, in <module>
    from .handlers import BUILTIN_DIR_HANDLERS, BUILTIN_HANDLERS
  File "/external/unblob/unblob/handlers/__init__.py", line 8, in <module>
    from .archive.qnap import qnap_nas
  File "/external/unblob/unblob/handlers/archive/qnap/qnap_nas.py", line 81, in <module>
    hyperscan_stream_end_magic_db = build_stream_end_scan_db(FOOTER_PATTERN)
                                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/external/unblob/unblob/handlers/archive/qnap/qnap_nas.py", line 76, in build_stream_end_scan_db
    return StreamDatabase(
           ^^^^^^^^^^^^^^^
module.HyperscanCompileError: ('Unsupported architecture', -1) - not supported
    Upgrade your unblob installation to version 23.8.11 or higher

Furthermore, at the end of the output the following error message is displayed:

Some dependencies are missing - please check your installation

Looks like your docker container is outdated - please update your base image: sudo docker pull embeddedanalyzer/emba'.
ERROR: 1
[-] EMBA failed in docker mode!
[*] Final cleanup started.

The suggested docker command does not help. Apparently the most recent docker version is installed when executing sudo docker pull embeddedanalyzer/emba:

embark@embark-vm:~/embark$ sudo docker pull embeddedanalyzer/emba
[sudo] password for embark:
Using default tag: latest
latest: Pulling from embeddedanalyzer/emba
Digest: sha256:235928a6b8d3146c79216bea802f6e7b3befc5924c257e9b7364f544a79c5ef6
Status: Image is up to date for embeddedanalyzer/emba:latest
docker.io/embeddedanalyzer/emba:latest

According to the error message above the tool unblob is the issue. However according to pip it is installed:

pip freeze | grep -i unblob
unblob==23.10.31
unblob-native==0.1.1
torabi12 commented 10 months ago

Hello,

I used the same Ubuntu system and the installation went through fine on my side, I did not get any issues: kép

kép

User registration and login is also OK. I suggest to delete your VM, make a new Ubuntu Server installation with latest upgrades and do a snapshot of the clean system and install again EMBArk as did before. It will work I hope.

BenediktMKuehne commented 10 months ago

Could the issue be related to this? proxmox-discussion

Moneysac90 commented 10 months ago

I was able to resolve the issue. I am indeed using proxmox and the default CPU does not enable necessary features for unblob. After switching the CPU type from default to host it works fine. This of course depends on the host CPU.

Bildschirmfoto 2023-11-06 um 11 45 28

If possible it would be great to include a check/better error message.

m-1-k-3 commented 10 months ago

I included a link to the wiki https://github.com/e-m-b-a/emba/wiki/Installation#random-installation-topics

m-1-k-3 commented 10 months ago

If possible it would be great to include a check/better error message.

For this we need to find out which (missing) features are blocking the installation. Could you send us the output cat /proc/cpuinfo from the non-working and working system? If we find out how to figure out how, we can add an additional check and useful error message.

Moneysac90 commented 10 months ago

The working host CPU type:

embark@embark-vm:~$ cat /proc/cpuinfo
processor   : 0
vendor_id   : GenuineIntel
cpu family  : 6
model       : 151
model name  : 12th Gen Intel(R) Core(TM) i7-12700T
stepping    : 2
microcode   : 0x25
cpu MHz     : 1382.400
cache size  : 16384 KB
physical id : 0
siblings    : 8
core id     : 0
cpu cores   : 8
apicid      : 0
initial apicid  : 0
fpu     : yes
fpu_exception   : yes
cpuid level : 31
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq vmx ssse3 fma cx16 pdcm sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves avx_vnni arat umip pku ospke waitpkg gfni vaes vpclmulqdq rdpid movdiri movdir64b fsrm md_clear serialize arch_capabilities
vmx flags   : vnmi preemption_timer posted_intr invvpid ept_x_only ept_ad ept_1gb flexpriority apicv tsc_offset vtpr mtf vapic ept vpid unrestricted_guest vapic_reg vid shadow_vmcs pml tsc_scaling
bugs        : spectre_v1 spectre_v2 spec_store_bypass swapgs eibrs_pbrsb
bogomips    : 2764.80
clflush size    : 64
cache_alignment : 64
address sizes   : 46 bits physical, 48 bits virtual
power management:

And the non working default CPU type:

processor   : 0
vendor_id   : GenuineIntel
cpu family  : 15
model       : 6
model name  : Common KVM processor
stepping    : 1
microcode   : 0x1
cpu MHz     : 1382.400
cache size  : 16384 KB
physical id : 0
siblings    : 12
core id     : 0
cpu cores   : 12
apicid      : 0
initial apicid  : 0
fpu     : yes
fpu_exception   : yes
cpuid level : 13
wp      : yes
flags       : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx lm constant_tsc nopl xtopology cpuid tsc_known_freq pni cx16 x2apic hypervisor lahf_lm cpuid_fault pti
bugs        : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit mmio_unknown
bogomips    : 2764.80
clflush size    : 64
cache_alignment : 128
address sizes   : 40 bits physical, 48 bits virtual
power management:

I suspect unblob requires SSSE3. There are some hints that hyperscan needs SSSE3 For example:

Lynn-TX commented 10 months ago

Moneysac90... Thank you. I just tested this "After switching the CPU type from default to host" on our ProxMox system.

I will abandon the Linux Mint VM and clone the Kali VM on the ProbMox system.

Thank you again Moneysac90.

============= error from Linux Mint EMBA VM ( from term window buffer) ============== cpu_rec - ok unblob - ok Unblob version Traceback (most recent call last): File "/external/emba_venv/bin/unblob", line 3, in from unblob.cli import main File "/external/unblob/unblob/cli.py", line 20, in from .handlers import BUILTIN_DIR_HANDLERS, BUILTIN_HANDLERS File "/external/unblob/unblob/handlers/init.py", line 8, in from .archive.qnap import qnap_nas File "/external/unblob/unblob/handlers/archive/qnap/qnap_nas.py", line 81, in hyperscan_stream_end_magic_db = build_stream_end_scan_db(FOOTER_PATTERN) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/external/unblob/unblob/handlers/archive/qnap/qnap_nas.py", line 76, in build_stream_end_scan_db return StreamDatabase( ^^^^^^^^^^^^^^^ module.HyperscanCompileError: ('Unsupported architecture', -1) - not supported Upgrade your unblob installation to version 23.8.11 or higher

./helpers/helpers_emba_dependency_check.sh: line 476: [: 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000 0000000000: integer expression expected unrar - ok john - ok pixd visualizer - ok PHP iniscan - ok pixd image renderer - ok progpilot php ini checker - ok luacheck - ok patool - ok EnGenius decryptor - ok Android payload.bin extractor - ok Buffalo decryptor - ok ubireader image extractor - ok ubireader file extractor - ok UEFI image extractor - ok cve-search script - ok CVE Searchsploit - ok Routersploit EDB database - ok Routersploit CVE database - ok Metasploit CVE database - ok checksec script - ok sshdcc script - ok sudo-parser script - ok shellcheck script - ok fdtdump - ok linux-exploit-suggester.sh script - ok objdump disassembler - ok radare2 - ok bandit - python vulnerability scanner - ok qemu-[ARCH]-static - ok yara - ok cyclonedx - ok vmlinux-to-elf - ok STACS hash detection - ok Python virtual environment - ok

Some dependencies are missing - please check your installation

Looks like your docker container is outdated - please update your base image: sudo docker pull embeddedanalyzer/emba'. ERROR: 1 [-] EMBA failed in docker mode!

==================================================================