Closed AndrewFasano closed 3 months ago
Thank you for the very detailed report. I had a quick look and it's probably a bug in the CPIO extractor. I'll keep you posted.
The bug is triggered by a CPIO archive with the same entry stored twice:
7z l sample.cpio | grep thread_db-1.0
2012-01-11 23:17:40 ..... 32220 32220 /lib/libthread_db-1.0.so
2012-01-11 23:17:40 ..... 32220 32220 /lib/libthread_db-1.0.so
@AndrewFasano I opened the discussion on duplicate entries with a draft fix at https://github.com/onekey-sec/unblob/pull/756 , your feedback is very welcomed :)
$ cpio -t < 'M4100-V10.0.2.20.zip_extract/m4100v10.0.2.20.stk_extract/1201148-2097967.lzma_extract/lzma.uncompressed'| sort | uniq -c | sort -n
...
1 /var/run
1 /var/run/utmp
2 /lib/libthread_db-1.0.so
2 /sbin/cfe_env
2 /usr/bin/sort
2 /usr/bin/tail
2 /usr/bin/test
2 /usr/bin/tftp
2 /usr/bin/top
2 /usr/bin/traceroute
2 /usr/bin/uptime
2 /usr/bin/wc
2 /usr/bin/which
2 /usr/bin/xargs
2 /usr/bin/yes
2 /usr/sbin/chroot
It looks like only a couple of binaries got patched during the build process (and maybe some previously non-existing added), so I think, the proper solution would be to overwrite duplicate entries on extracting cpio
archives.
I would also limit the solution to the cpio
extractor, and would not make general behavior change in FileSystem
.
Describe the bug During extraction of at least 29 NETGEAR firmware images, unblob may try creating the same output file twice triggering an exception. As a result, some files that should be extracted are not.
To Reproduce Steps to reproduce the behavior:
wget https://www.downloads.netgear.com/files/GDC/M4100/M4100-V10.0.2.20.zip
unblob -v M4100-V10.0.2.20.zip
Expected behavior This error should not be raised, instead additional files should be extracted. I made a simpel change in file_utils.py's carve method (see below) to return early if the target file already exists and with this change an extra 75 files are created in
[extract_dir]/m4100v10.0.2.20.stk_extract/1201148-2097967.lzma_extract/lzma.uncompressed_extract
. I doubt this is the right fix, but it shows that this bug prevents some files from being extracted.Environment information:
Additional context I found this bug while doing some large-scale evaluations of filesystems produced by binwalk and unblob using fw2tar.
My (likely-incorrect) patch that results in additional files being created:
After fixing this, I got another error along the same vein in file_utils which I patched with: