ERROR: New firmware image will be larger than original image! and cramfsswap problem

[GoogleCodeExporter]

What steps will reproduce the problem?
1. I try to make changes in firmware 
tomato-E4200USB-NVRAM60K-1.28.RT-N-5xRAF-EN-MIPSR2-079V-AIO.bin for Asus 
RT-N16. With the imported changes or without them, I receive identical errors: 
   If I use - ./extract_firmware.sh tomato-E4200USB-NVRAM60K-1.28.RT-N-5xRAF-EN-MIPSR2-079V-AIO.bin fmk - that I receive "make [1]: *** [cramfsswap] Error 1"
 With - ./extract-ng.sh tomato-E4200USB-NVRAM60K-1.28.RT-N-5xRAF-EN-MIPSR2-079V-AIO.bin -  like without errors.
From the terminal:
a. root@ubuntu:~/firmware-mod-kit-read-only/trunk# ./extract_firmware.sh 
tomato-E4200USB-NVRAM60K-1.28.RT-N-5xRAF-EN-MIPSR2-079V-AIO.bin

 Firmware Mod Kit (extract) v0.71 beta, (c)2010-2011 Jeremy Collake, - Newer NG edition by Craig Heffner
 Incorrect usage.
 USAGE: ./extract_firmware.sh FIRMWARE_IMAGE.BIN WORKING_DIR
root@ubuntu:~/firmware-mod-kit-read-only/trunk# ./extract_firmware.sh 
tomato-E4200USB-NVRAM60K-1.28.RT-N-5xRAF-EN-MIPSR2-079V-AIO.bin fmk

 Firmware Mod Kit (extract) v0.71 beta, (c)2010-2011 Jeremy Collake, - Newer NG edition by Craig Heffner
 Checking for updates ...
  You have the latest version of this kit.
 LINUX system detected. Compatibility ok.
 Testing file system of fmk ...
 Building tools ...
 ERROR - Build did not succeed. Check log file, you are probably
         missing one or more necessary pre-requisites. See docs for
         a list of pre-requsites.
 CONTINUING anyway, attempting to use pre-built binaries (x32 linux)
 Preparing working directory ...
 Removing any previous files ...
 Creating directories ...
 Extracting firmware
 Attempting squashfs 3.0 lzma ...
 Firmware appears extracted correctly!
 Now make changes and run build_firmware.sh.
b.root@ubuntu:~/firmware-mod-kit-read-only/trunk# ./extract-ng.sh 
tomato-E4200USB-NVRAM60K-1.28.RT-N-5xRAF-EN-MIPSR2-079V-AIO.bin
Firmware Mod Kit (build-ng) 0.71 beta, (c)2011 Craig Heffner, Jeremy Collake
http://www.bitsum.com

Scanning firmware...

DECIMAL     HEX         DESCRIPTION
--------------------------------------------------------------------------------
-----------------------
16          0x1         
16          0x1         MSD

Extracting 945184 bytes of  header image at offset 0
Extracting squashfs file system at offset 945184
Extracting squashfs files...
Firmware extraction successful!
Firmware parts can be found in 'fmk/*'

2. When I rebuild, anyway, there are errors:
From the terminal:
a.root@ubuntu:~/firmware-mod-kit-read-only/trunk# ./build_firmware.sh fmk_new/ 
fmk/

 Firmware Mod Kit (extract) v0.71 beta, (c)2010-2011 Jeremy Collake, - Newer NG edition by Craig Heffner
 Checking for updates ...
  You have the latest version of this kit.
 LINUX system detected. Compatibility ok.
 Testing file system of fmk/ ...
./build_firmware.sh: 328: cannot create fmk//rootfs/etc/rebuild_info: Directory 
nonexistent
 Building tools ...
 ERROR - Build did not succeed. Check log file, you are probably
         missing one or more necessary pre-requisites. See docs for
         a list of pre-requsites.
 CONTINUING anyway, attempting to use pre-built binaries (x32 linux)
 Preparing output directory fmk_new/ ...
 Detected WRT squashfs-lzma style.
 Building squashfs-lzma filesystem ...
 Utilizing lzma standard variant ...
 Building base firmware image (generic) ...
 Building base firmware image (asus) ...
 Making fmk_new//custom_image_00001-wrtsl54gs.bin
 Making fmk_new//custom_image_00001-wrt54g.bin
 Making fmk_new//custom_image_00001-wrt54gs.bin
 Making fmk_new//custom_image_00001-wrt54gsv4.bin
 Making fmk_new//custom_image_00001-generic.bin
 Firmware images built.
итого 49264
-rw-r--r-- 1 root root 8404992 2011-10-29 10:54 custom_image_00001-asus.trx
lrwxrwxrwx 1 root root      22 2011-10-29 10:54 custom_image_00001-generic.bin 
-> custom_image_00001.trx
-rw-r--r-- 1 root root 8404992 2011-10-29 10:54 custom_image_00001.trx
-rw-r--r-- 1 root root 8406016 2011-10-29 10:54 custom_image_00001-wrt54g.bin
-rw-r--r-- 1 root root 8406016 2011-10-29 10:54 custom_image_00001-wrt54gs.bin
-rw-r--r-- 1 root root 8406016 2011-10-29 10:54 custom_image_00001-wrt54gsv4.bin
-rw-r--r-- 1 root root 8406016 2011-10-29 10:54 custom_image_00001-wrtsl54gs.bin
 All done!
b.root@ubuntu:~/firmware-mod-kit-read-only/trunk# ./build-ng.sh
Firmware Mod Kit (build-ng) 0.71 beta, (c)2011 Craig Heffner, Jeremy Collake
http://www.bitsum.com

Building new squashfs file system...
Creating little endian 3.0 filesystem on fmk/new-filesystem.squashfs, block 
size 65536.

Little endian filesystem, data block size 65536, compressed data, compressed 
metadata, compressed fragments
Filesystem size 7282.49 Kbytes (7.11 Mbytes)
    31.34% of uncompressed filesystem size (23238.36 Kbytes)
Inode table size 8380 bytes (8.18 Kbytes)
    23.72% of uncompressed inode table size (35328 bytes)
Directory table size 9431 bytes (9.21 Kbytes)
    55.14% of uncompressed directory table size (17105 bytes)
Number of duplicate files found 0
Number of inodes 1087
Number of files 843
Number of fragments 73
Number of symbolic links  182
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 62
Number of uids 1
    root (0)
Number of gids 0
ERROR: New firmware image will be larger than original image! This is not 
supported.
    Original file size: 8393728
    Current file size:  8404000
Quitting...
3. Tried to produce these operations without modifying original firmware - the 
same result.

From here question, whether it is possible to ignore these errors and to use 
modified firmware? Or it is all it is critical?

What is the expected output? What do you see instead?

What version of the product are you using? On what operating system?
Ubuntu 11.10 x64, Firmware Mod Kit (extract) v0.71 beta

Please provide any additional information below.

I am sorry for my English))).

Original issue reported on code.google.com by ogamys...@gmail.com on 29 Oct 2011 at 7:32

Attachments:

• extract.log
• build.log
• binwalk.log
• config.log

[GoogleCodeExporter]

extract-ng will not build firmware images that are larger than the original, as 
this could potentially cause problems when writing the firmware image to the 
device. The reason the image is larger, even when un-modified, is likely due to 
differences in compression optimizations in our mksquashfs vs tomato's 
mksquashfs.

You have two options:

1) Remove anything that you don't need from the Tomato file system in order to 
reduce the size.
2) Comment out the file size checks in the extract-ng script (it will 
*probably* be OK, but there are no guarantees!! Doing this could brick your 
router!!!)

I should probably add an option to extract-ng that allows you to continue even 
if the firmware image size is too large, accompanied by an appropriate warning 
about the dangers of doing so.

Original comment by heffne...@gmail.com on 29 Oct 2011 at 10:30

• Changed state: Accepted

[GoogleCodeExporter]

I'd be really careful about that.. people ignore warnings. I'd continue forcing 
this behavior, and explain it. Make over-riding it a chore that only 
semi-advanced users can do. I mean, it doesn't take much to modify the script. 
If they can't modify the script, then they shouldn't be allowed to build 
over-size images. It is just too risky. You don't want people whining about 
bricked routers. Just my two cents on this.

Original comment by jeremy.collake@gmail.com on 29 Oct 2011 at 10:39

[GoogleCodeExporter]

Good point Jeremy.

I put in a more descriptive error message describing why build-ng won't build 
firmware images larger than the original and suggesting simple mitigations 
(decreasing file system size).

Original comment by heffne...@gmail.com on 4 Nov 2011 at 3:18

• Changed state: Fixed

[GoogleCodeExporter]

Excellent work ;). Thanks. I apologize for not having done more lately myself, 
I have been engaged in a deadline to finish a new version of one of my 
products. Obviously I somehow find time to come and run my mouth though ;p

Original comment by jeremy.collake@gmail.com on 4 Nov 2011 at 4:22

[GoogleCodeExporter]

No apologies needed, I've got a big project at my day job too, at least 4 or 5 
personal projects I'm always trying to work on, plus a wife and kids, so I get 
not having free time to work on stuff. :)  It took me a week to make this 
change, which was three short sentences. :P

Original comment by heffne...@gmail.com on 4 Nov 2011 at 4:29

[GoogleCodeExporter]

Such is life .. I recently put off a chore for a month due to constant 
distractions. Took 5 minutes to do ;p.

Original comment by jeremy.collake@gmail.com on 4 Nov 2011 at 4:42

[GoogleCodeExporter]

It should really also delete the new-firmware.bin file as it exits. Because I 
saw the file existed, and the file size looked something like what it said the 
firmware would be, so I thought maybe this limitation had been removed, just 
with the textual warning. So now I bricked my device since the tool didn't 
write the final parts of the .bin firmware. (as I saw in the build-ng.sh file 
_after_ I was in a hurry to upload my modifications)

Original comment by haakon.n...@gmail.com on 17 Nov 2011 at 5:10

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Ignore previous comment, I forgot where were 'at' with this back and forth. 
Yes, I could have just read above ;p. 

Deletion of any existing firmware in the output folder is probably a good idea. 
Though, again, please do refer to the pre-requisites of use of the FMK being 
able to unbrick your device (something easily done most of the time).

Original comment by jeremy.collake@gmail.com on 17 Nov 2011 at 7:57

[GoogleCodeExporter]

So, accepted on the basis that we need additional protections for this scenario 
of user mis-use. Hey, I'm a user myself, as well as a coder, so I do the same 
type of mistakes. 

Original comment by jeremy.collake@gmail.com on 17 Nov 2011 at 7:58

• Changed state: Accepted

[GoogleCodeExporter]

Recommend firmware images be built, but renamed to something like 
'do-not-flash-me-unless-you-are-sure-i-am-going-to-fit-and-can-ubrick.bin', if 
you get my drift ;). Up to you. Close if you want to, this is really user error.

Original comment by jeremy.collake@gmail.com on 18 Nov 2011 at 6:21

[GoogleCodeExporter]

Changed to enhancement since was user error, and note that non-NG (my old crap) 
will not be updated.

Original comment by jeremy.collake@gmail.com on 18 Nov 2011 at 6:25

• Added labels: Type-Enhancement
• Removed labels: Type-Defect

[GoogleCodeExporter]

Original comment by jeremy.collake@gmail.com on 18 Nov 2011 at 6:25

• Added labels: Priority-Low
• Removed labels: Priority-Medium

[GoogleCodeExporter]

Updated build-ng to delete the firmware image if the file size is to large. 
Advanced users can always change the script to skip the file size check as 
before; non-advanced users probably shouldn't risk flashing a firmware image of 
the wrong size. 

Original comment by heffne...@gmail.com on 23 Nov 2011 at 12:15

• Changed state: Fixed

[GoogleCodeExporter]

Best solution I think. I mean, the script is really simple, and any user who is 
flashing firmwares should be able to modify if, if needed. That then puts the 
liability 100% on them.

Original comment by jeremy.collake@gmail.com on 23 Nov 2011 at 12:41