Unchanged firmware output size doesn't match original size and bricks device

[GoogleCodeExporter]

What steps will reproduce the problem?

1. sudo ./extract-firmware.sh wndr3700v4-webflash.bin 
#DD-WRT r21676
2. sudo ./build-firmware.sh
#Have tried with -nopad and -min 

What is the expected output? What do you see instead?

I've extracted the firmware, changed nothing, and rebuilt it.
Original file size: 19058716
Current file size:  19062812 (plus footer of 0 bytes)
This should be the same size if nothing has changed. Other than that I see no 
error messages. If I comment out the size check and let it build normally, 
everything seems to build fine. Trying to flash this on a Netgear WNDR3700v4 
running DD-WRT bricks it; have to restore using tftp to stock.

What version of the product are you using? On what operating system?

firmware-mod-kit v0.99 on Ubuntu 12.10 Kernel 3.2.0-29

Please provide any additional information below.
I've modified firmware on a variety of Linksys/Cisco routers with no issues 
using this exact same build environment. 

Original issue reported on code.google.com by mrnova...@gmail.com on 11 Jun 2013 at 9:46

[GoogleCodeExporter]

Scanning firmware...

Scan Time:     2013-06-11 17:31:22
Signatures:    193
Target File:   /media/WININSTALL/04-15-2013-r21286/wndr3700v4-webflash.bin
MD5 Checksum:  f359cca221435e6df54ab87ec7bbad15

DECIMAL     HEX         DESCRIPTION
--------------------------------------------------------------------------------
-----------------------
0           0x0         TRX firmware header, little endian, header size: 28 
bytes,  image size: 19058716 bytes, CRC32: 0xFE96801 flags/version: 0x10001
92          0x5C        LZMA compressed data, properties: 0x6D, dictionary size: 
33554432 bytes, uncompressed size: 3410832 bytes
1310748     0x14001C    Squashfs filesystem, big endian, DD-WRT signature, 
version 3.0, size: 17745137 bytes,  2104 inodes, blocksize: 131072 bytes, 
created: Mon Apr 15 05:09:29 2013 

Extracting 1310748 bytes of trx header image at offset 0
Extracting squashfs file system at offset 1310748
Extracting squashfs files...
Firmware extraction successful!

@ubuntu:~/firmware_mod_kit/new$ sudo ./build-firmware.sh -nopad
Firmware Mod Kit (build) 0.99, (c)2011-2013 Craig Heffner, Jeremy Collake

Preparing tools ...
Building new squashfs file system... (this may take several minutes!)
Squahfs block size is 128 Kb
Creating big endian 3.0 filesystem on 
/home/firmware_mod_kit/new/fmk/new-filesystem.squashfs, block size 131072.

Big endian filesystem, data block size 131072, compressed data, compressed 
metadata, compressed fragments
Filesystem size 17332.40 Kbytes (16.93 Mbytes)
    29.10% of uncompressed filesystem size (59556.64 Kbytes)
Inode table size 16827 bytes (16.43 Kbytes)
    24.90% of uncompressed inode table size (67572 bytes)
Directory table size 18306 bytes (17.88 Kbytes)
    51.23% of uncompressed directory table size (35735 bytes)
Number of duplicate files found 27
Number of inodes 2104
Number of files 1547
Number of fragments 156
Number of symbolic links  309
Number of device nodes 0
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 248
Number of uids 1
    root (0)
Number of gids 0
ERROR: New firmware image will be larger than original image!
       Building firmware images larger than the original can brick your device!
       Try re-running with the -min option, or remove any unnecessary files.
       REFUSING to create new firmware image.

       Original file size: 19058716
       Current file size:  19062812 (plus footer of 0 bytes)

       Quitting...

Original comment by mrnova...@gmail.com on 11 Jun 2013 at 9:48

[GoogleCodeExporter]

DD-WRT uses a modified squashfs that brute forces the best LZMA compression 
parameters. We have the same modified copy. In fact, I was the author of this 
original modification to squashfs. However, doing this is slow and nets only a 
marginal benefit for most data sets. So, I disabled this brute force search for 
the best LZMA compression parameters. That accounts for the very slight size 
difference of ~4KB on 18.5MB! 

The cause of the brick is probably a different issue than this slight size 
difference. Someone will have to look closer into this to try to surmise the 
issue.

Original comment by jeremy.collake@gmail.com on 12 Jun 2013 at 4:50

[GoogleCodeExporter]

There are a couple of issues that could arise from flashing an image larger 
than the original. Without knowing the flash layout, doing so may be dangerous 
and is why the build script refuses to continue if the new image is larger than 
the original. 

However, since you took out the size checks, my first guess would be that the 
size reported in the TRX header is not correct. IIRC, the size field is not 
updated in the TRX header, only the checksum; so while the checksum was correct 
and the firmware upgrade file was accepted, the header reported that the file 
was smaller than it actually was. This could cause problems.

Original comment by heffne...@gmail.com on 12 Jun 2013 at 6:27

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Today I tried removing some files (non essential) to create a smaller image. 
Successfully created a new image that was smaller than original, but router is 
still bricking so there must be some deeper incompatibility with the WNDR3700v4.

Original comment by mrnova...@gmail.com on 12 Jun 2013 at 8:41

[GoogleCodeExporter]

I have done the same as "mrnova" with 3 TP-Link WR740v4, all with smaller 
firmware after removing some unused files: all 3 TP-Link where bricked with all 
led's blinking after firmware flash.
Something has to be wrong on building the new firmware.

Original comment by michaelf...@gmail.com on 22 Jun 2013 at 7:47

[GoogleCodeExporter]

same issue on openwrt for rspro. 
I can extract properly the firmware, but I didn't succeed to rebuild it. I 
didn't change anything but when I try to build it again, I get the error on 
different size.

What can I try to understand the cause?

Original comment by loffym...@gmail.com on 12 Oct 2013 at 6:01

[GoogleCodeExporter]

 May be it can help to understand better what is happening.
 From extract.log:

 untrx 0.54 beta - (c)2006-2010 Jeremy Collake
 Opening openwrt-ar71xx-generic-ubnt-rspro-squashfs-factory.bin
 read 3539356 bytes
 ERROR trx header not found
 splitter3 0.10 beta - (c)2010 Jeremy Collake
 Opening openwrt-ar71xx-generic-ubnt-rspro-squashfs-factory.bin
 read 3539356 bytes
 SQUASHFS magic: 0x73717368
 SQUASHFS version: 4.0
 Found segment type 0x8 Kernel length is f0184
 File system length is 26fe7c
 Trailer is 19c bytes
  Writing fmk//image_parts/vmlinuz
    size 983428 from offset 0 ...
 SQUASHFS magic: 0x73717368
 SQUASHFS version: 4.0
  ! WARNING: Unknown squashfs version.
  Writing fmk//image_parts/squashfs-lzma-image-x_x
    size 2555516 from offset 983428 ...
  Writing fmk//image_parts/hwid.txt
    size 412 from offset 3538944 ...
  Done!

Original comment by loffym...@gmail.com on 12 Oct 2013 at 9:05

[GoogleCodeExporter]

If you need the original image to reproduce the problem please feel free to ask 
me 

Original comment by loffym...@gmail.com on 12 Oct 2013 at 9:39