android-sparse: truncated image generated

[niziak]

Scenario

Hi. I'm using NXP i.MX8MM. It is flashed using uuu PC software and U-Boot in fastboot mode on target. To speedup flashing process I've decided to use Android sparse format (which is natively supported by U-Boot) and it works when:

• uuu generate sparse format for u-boot on fly from raw sdcard image.
• I've created sparse image externally using Android Tools img2simg

but flashing stops near end of process when I'm using genimage's android-sparse image. Also it is not possible to convert generated by genimage sparse file using Andorid Tools simg2img. Dumping content of genimage's sparse file shows missing blocks:

./simg_dump.py -v sdcard.sparse 
sdcard.sparse: Total of 91137 4096-byte output blocks in 37 input chunks.
            input_bytes      output_blocks
chunk    offset     number  offset  number
   1         40     270336       0      66 Raw data           
   2     270388          4      66       1 Fill with 0x00000000 
...
  35     434652          0   55298   10222 Don't care         
  36     434664          4   65520      17 Fill with 0x00000000 
  37     434680          4   65537       0 Unverified CRC32 0xAE9D394D 
         434684              65537         End
The header said we should have 91137 output blocks, but we saw 65537

Investigation summary

There is a bug in genimage, when last FS extent ends earlier than real file size. Block generation loop depends only on extent map not real file size.

Reproduction

• create empty sparse file tuncate -s 100M empty.sparse
• and append it to end of image:

image empty.ext4 {
  ext4 {
    label = "factory"
    use-mke2fs = "true"
  }
  size = "256M"
  empty = "true"
  temporary = "true"
}

image sdcard.img {
  hdimage {
    fill = "true"
  }
  partition rootfs2 {
    partition-type = 0x83
    image = "empty.ext4"
    size = 256M
  }

  partition emptysparse {
    partition-type = 0x83
    image = "empty.sparse"
  }
}

image sdcard.sparse {
    android-sparse {
        image = sdcard.img
    }
}

[niziak]

U-Boot still complains, even with fixed genimage. U-Boot message returned to uuu:

20%Bogus chunk size for chunk type Dont Care1:223>Fail Bogus chunk size for chunk type Dont Care(23.31s)

There 2nd problem with U-Boot. Message above is generated in U-Boot's CRC32 chunk handler. Error message is misleading. U-Boot incorrectly handle CRC32 chunk size check: https://github.com/u-boot/u-boot/blob/master/lib/image-sparse.c#L291