use bmap files to skip flashing unused blocks

[dnbazhenov]

Use .bmap files created by WIC tool the same way as bmaptool does, i.e. skip flashing irrelevant blocks. This is a -raw2sparce extension.

By default, a file with the same base name but with "bmap" extension would be searched to get block map from. An option '-no-bmap' would inhibit the search. Also, a specific bmap file can be specified in the flash command via '-bmap' option.

[nxpfrankli]

Can you fixed CI build error?

/home/runner/work/mfgtools/mfgtools/libuuu/bmap.cpp: In function ‘bool parse_image_size(bmap_t&, const tinyxml2::XMLElement)’: /home/runner/work/mfgtools/mfgtools/libuuu/bmap.cpp:42:24: error: ‘const class tinyxml2::XMLElement’ has no member named ‘Unsigned64Text’; did you mean ‘UnsignedText’? 42 | auto img_size = elem->Unsigned64Text(); | ^~~~~~ | UnsignedText /home/runner/work/mfgtools/mfgtools/libuuu/bmap.cpp: In function ‘bool parse_block_size(bmap_t&, const tinyxml2::XMLElement)’: /home/runner/work/mfgtools/mfgtools/libuuu/bmap.cpp:53:24: error: ‘const class tinyxml2::XMLElement’ has no member named ‘Unsigned64Text’; did you mean ‘UnsignedText’? 53 | auto blk_size = elem->Unsigned64Text(); | ^~~~~~ | UnsignedText /home/runner/work/mfgtools/mfgtools/libuuu/bmap.cpp: In function ‘bool parse_blocks_count(bmap_t&, const tinyxml2::XMLElement*)’: /home/runner/work/mfgtools/mfgtools/libuuu/bmap.cpp:64:25: error: ‘const class tinyxml2::XMLElement’ has no member named ‘Unsigned64Text’; did you mean ‘UnsignedText’? 64 | auto blk_count = elem->Unsigned64Text();

[ondrejhennel]

Hi. I tried to use this PR as we have a very large image with a lot of empty space.

I built it on linux (wsl) first. At first it did not work with either the builtin scripts or a custom one. This is because the version number generated from tags was incorrect. After removing the tags "uuu_db_bmap" and db_bmap", it worked correctly and sped up the flash significantly.

Then, I tried to build for Windows. Same issue with tags. It also couldn't find tinyxml2.h even with the submodules checked out and I had to install tinyxml2 using vcpkg. After this, the build suceeded, but the binary did not work correctly. The flashing finished after a few seconds (way too fast) and the image in flash was corrupt and could not boot.

There seems to be an issue that occurs only on Windows and not on Linux.

[nxpfrankli]

@ondrejhennel Thank you for your testing this PR.

[mothacehe]

On Linux, with the bmap file available, with the same name as the image, it works well and the flashing time is considerably reduced (thanks!).

However, if modify the flashing script this way:

# @_image   [_flash.bin] [_bmap] | wic image burn to emmc.
...
FB: flash -raw2sparse -bmap _bmap all _image

I see:

3:644>Start Cmd:FB: flash -raw2sparse -bmap /home/mathieu/tmp/s0-image-imx93-var-som.wic.bmap all /home/mathieu/tmp/s0-image-imx93-var-som.wic.zst/*

which is expected, but then it fails:

3:644 4/ 6 [partition does not exist ] FB: flash -raw2sparse -bmap /home/mathieu/tmp/s0-image-imx93-var-som.wic.bmap all /home/mathieu/tmp/s0-image-imx93-var-som.wic.zst/*

[dnbazhenov]

@nxpfrankli @ondrejhennel I can't deduce how the tags thing should work. The build is not working without a tag.

[ondrejhennel]

@nxpfrankli @ondrejhennel I can't deduce how the tags thing should work. The build is not working without a tag.

During the build, git describe is used to generate a version number from the latest tag. The build will be successful if there is any tag. However, the version number is used to check compatibility with the scripts. If there is a tag like uuu_db_bmap, uuu will think it is incompatible with the builtin scripts and fail to run.

I think the easiest solution to make the build work is to just use the tags that are in the original repo and not add any others.

[dnbazhenov]

@nxpfrankli I have fixed CI issues @ondrejhennel @mothacehe I fixed the found issues. Please, give it another try. Also, note that I restored the default behavior in order to not surprise badly existing automation. Use either -bmap global option or -bmap binding to the flash command.

[ondrejhennel]

The flashing with bmap now works correctly on Windows. However, there is some strange behavior with the command-line switches.

Here are some examples:

> uuu -b emmc_all .\imx-boot .\image.wic
uuu (Universal Update Utility) for nxp imx chips -- libuuu_1.5.125-15-g5283d01

Wait for Known USB Device Appear...
Error: Missed file name

-----------------------------------------------------------------------------------------------

> uuu -no-bmap -b emmc_all .\imx-boot .\image.wic
uuu (Universal Update Utility) for nxp imx chips -- libuuu_1.5.125-15-g5283d01

Wait for Known USB Device Appear...
Error: Missed file name

-----------------------------------------------------------------------------------------------

> uuu -bmap .\image.bmap -b emmc_all .\imx-boot .\image.wic
uuu (Universal Update Utility) for nxp imx chips -- libuuu_1.5.125-15-g5283d01

Success 0    Failure 0

2:3      1/ 0 [                                      ]
# gets stuck like this forever

The flags only work correctly in a custom script in a command such as FB: flash -raw2sparse -bmap image.bmap all image.wic or FB: flash -raw2sparse -no-bmap all image.wic. When no flag is specified (FB: flash -raw2sparse all image.wic), Error: Missed file name occurs.

[dnbazhenov]

@ondrejhennel Thanks for reporting. I found where I screwed with the command line parsing. Could you try with the fix? I locally verified so it should work as expected.

[ondrejhennel]

When using the -bmap command line flag, it still gets stuck like this.

> uuu -bmap .\image.bmap -b emmc_all .\imx-boot .\image.wic
uuu (Universal Update Utility) for nxp imx chips -- libuuu_1.5.125-15-g5283d01

Success 0    Failure 0

2:3      1/ 0 [                                      ]
# gets stuck like this forever

Other than that, it works correctly now.

[dnbazhenov]

When using the -bmap command line flag, it still gets stuck like this.

> uuu -bmap .\image.bmap -b emmc_all .\imx-boot .\image.wic
# gets stuck like this forever

This is an incorrect usage. The global -bmap option doesn't take an an argument. I'm not sure thath the stuck is related to the command handler itself.

[marcusfolkesson]

I've tested this PR locally with success.