python-intelhex / intelhex

Python IntelHex library
BSD 3-Clause "New" or "Revised" License
201 stars 107 forks source link

Add option/parameters to strip padding from the end of binary data. #64

Closed fpagliughi closed 9 months ago

fpagliughi commented 10 months ago

Sometimes when converting iHex -> binary and you specify a range, it is not because you want to fill that range, but rather because you want to limit the output to (at most) that range.

For example, a PIC18F might have 32kB of Flash, but has a few bytes of config memory that loads at address 3M. So when you create a .bin file for it, the 32kB part winds up with a 3MB binary! And, if you're loading the binary through a bootloader, you don't need the config memory anyway.

So, you might limit the range to 0x800:0x7FFF (above the bootloader to the end of Flash). But if the app is only 4kB, then >80% of the binary is padding, and sending it to the bootloader taks considerably longer than necessary.

This PR adds a strip option to the functions that output to binary, that can tell them to strip off any padding at the end of the binary array. It also adds a --strip option to the hex2bin.py script to pass that option along.

This probably isn't an optimal solution, and I assume there's other ways to accomplish the same goal, but this update fit our needs.

bialix commented 10 months ago

Hello, I understand your use-case but I don't like the parameter name --strip. I'd rather have parameter --filter START:END to initially cut desired part of hex file and then convert that part into binary. Similar to what --range now does, but without mandatory padding at the end of the file. It would be much easier to implement.

fpagliughi commented 10 months ago

OK. I’ll see if I can figure that out.

fpagliughi commented 9 months ago

OK. I got it. Changed the nomenclature from 'strip' to 'filter', used like:

hex2bin.py -f 0x800:0x7FFF  production.hex

When specifying the filter option, it resets the end of the range to the max address in that range, thus limiting the end of the range to prevent padding from being added.

To help with this, I added a few member functions to get the min, max, or list of addresses within a range. I can see these as being generally useful to people who may want to examine what is being populated within a specific memory/Flash area.

I'm not great at Python, so please feel free to change or optimize this in any way to improve the implementation.

bialix commented 9 months ago

Well, you did too much changes. As I said it would be much easier to filter the content. There is already support for such operation in the core IntelHex using slice operation:

ih = IntelHex("file.hex")
ih2 = ih[start:end+1]   # filtering
ih2.tobinfile("file.bin")

I see your confusion: the API of hex2bin() function is not very nice - that's really problem in existing code. I'll try to code up something later.

bialix commented 9 months ago

And probably we should add explanation of slice operations to manual.

fpagliughi commented 9 months ago

Wow... I had not idea you could do that; and I didn't believe it would work until I actually tried it!

OK, I think I understand what you mean. This can just be a few lines in the hex2bin function? No need to go any deeper in the library? I can make one more quick attempt, and then if that's still not right, then I will give up. :-)

fpagliughi commented 9 months ago

It was easier to just start over with a new branch and PR. I will close this now. See #65