avrdude is slow

[sprintersb]

avrdude is slow when is flashes files that have pages that consist only of 0xff.

Such pages can be skipped which speeds up flashing. This applies independent of whether an erase is performed.

This happens when a program has several parts located far away from each other. For example, a program that hosts version information at the end of flash, or that hosts .rodata at the end of flash.

When the program (elf) has multiple program headers that have their LMA in flash, then it is enough the just flash these program headers. When ihex or similar is used, it should be easy enough to check for all 0xff's in a page.

[stefanrueger]

Which programmer and which parts does this issue talk about?

AVRDUDE knows some 160 programmers and some 330 parts. It is not always the case that writing flash pages with only 0xff is a NOP: for example, some bootloaders or programmers serving parts with U/PDI interfaces can, and do, deploy page erase before writing. To complicate things further, some bootloaders, eg, optiboot, ignore a chip erase command. When AVRDUDE cannot rely on CE, it has to send pages with 0xff out of necessity to ensure the flash is erased where the uploaded file wants it to be. Not doing so would risk leaving the, eg, previously flashed secret Coca Cola recipe exposed.

There is some optimisation for 0xff pages when AVRDUDE can be sure the respective programmer carries out a CE and the protocol/programmer allows this. It is likely that not every optimisation for every programmer is taken, and we encourage a PR when further optimisation is possible in specific cases.

[sprintersb]

Which programmer and which parts does this issue talk about?

It's about all programmers and all devices.

When a sequence of 0xff is written with programmer X, then it would also be written with programmer Y. No?

The only dependence on device is what the page size is, because only hole pages can be skipped, not partial pages.

Detection of pages with only 0xff's in them should be feasible, and detecting such pages will be must faster that writing them with no effect.

[sprintersb]

And erasing is carried out by an erase cycle, not by writing 0xff's, as far as I understand.

Writing 0xff's is void, no matter if an erase cycle is issued or not (without an erase cycle, you cannot set bits that were 0 previously, but that applies to all values, not just 0xff.

As I said, the becaviour is independent of whether an erase cycle is issued. If it's not issued, that writing 0xff's wont't set bits that are 0 bevore.

[stefanrueger]

Writing 0xff's is void, no matter if an erase cycle is issued or not

Fist of all, we are talking flash memory here not eeprom or other memories. And then, some programmers hide the physical nature of NOR-memory. Take, for example, -c arduino that serves an optiboot bootloader. This bootloader deploys an SPM page erase cycle before writing the page. In this case the flash memory does not look like NOR-flash to AVRDUDE. Sending an 0xff-page actually sets bits even if they were not set before. This is because optiboot has decided to use SPM page erase before it writes a page. Optiboot could have been programmed so that flash memory actually looks like NOR-flash, but it wasn't. It is [edit: can be] a security risk not to send 0xff-pages: Assume you have secret data in flash of a part with the optiboot bootloader. The user wants to overwrite the secrets. Optiboot ignores CE, so the only way to overwrite the secrets is to send 0xff-pages, not? And how would that work if AVRDUDE decided to drop all 0xff-pages?

There are parts that allow external page erase, eg, the XMEGAs or modern AVR parts (those with UPDI interface). Here the physical programmer can erase pages before writing a page. Again, depending on the protocol and execution, flash does not necessarily look like NOR memory to AVRDUDE as this issue wrongly assumes.

Summarising, not all flash memory looks like NOR-flash via all programmers. AVRDUDE knows a lot about individual programmers and has (quite a few) optimisations under its hood. The one suggested by this issue is not amongst them for the reasons outlined above.

If you find a specific programmer that can be optimised, kindly submit a PR

[sprintersb]

Ok, thank you very much for the explantions.

[mcuee]

@sprintersb

If you do not have a specific case, I will close this issue.

[sprintersb]

I have one final question though.

When the layout of the program is like A-FF-B where A is cose at the start of program memory, B is at the end, and FF is a stretch of FF's between, then you are saying that it's important to write the FF's.

But then the program is like A-FF with FF up to the end, then avrdude won't write the final FF's. So why is avrdude allowed to ignore the FF's in that case?