search

pfalcon / uzlib

Radically unbloated DEFLATE/zlib/gzip compression/decompression library. Can decompress any gzip/zlib data, and offers simplified compressor which produces gzip-compatible output, while requiring much less resources (and providing less compression ratio of course).
Other
303 stars 82 forks source link

using inflate for esp8285 OTA #22

Closed d-a-v closed 5 years ago

d-a-v commented 5 years ago

On esp8285 (or esp8266 with only 1M flash chips), OTA updates are limited to usually half of the flash size (download to free flash, then copy to beginning of flash).

esp8266 flash images compress to around 0.7 of the original size (with gzip -9)

By compressing, sending and storing a compressed flash image, one can expect to gain roughly 100KB of flash image size and this is huge (1/(1+0.7)-(1/2))*1M. Inflating would happen on-the-fly while flashing (of course, inflate code would have to stay in ram).

Without using any ram-dictionary, already inflated chunks need to be always accessible. Luckily this is compatible with this use-case.

However, uncompressed data are stored in 32bit-only accessible memory, so every read access to d->dest (already flashed, not in a temporary ram block), directly or through TINF_PUT must use a "32bits aligner" facility.

I already know official esptool.py is doing something similar, but it's using another deflate implementation that seems to require a 32KB ram block (which is always possible in that case), I find the source code less readable and I'm not sure it can do the job without a ram dictionary.

I'm not asking for help. Due to your knowledge of the chip and its limitations, I was just wondering what you would think of this use-case.

(edit: fortunately, first result of google's 'zlib esp8266' is a link to uzlib)

pfalcon commented 5 years ago

If you're as concerned with the need to waste half of flash for OTA as myself, you simple wouldn't spend half of it on OTA, just the same as I don't: https://github.com/pfalcon/yaota8266 .

If you want to go half-way between these solutions - reserving space for 2 active firmwares, and reserving for just 1 - you definitely can. I don't have specific suggestions. But you may want to look at https://github.com/pfalcon/uzlib/issues/13 as it mentions something OTA.

pfalcon commented 5 years ago

Without using any ram-dictionary, already inflated chunks need to be always accessible. Luckily this is compatible with this use-case.

Ah, and this seems to be what https://github.com/pfalcon/uzlib/pull/4 does. (Note that it's not suitable for mainline as is, and probably no longer applies to the latest uzlib.)

I'm closing this as won't be able to look into more details of this any time soon. Hopefully, the info provided is enough to give you ideas/decisions.

d-a-v commented 5 years ago

Well thanks for those pointers! I now have to decide where to go