search

google / brotli

Brotli compression format
MIT License
13.62k stars 1.24k forks source link

using brotli compression on embedded iot with limited RAM #1018

Open ziv-augury opened 1 year ago

ziv-augury commented 1 year ago

hello brotli i want to use the brotli compression which gave good compression rates on the type of data i need to compress (tested with the cli tool) . but before digging on how to use brotli compression in my (zephyr OS based) application i want to make sure its possible. in total the RAM of my device is 64k (ARM cortex M4 64Mhz), so with all the rest of the application i would say that max footprint can be ~4k, preferably smaller (including buffer for input by chunks of 256/512 bytes max). i do not care so much for speedy compression but rather have compress rate high as possible.

can brotli work in this conditions ? are the windows and other allocation sizes configurable ? i read that it uses a dictionary - what size of dictionary will it use for ~32kbyte of data ?

hope to read you soon best regards Ziv

eustas commented 1 year ago

Currently brotli encoder won't fit such tight requirements. I'd recommend exploring compress_fragment{|_two_pass}.{c|h} it is quite possible making a standalone encoder on the base of those and then continue tuning it to fit the restrictions...