search

martinberlin / brotli

Brotli compression format forked to work in ESP32
https://fasani.de
MIT License
25 stars 1 forks source link

Esp32 compress file on sdcard #3

Closed bhupiister closed 2 years ago

bhupiister commented 3 years ago

Can this be used to compress binary file kept on sdcard? I am using platformio

bhupiister commented 3 years ago

@martinberlin : I need to compress bin file on esp32 SD card. I am facing trouble selecting Buffer size. `4521 bytes read into inBuffer from: /sample Calling BrotliEncoderCompress Compression params: 1 quality 22 lgwin status: 0 105 microseconds spend compressing 30000 bytes after compression decompressBuffer() - - - - - - - - - - - - - - - - - - - - - - - - - -

status: 0 Decompress output: - - - - - - - - - - - - - - - - - - - - - - - - - -`

And If it gets compressed successfully, do you have any function to convert it into some file again like with extension of .gzip or anything else? or i need to do that by my own reading the compressed buffer?

martinberlin commented 3 years ago

Can this be used to compress binary file kept on sdcard? I am using platformio

I think yes. But depending on how big is the buffer size you may find limits and I never tried to use this with SPRAM.

And If it gets compressed successfully, do you have any function to convert it into some file again like with extension of .gzip or anything else?

GZIP? No no this is brotli. If you want another compression method try MiniZ that is the standard compression in ESP32 when you upload your program and used by the flasher stub. Check also my compression examples here in this repo. You can compress and decompress with Brotli.

bhupiister commented 3 years ago

@martinberlin Sorry to mention gzip extension, that was just for example sake and I intent to compress and decompress with same library. What I wanted to ask is that in your data folder you have files of two types normal and compressed ones. Did you compress and decompress them and on esp32? And by this I mean, after compressing, did you create files with compressed extension on esp32 itself?

martinberlin commented 3 years ago

Most of the examples use decompression. I use miniz on the command line on linux, you can find the file here: https://github.com/martinberlin/udpx/tree/master/examples

But there is a compression example and this has been used by other people. Note that I just refactored the header files in order to get this library compiling on the ESP32. But it's not my creation at all, it was done by Google team, so all I can suggest is to keep on trying and to check other issues where we deal with compression.

Also try miniz as an alternative to compress /decompress since it should be more stable and it's the de-facto standard library to do this in the ESP32. It compresses less than Brotli, but is more stable and it proved to be a better choice than Brotli in some of my projects.