search

martinberlin / brotli

Brotli compression format forked to work in ESP32
https://fasani.de
MIT License
25 stars 1 forks source link

Compression Example on the ESP32 #1

Open vangalvin opened 4 years ago

vangalvin commented 4 years ago

Hi Martin, I was wondering if you by any chance had a compression example for the ESP32. I have quite a few sensors and would like to be able to reduce the size of the overall transmitted data if possible before it is sent to a remote web site and feed in to a javascript decompression routine then stored in a database.

Regards Andrew

martinberlin commented 4 years ago

I tried so far but still getting an exception in the looptask. I asked for support in Brotli main library but I guess first need to test the same in a C compiled file out of ESP32 environment to see if my input makes sense. Will update it here if I can get it to work

vangalvin commented 4 years ago

thanks @martinberlin :)

martinberlin commented 4 years ago

Try the feature/compress branch

Reading file '/sensors.json' from FS

349 bytes read into inBuffer from: /sensors.json Calling BrotliEncoderCompress Compression params: 1 quality 22 lgwin status: 1 2131 microseconds spend compressing 219 bytes after compression decompressBuffer() - - - - - - - - - - - - - - - - - - - - - - - - - -

status: 1 Decompress output: - - - - - - - - - - - - - - - - - - - - - - - - - - { "DS1": [-16.931, -16.931, -37.938, 29.703, 47.683, 100.362, 74.003, 107.58, -16.549, 70.011, 85.483, 118.478, -49.773, 54.442, 114.613, 99.228, 16.555, 43.453, 10.91, 33.173, 14.56, -54.855, 93.051, 109.894, 60.431, 88.935, 5.121, -46.954, 59.805, -29.47, 87.34, -54.071, 8.601, 104.404, 30.632, 45.635, 77.765, 93.987, 26.215, 50.569, -28.651] }

However, trying to increase quality will fail. Anyways it's great to have it working. Please contact me per E-Mail if you need further help with your implementation and feel free to close this if it's working for you.

vangalvin commented 4 years ago

Awesome, I will give that a go shortly and report on how it works.

martinberlin commented 4 years ago

Note if you want to go to the lowest transmission protocol, json is definitively not the way to go. But if you do not have any other alternative than json, reduce the floats to integers, and try to compress again. Also, the DS1 could be 1 character if you decide so. All little reductions here and there will help you to get a smaller size.

vangalvin commented 4 years ago

Thanks Martin, Yes I have changed DS1 to be a value of 01 (there will eventually be about 20 units in place) Also I am now using the raw value from the sensor (16 bit value) this did drop the size down and will be dumping the JSON formatting as I decided that signed 16bit's tailed on to each other can be read back and converted at the other end to the values and the json can be reconstructed there.