Greenheart / lifewheel

Reflect on Your Life Balance. Offline-first, privacy-friendly web app for your personal well-being.
https://reconnect.earth/lifewheel
GNU Affero General Public License v3.0
11 stars 1 forks source link

Improve data compression #2

Closed Greenheart closed 11 months ago

Greenheart commented 1 year ago

First, test if https://github.com/nodeca/pako can compress data more efficiently than we can do ourselves. Also experiment with various strategies, maybe specifying a dictionary and finally try enabling/disabling the header to use the raw mode which uses slightly less data.

Or maybe we should use a custom solution

Maybe we get better results with a custom solution.

The biggest advantage is that we don't need any external dependency, and the solution will keep working in the future.

However, this is likely more work and will require manual updates if we change the shape of the data.


The binary data used to encode links currently use one byte even for small numbers. This is quite inefficient.

Take for example the LifewheelState which is represented by 8 numbers between 1 to 10.

Since we currently only need to store 10 distinct values for each position in the LifewheelState, we need 4 bits to represent 10 as binary (1010).

Since we only need 4 bits to represent each value, this means we can compress the data without any data loss by storing two values in the same byte (8 bits). Especially for links, this will save important characters, which ultimately helps keeping the links a bit shorter.

Proposed solution

Store

Code use for experiment:

function dec2bin(dec) {
  return (dec >>> 0).toString(2);
}

function bin2dec(bin) {
  return parseInt(bin, 2)
}

dec2bin(10) // returns '1010'
bin2dec('1010') // returns 10
Greenheart commented 11 months ago

Now resolved, using a combination of pako (zlib) compression and encoding the reflection data slightly more efficiently.