search

floledermann / mapzr.com

mapzr.com - a "swiss army knife" for working with maps on the web.
http://mapzr.com
1 stars 0 forks source link

Improved compression (Huffman? UTF-8 - LZW? flatgeobuf?) #13

Open floledermann opened 4 years ago

floledermann commented 4 years ago

Huffman encoding may work better for our data than the current LZW, as there is a lot of redundancy. Some gains may be lost in the base64 encoding though.

floledermann commented 4 years ago

It turns out Huffman encoding is not terribly efficient due to the large symbol space of GeoJSON with human language. See https://observablehq.com/d/14c3f51be9437b4c

Furthermore, the LZ-String library we are currently using is hardcoded to UTF-16, using 2 byte per character! This should be moved to UTF-8 as soon as possible!

Maybe look into http://lzma-js.github.io/LZMA-JS/demos/advanced_demo.html

floledermann commented 4 years ago

Consider using flatgeobuf https://github.com/bjornharrtell/flatgeobuf , Base64 encoded