Add optimisation function: optimise_tile, called with a string of bytes, will decode it, optimise it and re-encode it. The optimisations are:
String table re-ordering. Features in a layer reference their tags by key and value indices, which are encoded as variable length integers. These are generally assigned on a first-seen basis, but this can lead to commonly-used keys or values having larger indices, which can take up more space in the tile. Since the order of items in the string table doesn't matter, we can optimise it and recover a few bytes from the tile.
Multilinestring re-ordering. Multilinestrings are encoded as a sequence of move to and line to commands. The move to commands separate one linestring from another, and encode the relative position of the start of the new linestring relative to the end of the old one. Since the order of linestrings in a multilinestring isn't significant, we can potentially get a smaller file by choosing the closest linestring as the next one.
Both of these together reduce tile size by a few percent (I've seen 1-4% in my tests).
NOTE: I'm not sure what's up with the test failure. It seems to fail when run as python setup.py test, but not when the tests are run explicitly (e.g: for i in tests/test_*.py; do j=`echo $i | sed "s,\.py,,;s,/,\.,"`; python -m unittest $j; done). As far as I can tell, it seems to be something to do with unittest "manually" loading modules and ending up loading something from protobuf twice, which it doesn't like. Perhaps related to https://github.com/protocolbuffers/protobuf/issues/3276. In any case, seems like an upstream bug, and I haven't figured out a work-around other than running the tests individually.
Add optimisation function:
optimise_tile, called with a string of bytes, will decode it, optimise it and re-encode it. The optimisations are:move toandline tocommands. Themove tocommands separate one linestring from another, and encode the relative position of the start of the new linestring relative to the end of the old one. Since the order of linestrings in a multilinestring isn't significant, we can potentially get a smaller file by choosing the closest linestring as the next one.Both of these together reduce tile size by a few percent (I've seen 1-4% in my tests).
NOTE: I'm not sure what's up with the test failure. It seems to fail when run as
python setup.py test, but not when the tests are run explicitly (e.g:for i in tests/test_*.py; do j=`echo $i | sed "s,\.py,,;s,/,\.,"`; python -m unittest $j; done). As far as I can tell, it seems to be something to do with unittest "manually" loading modules and ending up loading something from protobuf twice, which it doesn't like. Perhaps related to https://github.com/protocolbuffers/protobuf/issues/3276. In any case, seems like an upstream bug, and I haven't figured out a work-around other than running the tests individually.