Closed okdistribute closed 3 years ago
I think the o5m is also causing a huge inflation in the output file size when I run the ingest on a small metro extract (I stopped it after it wrote a gigabyte for a 7M file). My guess is that the persistent o5m string table is getting written every time but that only makes sense when there are many o5m records.
This has been addressed in the leveldb code which uses a compact binary encoder and decoder to store intermediate representations of osm features.
swap out o5m for json
maybe we could gzip the json at least. if we can use the browser's built-in gzip decompressor that would be ideal but otherwise we can probably use a wasm un-gzip