search

missinglink / pbf

utilities for parsing OpenStreetMap PBF files and extracting geographic data
MIT License
21 stars 9 forks source link

xroad: memory optimizations #28

Closed missinglink closed 3 years ago

missinglink commented 3 years ago

DRAFT: refactor to curtail the rampant memory usage

missinglink commented 3 years ago

This PR reduces maximum memory usage for an NY extract from ~2.8GB to ~1GB.

missinglink commented 3 years ago

I tested this a bunch, it seems to work great, I ran it against california and it used 1/3 the RAM and ran 10% faster