alexey-zakharenkov / subways

Subway Preprocessor and Validator
Apache License 2.0
12 stars 1 forks source link

Optimize validation procedure #1

Closed alexey-zakharenkov closed 3 years ago

alexey-zakharenkov commented 3 years ago

This PR optimizes a planet preparation for subway validation, enhancing time/ram/storage metrics.

Main improvement is to use a planet extract containing only cities with metro. Compared to the previous version, it now takes 65 minutes vs 120 to perform a complete validation in some computational environment, and 25 minutes vs 120 if to make use of a file produced during previous validation run. This intermediate file ought to be re-generated (leading to 65-minutes-long execution) only when a new metro network has been added or an existing network BBOX has been extended, i.e. once in 2-4 weeks. In between such metro extension events, the validation would require 22 Gb vs current 185 Gb of disk space.