osm-without-borders / cosmogony

easy to use & easy to update geographic regions
http://cosmogony.world
Apache License 2.0
103 stars 15 forks source link

add a buffer arround input file #148

Closed remi-dupre closed 1 year ago

remi-dupre commented 1 year ago

We are doing unbuffered I/O while reading the input file, which can lead leads to very slow computation on some environments (in our case PVCs in a kube clusters). I didn't dig into specific details but I saw that osmpbfreader doesn't wrap the reader with a buffer and makes atomic reads which would lead to an I/O operation each (often just a u32).