search

tkrajina / srtm.py

Geo elevation data parser for "The Shuttle Radar Topography Mission" data
Apache License 2.0
240 stars 57 forks source link

Add batch_mode option to get_file to mitigate memory issues caused by local caching of files #33

Closed jameslamb closed 6 years ago

jameslamb commented 7 years ago

I love this module and find it incredibly useful. I also understand that local caching of the pulled files makes it very fast when you are making many queries for data in the same geographic area.

When using this module to label a large, global dataset (making millions of requests), I found that I was running into memory issues because of the diversity of files needed to satisfy my requests. In this PR, I propose a small change to address this. With this change, users who are running large ETL jobs can reliably use this module without worrying about running out of memory.

jxieeducation commented 7 years ago

Wow, this is incredibly useful. Going to test out the patch tonight.

tkrajina commented 6 years ago

@jameslamb Merged, thanks.