search

pysal / tobler

Spatial interpolation, Dasymetric Mapping, & Change of Support
https://pysal.org/tobler
BSD 3-Clause "New" or "Revised" License
146 stars 30 forks source link

handle remote rasters #49

Closed knaaptime closed 4 years ago

knaaptime commented 4 years ago

ideally users should be able to stream the NLCD data without downloading all 3+gb. rasterio can read remote files from s3, but requires a context manager to get around AWS authentication

with rasterio.env.Env(aws_unsigned=True):
    raster = rasterio.open(path)

we can use that scheme when we read files directly, but currently rasterstats is unable to inherit that context. I'm opening this here so we can figure out how best to handle

knaaptime commented 4 years ago

it took me forever to figure out this error was coming from rasterstats instead of rasterio

kannes commented 4 years ago

As a user I would feel weird (and it might be a privacy issue) if requests were silently done to remote servers when I am doing my analysis. I would rather expect myself needing to supply the data in question. This would also allow people in other countries or with slightly different use-cases to use your tool.

knaaptime commented 4 years ago

hi, thanks for the feedback. The function i'm referencing here actually takes a path to a raster, unless a user passes a 'convenience string' as a shorcut--and if a user wants to leverage one of those conveniences, we should be able to stream the data instead of them to download all of it.

If a user wants to use data from another country, he or she can do so by supplying a path to the raster, in which case tobler makes no call to a remote server, silent or otherwise

kannes commented 4 years ago

That's awesome!

knaaptime commented 4 years ago

if the underlying issue is addressed upstream, we'll return to this