search

geotiffjs / geotiff.js

geotiff.js is a small library to parse TIFF files for visualization or analysis. It is written in pure JavaScript, and is usable in both the browser and node.js applications.
https://geotiffjs.github.io/
MIT License
880 stars 184 forks source link

[Feature requests] Support cloud storage URL (AWS, Google ...) #47

Open vincentsarago opened 6 years ago

vincentsarago commented 6 years ago

It will be really useful to support cloud storage url as s3://

I understand this is not an easy task, I could see if I can submit a PR (at least for AWS support)

cc @constantinius

constantinius commented 6 years ago

Hi @vincentsarago,

Thank you for for issuing this feature request. It definitely sounds like a really useful enhancement. Unfortunately I'm really not an expert for Amazon services. Looking into the matter, I'm a little confused about this s3:// protocol URL prefix. As far as I can tell, this is just a way to shorten the URL. I think I can remember that it also includes authorization management of some kind.

What would you say would be required to have a useful support of s3://?

On the other hand, this sounds like a good place for a customized Source. Such a Source must only have an async fetch(offset, length) method that actually makes the request. For HTTP based sources, I simply set the Range header to get slices of the file. It is then possible to use the fromSource() factory with such a source to instantiate a GeoTIFF object.

It seems like using the aws-sdk with the mentioned methods might yield the desired results.

Is this going into the desired direction?

vincentsarago commented 6 years ago

I need to dig more too.

On the other hand, this sounds like a good place for a customized Source. Such a Source must only have an async fetch(offset, length) method that actually makes the request. For HTTP based sources, I simply set the Range header to get slices of the file. It is then possible to use the fromSource() factory with such a source to instantiate a GeoTIFF object.

This is how we do it in COGDumper (python lib to read COG files) https://github.com/mapbox/COGDumper/blob/master/cogdumper/s3dumper.py#L20-L25