search

radiantearth / stac-browser

A full-fledged UI in Vue for browsing and searching static STAC catalogs and STAC APIs
https://radiantearth.github.io/stac-browser
ISC License
282 stars 140 forks source link

Support for S3 private buckets #83

Open g2giovanni opened 3 years ago

g2giovanni commented 3 years ago

Hello everyone, is there a way to use stac-browser with COG files stored on a private S3 bucket?

m-mohr commented 3 years ago

I don't think so, but I've never worked with private S3 buckets so I can't really tell, unfortunately.

rowanwins commented 2 years ago

Ultimately I suspect this is a problem that will need to be solved outside of the stac-browser client app.

If we limit the problem to the rendering of COGS then I can see two main approaches

  1. Use presigned-urls. Currently when the client requests tiles it sends a url like https://myendpoint/cog/tiles/1/2/3?url=https://somewhere.com/mycog.tif. As it stands the Tiling Infrastructure (TiTiler or RadiantTiler) doesn't have access to that COG file. However presigned urls are a way of providing access to secure file via a URL. So the request to the tiler might look something like https://myendpoint/cog/tiles/1/2/3?url=https://somewhere.com/mycog.tif?a-bunch-of-aws-presigned-params.
    You would however need a server with the bucket credentials that could generate the pre-signed url for you, and we'd also need to look at whether TiTiler and RadiantTiler would respect those url parameters. Maybe within the app there could be a config flag for the app that specified an endpoint for retrieving presigned urls, but we leave it up to each deployer to write their own server for generating those urls?

  2. Enhance TiTiler or RadiantTiler to better support auth Somehow write the AWS auth process into TiTiler or the RadiantTiler.

Hope that helps

m-mohr commented 2 years ago

Yes, I suspect the same.

One note regarding the tiling: STAC Browser also supports client side COGs (via stac-layer/geotiff.js) and TiTiler or RadiantTiles might not even be involved.

DanielJDufour commented 2 years ago

Hi, all 👋

I'm a big fan of the pre-signed URLs that @rowanwins mentioned. They definitely make things easier with the client-side visualization libraries. I think it's important to note that (last I checked), geotiff.js doesn't provide an easy way to provide authentication headers to the internal metadata and tile requests. We could potentially add it through a PR to geotiff.js.

I think TiTiler (because it uses rasterio/gdal) already supports reading from private or requester pays S3 buckets. You just have to set the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY environmental variables. I think I tested this a year or so ago accessing NAIP requester pays imagery, but can't recall 100%. Of course with the caveat that this is for buckets that your S3 account has access to. I'm honestly not sure what to do if an external user is "requesting" access to a bucket owned by someone else.

I like @rowanwins idea of having an authentication/signing endpoint.

No solutions, just thought I could provide a little more context :-)

rowanwins commented 2 years ago

I think geotiff.js would work fine with presigned urls as the auth info is sent via url params rather than headers 🤞

DanielJDufour commented 2 years ago

Agreed 👍 I'll keep my fingers crossed, too. haha