search

Esri / OptimizeRasters

OptimizeRasters is a set of tools for converting raster data to optimized Tiled TIF or MRF files, moving data to cloud storage, and creating Raster Proxies.
Apache License 2.0
127 stars 50 forks source link

Support for generating raster proxies of encrypted MRF files on S3 bucket ? #133

Closed AllisonRob closed 2 years ago

AllisonRob commented 3 years ago

Hi, We have a client using the OptimizeRasters tool in ArcGIS Pro 2.7, to create raster proxies for MRF files that are stored in an Amazon S3 bucket. The destination location for the raster proxies is on the client's internal file share. This worked successfully. Configuration changes were then made on the S3 bucket to encrypt some of the MRF rasters. Since making this change and then attempting to use the OptimizeRasters tool, the tool subsequently fails with HTTP 400 errors (ie. "HTTP Error 400: Bad Request") and no raster proxies are created, when the target MRF file is an encrypted one. If an unencrypted MRF is instead targeted, OptimizeRasters works successfully to create the raster proxy files, as before. I have scanned the OptimizeRasters_UserDoc.pdf and other contained documentation on this site, but there is no mention of this kind of scenario being accommodated or otherwise. Are you able to advise on whether the OptimizeRasters tool supports generation of raster proxies for encrypted target MRF rasters ?(ie. Support for generating raster proxies of encrypted MRF files on S3 bucket)

Looking forward to hearing any advice on this matter. Thanks very much, Allison Robertson Esri Australia

Chamlika commented 3 years ago

@AllisonRob Hi, OptimizeRasters requires the source input raster to be recognized as a valid raster so that the internal components/GDAL can work accordingly. For me this looks like the OptimizeRasters request needs to have certain header information/URL query parameters before the request is made to ensure the AWS/cloud providers to honor the incoming request as a valid URL request. This could explain why you get the (HTTP error 400). Presently we'll need to check if this is an isolated case related to your S3 configuration, or more of a general issue with OptimizeRasters. We'll look into this, thanks! @abhiataero FYI

AllisonRob commented 3 years ago

Hi @Chamlika and @abhiataero, Thanks for your initial thoughts on this issue. As requested separately, sample imagery from the client site is presently being made available, and I will be emailing Abhijit directly. Regards, Allison

abhiataero commented 3 years ago

@AllisonRob can you confirm is this works so we can close this issue

AllisonRob commented 3 years ago

Hi @abhiataero, I am still waiting to hear from the client, but given I was able to reproduce the issue, and then see that using the "Secured Bucket" option (in the latest update) resolved it, I can confirm that it works and the issue can be closed.
Thanks for your time and attention to this issue. Regards, Allison