search

Esri / OptimizeRasters

OptimizeRasters is a set of tools for converting raster data to optimized Tiled TIF or MRF files, moving data to cloud storage, and creating Raster Proxies.
Apache License 2.0
127 stars 50 forks source link

Problem converting WV3 to MRF #24

Closed ghost closed 6 years ago

ghost commented 8 years ago

I installed OptimizeRasters yesterday and have been experimenting with converting WorldView-3 imagery to MRF. I started by trying to use OptomizeRasters to convert and upload the MRF files to S3. The process seemed to work but when I checked the files on S3 they were very small -- just a few bytes. They obviously didn't convert. I didn't check the log file to see if there were errors.

I decided to pull back and try something simpler. I just used the DG template to convert to MRF on the local file system (I made a minor modification to the xml to create more pyramids up to a factor of 1024). The scrypt seemed to complete successfully but after looking, I saw errors I don't understand (I do understand the pyramid level warnings).

I tried to access the .imd files in the output MRF folder using ArcMap. ArcMap warned me that the images had inconsistent spatial references. I checked, and the ntf files, which I understand are now MRF, did not had a spatial reference. I tried to manually add a reference using ArcCatalog. It did not take.

I have attached the log file from OptimizeRasters.

Greg Morlock OptimizeRastersLog.txt

PBecker92373 commented 8 years ago

Greg Let us take a look at this. I believe this is something related to how GDAL_Translate handles NITF files. A preliminary review of this shows that the RPCs are being lost.

The pyramid level warnings are just to define that you have defined pyramids that are smaller than 1 tile so will not be created. It is better to define too many pyramids and get some warnings back then have too few and end up with gaps. I’m hoping in a future version we can remove the warnings or just create the correct levels.

_Peter

From: gmorlock [mailto:notifications@github.com] Sent: Friday, March 11, 2016 10:03 AM To: Esri/OptimizeRasters OptimizeRasters@noreply.github.com Subject: [OptimizeRasters] Problem converting WV3 to MRF (#24)

I installed OptimizeRasters yesterday and have been experimenting with converting WorldView-3 imagery to MRF. I started by trying to use OptomizeRasters to convert and upload the MRF files to S3. The process seemed to work but when I checked the files on S3 they were very small -- just a few bytes. They obviously didn't convert. I didn't check the log file to see if there were errors.

I decided to pull back and try something simpler. I just used the DG template to convert to MRF on the local file system (I made a minor modification to the xml to create more pyramids up to a factor of 1024). The scrypt seemed to complete successfully but after looking, I saw errors I don't understand (I do understand the pyramid level warnings).

I tried to access the .imd files in the output MRF folder using ArcMap. ArcMap warned me that the images had inconsistent spatial references. I checked, and the ntf files, which I understand are now MRF, did not had a spatial reference. I tried to manually add a reference using ArcCatalog. It did not take.

I have attached the log file from OptimizeRasters.

Greg Morlock OptimizeRastersLog.txthttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Esri_OptimizeRasters_files_169694_OptimizeRastersLog.txt&d=CwMCaQ&c=n6-cguzQvX_tUIrZOS_4Og&r=ffT1KWB_URwr_xV7pCjR7w&m=jmFrvPNIIRmSiQr6V15VlEleWW4oEYUR9_N0nXZbKZc&s=oA97pZupU9--q7CQI4x25ms4Wj1pGQnH1HSAR49TpJQ&e=

— Reply to this email directly or view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Esri_OptimizeRasters_issues_24&d=CwMCaQ&c=n6-cguzQvX_tUIrZOS_4Og&r=ffT1KWB_URwr_xV7pCjR7w&m=jmFrvPNIIRmSiQr6V15VlEleWW4oEYUR9_N0nXZbKZc&s=r_NYukVY2zCrTYGu96Bxr_aS7PXHMUXjyyEWKxBpZPY&e=.

ghost commented 8 years ago

OK, thanks. I look forward to hearing from you.

Greg

On Fri, Mar 11, 2016 at 12:51 PM, Peter Becker notifications@github.com wrote:

Greg Let us take a look at this. I believe this is something related to how GDAL_Translate handles NITF files. A preliminary review of this shows that the RPCs are being lost.

The pyramid level warnings are just to define that you have defined pyramids that are smaller than 1 tile so will not be created. It is better to define too many pyramids and get some warnings back then have too few and end up with gaps. I’m hoping in a future version we can remove the warnings or just create the correct levels.

_Peter

From: gmorlock [mailto:notifications@github.com] Sent: Friday, March 11, 2016 10:03 AM To: Esri/OptimizeRasters OptimizeRasters@noreply.github.com Subject: [OptimizeRasters] Problem converting WV3 to MRF (#24)

I installed OptimizeRasters yesterday and have been experimenting with converting WorldView-3 imagery to MRF. I started by trying to use OptomizeRasters to convert and upload the MRF files to S3. The process seemed to work but when I checked the files on S3 they were very small -- just a few bytes. They obviously didn't convert. I didn't check the log file to see if there were errors.

I decided to pull back and try something simpler. I just used the DG template to convert to MRF on the local file system (I made a minor modification to the xml to create more pyramids up to a factor of 1024). The scrypt seemed to complete successfully but after looking, I saw errors I don't understand (I do understand the pyramid level warnings).

I tried to access the .imd files in the output MRF folder using ArcMap. ArcMap warned me that the images had inconsistent spatial references. I checked, and the ntf files, which I understand are now MRF, did not had a spatial reference. I tried to manually add a reference using ArcCatalog. It did not take.

I have attached the log file from OptimizeRasters.

Greg Morlock OptimizeRastersLog.txt< https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Esri_OptimizeRasters_files_169694_OptimizeRastersLog.txt&d=CwMCaQ&c=n6-cguzQvX_tUIrZOS_4Og&r=ffT1KWB_URwr_xV7pCjR7w&m=jmFrvPNIIRmSiQr6V15VlEleWW4oEYUR9_N0nXZbKZc&s=oA97pZupU9--q7CQI4x25ms4Wj1pGQnH1HSAR49TpJQ&e=>

— Reply to this email directly or view it on GitHub< https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_Esri_OptimizeRasters_issues_24&d=CwMCaQ&c=n6-cguzQvX_tUIrZOS_4Og&r=ffT1KWB_URwr_xV7pCjR7w&m=jmFrvPNIIRmSiQr6V15VlEleWW4oEYUR9_N0nXZbKZc&s=r_NYukVY2zCrTYGu96Bxr_aS7PXHMUXjyyEWKxBpZPY&e=>.

— Reply to this email directly or view it on GitHub https://github.com/Esri/OptimizeRasters/issues/24#issuecomment-195520240 .

ghost commented 8 years ago

Peter, After our conversation this morning, just checking on the status of this problem.

PBecker92373 commented 8 years ago

The existing version does loose the RPC information from the NITF files. A fix is going into OptimizeRasters to correct this but it does require and update to the GDAL part so will take a bit more time. There is also a requirement to get ArcGIS to pick up the RPCs correctly from the Aux.xml files which should be in the next 10.4.1 release.

ghost commented 8 years ago

OK, Thanks.

On Tue, Apr 26, 2016 at 2:39 PM, Peter Becker notifications@github.com wrote:

The existing version does loose the RPC information from the NITF files. A fix is going into OptimizeRasters to correct this but it does require and update to the GDAL part so will take a bit more time. There is also a requirement to get ArcGIS to pick up the RPCs correctly from the Aux.xml files which should be in the next 10.4.1 release.

— You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub https://github.com/Esri/OptimizeRasters/issues/24#issuecomment-214878719

abhiataero commented 6 years ago

The issue has been fixed