awslabs / open-data-docs

Additional documentation for publicly available datasets on AWS
Other
134 stars 25 forks source link

`.tif` files for some states in single year have heterogeneous projection #24

Open ViswanathB opened 1 year ago

ViswanathB commented 1 year ago

I am using the data in s3://naip-visualization/ to generate .vrt files and later use those files to get the raster image based on shapely boundary with rasterio.

However this is working only in few cases. For instance, I was able to build this for RI all four years available 2012, 2014, 2016,2018. Also for AL for the year 2013. I am starting to see issues for few other states.

AZ 2015: _Warning 1: gdalbuildvrt does not support heterogeneous projection: expected NAD83 / UTM zone 12N, got NAD83 / UTM zone 11N. Skipping /vsis3/naip-visualization/az/2015/100cm/rgb/32114/m_3211435_se_11_1_20150714.tif Warning 1: gdalbuildvrt does not support heterogeneous projection: expected NAD83 / UTM zone 12N, got NAD83 / UTM zone 11N. Skipping /vsis3/naip-visualization/az/2015/100cm/rgb/32114/m_3211435_sw_11_1_20150708.tif Warning 1: gdalbuildvrt does not support heterogeneous projection: expected NAD83 / UTM zone 12N, got NAD83 / UTM zone 11N. Skipping /vsis3/naip-visualization/az/2015/100cm/rgb/32114/m_3211436_ne_11_1_20150725.tif Warning 1: gdalbuildvrt does not support heterogeneous projection: expected NAD83 / UTM zone 12N, got NAD83 / UTM zone 11N. Skipping /vsis3/naip-visualization/az/2015/100cm/rgb/32114/m_3211436_nw_11_1_20150708.tif Warning 1: gdalbuildvrt does not support heterogeneous projection: expected NAD83 / UTM zone 12N, got NAD83 / UTM zone 11N. Skipping /vsis3/naip-visualization/az/2015/100cm/rgb/32114/m_3211436_se_11_1_20150725.tif Warning 1: gdalbuildvrt does not support heterogeneous projection: expected NAD83 / UTM zone 12N, got NAD83 / UTM zone 11N. Skipping /vsis3/naip-visualization/az/2015/100cm/rgb/32114/m_3211436_sw_11_1_20150708.tif Warning 1: gdalbuildvrt does not support heterogeneous projection: expected NAD83 / UTM zone 12N, got NAD83 / UTM zone 11N. Skipping /vsis3/naip-visualization/az/2015/100cm/rgb/32114/m_3211437_ne_11_1_20150725.tif Warning 1: gdalbuildvrt does not support heterogeneous projection: expected NAD83 / UTM zone 12N, got NAD83 / UTM zone 11N. Skipping /vsis3/naip-visualization/az/2015/100cm/rgb/32114/m_3211437_nw_11_1_20150725.tif Warning 1: gdalbuildvrt does not support heterogeneous projection: expected NAD83 / UTM zone 12N, got NAD83 / UTM zone 11N. Skipping /vsis3/naip-visualization/az/2015/100cm/rgb/32114/m_3211437_se_11_1_20150725.tif Warning 1: gdalbuildvrt does not support heterogeneous projection: expected NAD83 / UTM zone 12N, got NAD83 / UTM zone 11N. Skipping /vsis3/naip-visualization/az/2015/100cm/rgb/32114/m_3211437_sw_11_120150725.tif

CO 2013:

_Skipping /vsis3/naip-visualization/co/2013/100cm/rgb/38108/m_3810837_ne_12_1_20130920.tif Warning 1: gdalbuildvrt does not support heterogeneous projection: expected NAD83 / UTM zone 13N, got NAD83 / UTM zone 12N. Skipping /vsis3/naip-visualization/co/2017/100cm/rgb/38108/m_3810837_se_12_1_20171007.tif Warning 1: gdalbuildvrt does not support heterogeneous projection: expected NAD83 / UTM zone 12N, got NAD83 / UTM zone 11N. Skipping /vsis3/naip-visualization/az/2017/60cm/rgb/32114/m_3211440_se_11_h_20170702.tif Warning 1: gdalbuildvrt does not support heterogeneous projection: expected NAD83 / UTM zone 13N, got NAD83 / UTM zone 12N. Skipping /vsis3/naip-visualization/co/2013/100cm/rgb/38108/m_3810837_nw_12_1_20130920.tif Warning 1: gdalbuildvrt does not support heterogeneous projection: expected NAD83 / UTM zone 13N, got NAD83 / UTM zone 12N. Skipping /vsis3/naip-visualization/co/2015/100cm/rgb/38108/m_3810833_ne_12_1_20150909.tif Warning 1: gdalbuildvrt does not support heterogeneous projection: expected NAD83 / UTM zone 13N, got NAD83 / UTM zone 12N. Skipping /vsis3/naip-visualization/co/2017/100cm/rgb/38108/m_3810837_sw_12_1_20171007.tif Warning 1: gdalbuildvrt does not support heterogeneous projection: expected NAD83 / UTM zone 13N, got NAD83 / UTM zone 12N. Skipping /vsis3/naip-visualization/co/2013/100cm/rgb/38108/m_3810837_se_12_1_20130920.tif Warning 1: gdalbuildvrt does not support heterogeneous projection: expected NAD83 / UTM zone 13N, got NAD83 / UTM zone 12N. Skipping /vsis3/naip-visualization/co/2017/100cm/rgb/38108/m_3810838_ne_12_1_20170826.tif Warning 1: gdalbuildvrt does not support heterogeneous projection: expected NAD83 / UTM zone 13N, got NAD83 / UTM zone 12N. Skipping /vsis3/naip-visualization/co/2013/100cm/rgb/38108/m_3810837_sw_12_1_20130920.tif Warning 1: gdalbuildvrt does not support heterogeneous projection: expected NAD83 / UTM zone 12N, got NAD83 / UTM zone 11N. Skipping /vsis3/naip-visualization/az/2017/60cm/rgb/32114/m_3211440_sw_11_h_20170702.tif Warning 1: gdalbuildvrt does not support heterogeneous projection: expected NAD83 / UTM zone 13N, got NAD83 / UTM zone 12N. Skipping /vsis3/naip-visualization/co/2017/100cm/rgb/38108/m_3810838_nw_12_1_20171007.tif Warning 1: gdalbuildvrt does not support heterogeneous projection: expected NAD83 / UTM zone 13N, got NAD83 / UTM zone 12N. Skipping /vsis3/naip-visualization/co/2013/100cm/rgb/38108/m_3810838_ne_12_1_20130804.tif Warning 1: gdalbuildvrt does not support heterogeneous projection: expected NAD83 / UTM zone 13N, got NAD83 / UTM zone 12N. Skipping /vsis3/naip-visualization/co/2015/100cm/rgb/38108/m_3810833_nw_12_1_20150909.tif Warning 1: gdalbuildvrt does not support heterogeneous projection: expected NAD83 / UTM zone 13N, got NAD83 / UTM zone 12N. Skipping /vsis3/naip-visualization/co/2017/100cm/rgb/38108/m_3810838_se_12_120171008.tif Warning 1: gdalbuildvrt does not support heterogeneous projection: expected NAD8

I did see this for few other states too. Now, There can be users directly indexing into files and in someway figure out which tif to access based on the boundary but this would make it tough to use rasterio.

VRT is really good and rasterio takes care of stitching images together for overlapping boundaries between quarter quad tiles.

ViswanathB commented 1 year ago

@jeffejefe Appreciate your insight on this one. Is there an alias/tag specific to this naip I can include in future issues? I see this repo has other open data stuff too

jeffejefe commented 1 year ago

Hi @ViswanathB you can try the original author of the NAIP doc, @ abhiataero. Or email the contact as you did prior.

ViswanathB commented 1 year ago

@abhiataero Tagging for help!

ViswanathB commented 1 year ago

@abhiataero Tagging for help!

jeffejefe commented 1 year ago

Hi @ViswanathB one thing worth noting is that you can have more than 1 UTM zone in the same state so your condition of requiring a single homogeneous projection to use gdalbuildvrt will not work at the state prefix level. It might make sense to group gdalbuildvrt jobs by quad level or something where you won't have that conflict. There are probably other sources but here's a link to the shapefile index provided for each state: https://www.fpacbc.usda.gov/geo/customer-service/naip-quarter-quad-shapefiles/index.html