4 Band Processing

[qjhart]

One NAIP year is distributed using two sets of 3-band imagery rather than one four band image. This should be updated to match the other years. We should be able to do this by building a set of VRT files and then using gdal_translate to create a new geotiff from those vrt files.

[ghost]

I think 2005 would be the year with 3-band imagery.

[qjhart]

The CERES data for the 2005 imagery is pretty messed up. I have made some modifications to the data on /zat before starting to try and simplify things. Basically, I've moved all the CIR (c.tif) data and all the Natural color (n.tif) data into the same directory structure ../naip/2005/usda_file_archive/32114/* . I'm in the process of copying over 1 or two quads worth of data, so that we can see about automating the task of building these images. I will let you know when the data is online on quinn.cws.ucdavis.edu.

[qjhart]

@gwatprg I have created a group ca-naip on my computer, quinn.cws.ucdavis.edu. We've previously added on that computer. At file://quinn.cws.ucdavis.edu/mnt/tmp/naip are some example 2005 geotiff data. We can start with that to create the vrt files, and the new 4-band geotiffs.

[ghost]

@qjhart used gdalbuildvrt to write 4 separate 1-band files, taking RGB from the n.tif and band 1 from the c.tif. combined those using gdalbuildvrt -separate into one 4-band file. Used gdal_translate to output the GeoTIFF. I can include the individual commands, if you want.

I assume this is not correct. I noticed two errors. First, while the natural color RGB of the output m_3211401_ne_11_1_20050529.tif visually looks like the original n_3211401_ne_11_1_20050529.tif, the cir does not (used band4 as red, band 1 as green and band 2 as blue). Secondly, gdalbuildvrt specifying the band number gives this error, but appears to build the vrt correctly: Error in `gdalbuildvrt': double free or corruption (!prev): 0x0000000002256470

[ghost]

@qjhart It looks like you created a new vrt using 3 bands from the c tiff file and band 1 from the n tiff file. I thought the n_ was the natural color files, and I thought the order of the bands in the combined files, from what I had read, was Red, Green, Blue, Near Infrared - hence the near infrared is expected to be band 4.

[ghost]

@qjhart I ran separated band 2 from the c_ file (which should be the red band) and band 1 from the nfile (which should also be the red band) ran gdalcompare on my laptop and they are different. Similarly the c band 3 (which should be green) and n_ band 2 (which should also be green) are somewhat different. This is creating the visual difference for the 4 band file comparing with the cir file.

[qjhart]

@gwatprg OK, you are correct, n=natural and c=color IR. I screwed that up. I actually don't know what bands are used in the CIR version, let me check. However, what i hope is we can use a single template, as in the m_3211401_ne.vrt file, and then just change the names. We need to verifythe rows and columns are the same for every file.

[qjhart]

This is the command I used for the translation BTW.

gdal_translate -co COMPRESS=DEFLATE -co PREDICTOR=2 -co ZLEVEL=9 m_3211401_ne.vrt m_3211401_ne.tif

[ghost]

@qjhart Alternatively, I could create a script/Make file and loop over the files in the directory. If we build the vrt files each time, then it doesn't matter if there are different numbers of rows and columns. However, that would create a whole bunch of vrt files, which would then could be deleted as the last statement.

[qjhart]

I think we are saying the same thing, I just think it's easier to make the vrt file from template than from a number of createvrt commands.

Regarding the bands, you are correct, the CIR green and blue do not seem to correspond to any of the NAT bands. You'll need to search a bit to see if you can figure out why that is.

[ghost]

@qjhart I'll look into this some more.

[ghost]

I now have the answer through several sources. According to the both the DFG metadata and USDA (Louise Mathews), the CIR product is NOT a NAIP product in 2005. It was acquired separately from the vendor: "Color infrared (CIR) imagery acquired during NAIP 2005 flights. The source CIR imagery was acquired during the summer of 2005 and purchased from the North West Group by three state agencies (California Dept. of Fish and Game, California Dept. of Transportation, and California Dept. of Water Resources). No access constraints, but there are use constraints (see product metadata). CIR coverage was not available in all areas. THIS CIR IMAGERY IS NOT A NAIP PRODUCT. Band1=NearIR, Band2=R, Band3=G"

These probably should not be combined. Will Patterson commented: "The 3-Band RGB NAIP 2005 imagery (including DOQQs) were official NAIP products, but the separate CIR DOQQs were not - the CIR data was acquired during NAIP flights but released under a separate agreement not through the NAIP program. We call that set the "California CIR 2005" set of imagery. Attached are some metadata related to the set, there are no access restrictions but note there are use restrictions. Some metadata are also available for the service of that imagery, which we are hosting here: https://map.dfg.ca.gov/arcgis/rest/services/Base_Remote_Sensing/California_CIR_2005/ImageServer

• Also per the metadata, the band order of the California CIR 2005 product is: Band 1 NIR, Band 2 Red, Band 3 Green
• Regarding merging the 2005 IR data back with the 2005 RGB data to make a 4-band product, that may not be a good idea either because of the usage limitations/licensing (review metadata), or also potentially because the CIR DOQQs may use a slightly different mix of dates than the RGB DOQQs - it's been a while since I compared those products, but I thought I recall seeing some strange things between them like cars that didn't appear in the CIR but did in the RGB, or they had moved slightly, anyway, something to watch for if you try it. Let me know if you have further questions, and there's a contact (Kim Welter) for the contractor listed in the metadata you can try."

[ghost]

Also, note that we will have to comply with the use restrictions for the CIR data from 2005, if we are going to serve it.