FLIR temp conversion with window: results from in-house calibration

[NewcombMaria]

@max-zilla and @dlebauer and others, the in-house FLIR tests in the controlled temperature room are complete, and @anfrench has done in-depth analyses resulting in two alternative methods to read raw FLIR data (DN or digital number values) and ambient temperature data and converts DN to temperatures. The FLIR camera is installed in the camera box now and will be collecting data soon, after some connectivity steps with the server are worked out.

Andy wrote two R scripts, one for each of the alternative temp conversion methods he has developed. These scripts are copied to the directory https://drive.google.com/drive/folders/1dDPjEqCw86jg2BAScvBe2dvn66omIHks

One method (FLIRgantry2TC_direct.r) as stated by Andy: "works by interpolation between cubic polynomials that were fit to the temperature/DN data sets collected in the constant temperature room. You ran the tests at 5,15,25,35,40 and 45C, so there are 6 sets of cubic polynomials. The coefficients are embedded in the R script. The R script reads each DN value one at a time, computes 6 target temperatures corresponding to the 6 ambient room temperatures, then linearly interpolates the target temperature based on the observed ambient temperature. The rest of the routine writes the binary float data plus a header file."

The other method (FLIRgantry2TC.r) reads the raw FLIR data and associated ambient temp data and reads an associated csv file (copied at same directory) that's a table of temps and estimated spectral radiances for the camera and converts raw DN to temps as a 4-byte float binary output file.

Here's graph of blackbody target temp related to DN for 6 room temps:

[NewcombMaria]

In the recent calibration and troubleshooting efforts we've learned that the correct documentation for the FLIR camera is Model A655sc
On the website and in the metadata it's reported as FLIR model SC615.

We also now have a new window for the protective housing for the FLIR camera. The new window (Part number T198233SP: SP PROTECTIVE WINDOW ASSY Large) is approx 70mm diamter, measured on the inside of the cap. The original window that had problems with water deposits that appear to have damaged the window material was 48mm diameter. The calibration tests were completed with the new window.

[max-zilla]

A branch has been created for this work: https://github.com/terraref/extractors-multispectral/tree/flir2tif-temp-conversion

[max-zilla]

@NewcombMaria @anfrench I moved the R scripts and CSV file into the branch but have some questions:

• which of the two R scripts is preferred to use?
• the FLIR data we are moving to NCSA and converting to GeoTIFFs are _ir.bin files, but in these scripts the files are ".raw" files. are these the same as the bin files? if not, where are the raw files converted to bin? that happens before our pipelines begin so this can't be integrated into our extraction pipeline in that case.

[NewcombMaria]

@max-zilla, from conversation with @anfrench:

• Andy's preference out of the two R scripts: FLIRgantry2TC_direct.r

• the _ir.bin files are equivalent to the raw files and the R script should work with the _ir.bin files that are in the gantry cache and transferred to NCSA. Andy compared the .bin files to the raw files and they are equivalent format.

[dlebauer]

refactor to put parameters and functions in sensor metadata?

[robkooper]

@rickw-ward is interested in the flir data, lets see where we are wit the pipeline and the actual data.

[max-zilla]

@rickw-ward we are currently catching up from system outage last week and are processing 16th-17th. The processed FLIR data can be found in globus here: /ua-mac/Level_1/ir_geotiff/

...as geotiffs. there are also some full-field stitches of geotiff data here: /ua-mac/Level_1/fullfield/

At Rob's suggestion I ran a similar graph to the DN one presented above using our previous algorithm: At least it's pretty comparable to what we have above so the data isn't completely untrustworthy (we still plan to reprocess with new formula)

The 6/16 collection is in Clowder here: https://terraref.ncsa.illinois.edu/clowder/collection/5b31100f4f0c2b43c0582181

[max-zilla]

@dlebauer mentions: is this valid for historical data or just a particular date? can we parameterize some of the R script?

[anfrench]

I’m not sure what is being asked, the R script is intended to be applicable to data being collected now with the new thermal infrared window. Data collected using the older window will require using different coefficients—the transmissivities of the thermal windows are all different. –Andy French

From: Max Burnette [mailto:notifications@github.com] Sent: Thursday, July 5, 2018 11:43 AM To: terraref/reference-data Cc: French, Andrew; Mention Subject: Re: [terraref/reference-data] FLIR temp conversion with window: results from in-house calibration (#230)

@dlebauerhttps://github.com/dlebauer mentions: is this valid for historical data or just a particular date? can we parameterize some of the R script?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/terraref/reference-data/issues/230#issuecomment-402816494, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AVfPSVR8F-MfP3YONgWDyBuHnyYNBZAvks5uDl4mgaJpZM4SK2E4.

This electronic message contains information generated by the USDA solely for the intended recipients. Any unauthorized interception of this message or the use or disclosure of the information it contains may violate the law and subject the violator to civil or criminal penalties. If you believe you have received this message in error, please notify the sender and delete the email immediately.

[rickw-ward]

I'm working on generating plot level data from flir geotiffs @max-zilla referenced above. These are uncorrected, so I'm just working on workflow. I've attached a screen shot of Qgis . The screen shot is of a portion of the gantry polygons from a year ago, with identify results show in right panel for two Flir geotiffs that overlap with each other and a plot polygon. @NewcombMaria , this shows the grayscale images.

Some attribute of the geotiffs or the polygons prevents me from generating plot means of the band 1 values in the geotiffs (degrees Kelvin i think). But if I create a new polygon feature, it will show the means of geotiff values for its extent.

[jdemieville-ua]

Follow-up for new enclosure and window located here: https://github.com/terraref/computing-pipeline/issues/610