Write extractor for FLIR data

[dlebauer]

Description

Convert NDVI binary files to png thumbnail + geoTIFF or netCDF raster data product.

(Updated Sept 16 2016)

For data generated through Sept (?15?) 2016 (provided in units of mK)

• Per https://github.com/terraref/computing-pipeline/issues/151#issuecomment-247247854 FLIR output was originally calibrated to output Temperature in units of 100mK. (Hence a saved pixel value of 3000 translated into 300K). We should be able to process these manually ( as T(C) = x/100,000 - 273.15)
• this does not need an extractor, though that may be the easiest way to do just a script to process the files and put them in level 1 data products as geoTIFFs

  for data generated as radiance values

• On / around Sept 14, this was changed to output uncalibrated radiance and now parameters for converting to C are provided in metadata. The equation is provided in https://github.com/terraref/computing-pipeline/blob/master/scripts/FLIR/FlirRawToTemperature.m. This should be used for the official extractor.

Steps:

• [x] Determine if it is easier to call the FlirRawToTemperature.m or re-write (e.g. in Python within extractor)
• [ ] Update FlirRawToTemperature.m with metadata parameters (?)
• [x] Add geospatial attributes and metadata to file
• [ ] document
• [x] write test.sh to work on sample data and compare to sample output

This is a subtask of #64

[ZongyangLi]

@dlebauer Now we have a python script converting flir data into pngs and geotiffs. Here is an example:

FLIR's pixel value range from 2800 to 3300, I re-scale it to a visible range. My question is:

1. how to calibrate values to Temperature in degrees C?
2. Is that means an average value in the image?

[dlebauer]

From the GetFLIR.m file that @TinoDornbusch wrote, I infer that temperature = ((value - 2800) / (3300 - 2800))

Does that produce reasonable values?

@LTBen is this in the protocol for calibrating FLIR data? Can you please share a copy?

[ZongyangLi]

From this equation we will get a value normalized in (0, 1), which I used to re-scale it to (0, 255) creating the visible images. This may not be the temperature in degrees C.

[TinoDornbusch]

I transferred on our findings transformation of FLIR raw values to temperature via Email to the Maricopa team. However it did not end up here. Here it comes:

The output of the FLIR is currently in 100mK. Hence a saved pixel value of 3000 translated into 300K.

FLIR does use a calibration algorithm, which I will post here as MATLAB script. By default this calibration is applied internally by the camera. Hence until now data is already calibrated using a fixed set of parameters, which is now also provided in the FLIR metadata:

  "output data format": "16bit Grey",
  "calibration available": "calibration parameters from FLIR to convert gray value to temperature",
  "sensor computer interface": "GigE",
  "sensor id": "ir camera box",
  "calibration R": "16923.6",
  "calibration B": "1434.6",
  "calibration F": "1",
  "calibration J1": "70.0704",
  "calibration J0": "4124",
  "calibration alpha1": "0.006569",
  "calibration alpha2": "0.01262",
  "calibration X": "1.9",
  "calibration beta1": "-0.002276",
  "calibration beta2": "-0.00667"

[dlebauer]

So if an image has a rang 2800 to 3300, that is equivalent to 280-330 K / 27-57 C / 80-134F? The range and max seems high.

[TinoDornbusch]

Script: https://github.com/terraref/computing-pipeline/blob/master/scripts/FLIR/FlirRawToTemperature.m

[TinoDornbusch]

Forget about the script GetFLIR.m. This was just a helper to display data. Use FlirRawToTemperature.m.

However, here is the problem: Note that until today all data is already processed an output as 100 mK. If you want to change the calibration using different parameters, you need to "revert" the script, computing back the original 16bit raw pixel values and reapply the script with changed parameters.

[TinoDornbusch]

Here is the Email conversation I had with scientists at Maricopa. Sorry that this information has not leaked through.

---

Tino, when is the acceptance test? I am thinking a good cal/val test would rely on a transferred approach, namely use well-calibrated apogee infrared thermometers (do you have any?) in parallel with the FLIR camera. Ive thought about reference field targets but large ones are difficult except for the not-very-practical water ice state. Instead find or make targets with high emissivity, which would be full cover plants , open water (maybe a few ice chests with ambient temperature water; there are some details for this reference that need further discussion…specifically the bulk water temperature can be logged but that temperature is not quite the same as the water skin temperature, the latter subject to evaporative cooling) and wetted bare soil. ALARC has a temperature calibration room that can be used to check calibration of the apogee thermometers at differing ambient and target temperatures. If I am to do this part I would need some tech help, all of these are time consuming tasks. -Andy

---

Andrew,

Any thoughts on the reference temperature objects? Just want to know whether we can perform some joint measurements during the site acceptance test. You know best that IR cameras are difficult to calibrate & validate.

Thanks for your time Tino

---

Hi Andrew,

The relatively long lag time is actually a good news for the new campaign. Sounds that we might not need to move the FLIR out of the box south right now.I regard the FLIR data so far not so useful since we scanned along east-west and had shade for about 2min.

We will now also establish a scan mode to scan along north-south. This way shading before imaging is just a couple of seconds.

In Maricopa we briefly talked about blackbodies as temperature reference. We need some references to evaluate the temperature conversion. Any idea how to test FLIR camera best?

Tino

---

Tino, this sounds like a good plan, i.e. record raw output, also collect Tsens if you can without activating the internal calibration routine.

Today I ran an informal shading test to evaluate temperature change of the soil surface when shading is introduced. I used one of ALARC’s FLIR cameras and found a latency on the order of 30-50 seconds; this is a much greater time than I expected (I had been expecting <10 seconds). I think this observation (and follow-up tests) could be a helpful finding for TERRRA scans. I suspect the long lag time is indicative of high soil moisture conditions and could mean that the deleterious effect of box shading may not be severe for scans if the time between FLIR image collects and onset (or removal) of shading is less than about ½ minute. I realize this time will vary depending on time of day and protocol but do you have a rough idea of lag times for the just-finished sorghum experiment? -Andy

• Andy

---

Dear all,

We will set the camera to raw output. Internal Temperature calibration does not make sense. You have no control on parameters, from which ambient temperature and emmisivity have the main impact on temperature conversion. You know better.

Tsens is the temperature on the microbolometer, which FLIR uses for internal calibration. We have no control over that. We can just switch it on/off.

We will add all parameters required for the calibration function in the sensor metadata. Remaining environmental parameters (Relative humidity, ambient temperature, object distance, emissivity) need to be taken from environmental loggers. Humidity is in Maricopa at such short distance no problem (according to FLIR).

Best Tino

---

Tino, this isn’t my call to decide protocol; but yes Id suggest raw mode and also add Tsens (though not sure what that value means in this context, Sensor body temperature?). –Andy French

---

Andrew,

Yes it has been a hard time to get the FLIR calibration formulae. Now you can better assess if that procedure is applicable compared to approaches you use.

If you give the OK, I set the FLIR to “raw mode” removing the internal calibration. That you get raw values and can apply any correction formulae you want.

I could only find one temperature value in the FLIR settings that is not preset (TSens; see screenshot). All other relevant parameters -from which temperature and object emissivity are apparently the most important- need to be measured or estimated with other sensors.

If you want, we can add TSens to the variable MetaData of the FLIR.

Best Tino

---

Tino, this is the first time I’ve seen such equations, thanks for sending along. My experience with FLIR is that they are generally unwilling to share details of the camera calibrations. Ive not been able to extract internal reference temperature, but that would be very helpful to have since that governs the signal from the microbolometer. One suggestion for data collection is to set the target emissivity to 1.0, i.e. blackbody. Then what you have for data are brightness temperatures and one can proceed from there using one’s own skyview factors, surface and sky emissivity estimates. Surface emissivity values can vary a bit- 0.96 is for plants is reasonable though Id choose something closer to 0.97-- and strongly depend upon spectral response of the FLIR camera as well as upon the soil and vegetation fractions. For phenotyping work it may not matter if selected emissivities are inaccurate as long as you the way they are selected is consistent across the experiment. –Andy French

---

Tino, Thanks for this valuable update which has critical information for processing the FLIR data. Andy French may want to comment on suppressing the internal self-calibration. My impression was that the camera have an internal body reference temperature, which we would need to access. The values of emissivity that you report as being used by FLIR also merit discussion. Apogee’s values for plants are 0.96: http://www.apogeeinstruments.com/emissivity-correction-for-infrared-radiometer-sensors/ The same web page explains the need to consider sky temperature as well. As we discussed previously, a huge issue there is how to deal with the contribution of the instrument box to the net sky temperature. Best regards,

• Jeff

---

Dear all,

I am not sure how to deal with opening issues at Github, so I use the classical way. Sorry for the late reply, but things were a bit busy here after my long holidays in Maricopa.

We finally figured out the calibration procedures done internally by FLIR IR camera. Please not that FLIR did not want to give us their calibration formulae directly. We got this information indirectly from Rothamsted.

Please find attached on Octave script to transform Raw Pixel values to temperature. I added relevant internal camera parameters. To my understanding, the most important parameter is the object emissivity E, which is set by FLIR to 0.98. Note that there is difference between soil (E=0.93) and vegetation (E=0.98) as given by FLIR.

After some evaluation we found out that the internal calibration was switched on in the FLIR camera. Hence the calculation as stated in the script were already done by the camera. The output of the file is in cK, 3000 cK = 300K = 26.85°C.

I personally would not let make the camera do the calibration internally. You have no control on parameters, especially ambient / atmosphere temperature and object emissivity. That means if you want to recompute raw values for the existing data you need to invert the calibration function.

I would suggest to put the FLIR in Raw mode for the upcoming experiment and use the calibration script.

Hope this is clear and please open a GiTHUB issue for this.

Thanks Tino

---

HI Tino, Fully understood. I am hoping we can move toward a clear plan for understanding how best to use the system. Best regards, Jeff

---

Jeff,

I leave a detailed and diplomatic answer to my superiors. Note that I just got involved in the project mid April. Hence I am still in the middle of catching up while following my other duties. Just doing my best to understand and operate the system together with you.

Tino

---

Tino, In purchasing any imaging system or other complex field instrument, I would expect the provider to provide basic information on calibrations. Depending on the instrument this might consist of a combination of:

1. A statement that the instrument was factory calibrated and shown to be within the performance specifications.
2. Standard procedures for checking calibrations or re-calibrating the sensor at certain use intervals.
3. Recommendations for calibration, if required, during routine use. These could be considered level 1 calibrations, where the standards are typical laboratory-quality standards. These would be done under the best possible operating conditions and using formal standards. The next level of calibration (level 2) involves understanding how readings might be degraded under fairly optimal field conditions and the actual targets intended for end-use. For example, with either the 3-D or Stereo-VIS, one would want to understand if distance accuracy for sorghum plants is degraded relative to the aluminum standard. With the 3-D, one might expect that reflectance would vary due to leaf angle and that this would degrade the accuracy. With Stereo-VIS, leaf angles and shading might be issues. The final level (3) is under routine field use. This is where we would hope to understand how to deal with effects of diffuse vs. direct beam illumination, illumination angle, temperature drift (if any), movement due to wind or sensor positioning. My understanding was that LemnaTec (Ben) was working on documentation at level 1. Discussions of level 2 and 3 were started concerning the hyperspectral and thermal cameras but need to move forward. Discussions of interacting effects of illumination, shutter speed, movement, etc. have sort of started and stopped a couple of times. There was an encouraging note from Stewart that people (not clear who was involved) had concluded that we could scan at 0.33 m/s without degrading the StereoVIS images. I’m not sure what data are available, but Stewart indicated they were using formal tests of resolution. A key consideration for TERRA REF is that the data to be released will represent a “reference” set, which implies that data acquisition protocols are supported by formal testing.
   • Jeff

---

Jeff,

What do you mean by ‘LemnaTec formal calibration procedure`?

If you specify the location of the reference panels and visiting frequency, I can program the gantry.

Data analysis is the task of the Danforth team. Solmaz is working on that from LemnaTec side.

You are right, same thing for FLIR thermal camera. We are getting a solution soon.

TIno

---

https://drive.google.com/drive/folders/0B4JvLJOau3GmR3JRcHZRZEwwR1E This is a very busy time of the year in Maricopa, so we haven’t had a follow-up meeting on the calibration strategy. What I can report to date is:

1. Kelly Thorp compared three white ceramic tile samples to Spectralon panels on three days. File “2016 – Floor Tile Spectra.xlsx” shows the results and E-mail_chain… pdf has a discussion of interpretation of the results.
2. Kelly also noted that painted barium sulfate panels were used before Spectralon was available, and we have since located a short note that indicates that mixing barium sulfate with white paint can provide a more permanent, low-cost panel (Knighton & Bugbee, 2004). Edmund sells what seems to be a pre-mixed paint: http://www.edmundoptics.com/testing-detection/measurement-tools/pre-mix-white-reflectance-coating/1325/
3. Andy French’s “best bets” are: a. Use multiple Spectralon panels b. Mount a standard in the field of view of the imager
4. We still await: a. LemnaTec’s formal calibration procedure b. A working hyperspectral camera system so we can test panels with the scanner cameras c. Persons to pursue specific tasks like preparing barium panels I don’t think anyone budgeted (in time or funding) for this work, so we are basically having to advance this opportunistically. My impression is that the situation with the thermal camera is similar.
   • Jeff

[dlebauer]

I didn't realize these were calibrated ... So you recommend turning the FLIR to raw mode and then calculating T?

doing the math of updating previous FLIR images with new parameters should be straightforward if deemed necessary.

[TinoDornbusch]

I recommend to use raw values. Like this you have all freedom to play with the parameters. Note that the most crucial parameter is the Emissivity, that impacts conversion. Emissivity is different e.g. between soil and plants.

I switched the FLIR to raw output from today on. There is a valueable parameter Tsens (internal FLIR temperature). You can choose, where to measure temperature: Shutter, lens, front

This is currently not included to the Metadata. Do you want us to include this in Metadata? Which Temperature value do you want to output.

[dlebauer]

What will be most useful? @jeffwhiteaz ?

[TinoDornbusch]

Temperature Data from FLIR is written to the variable Metadata. Three temperatures are logged: Shutter, Lens, Front. These temperatures can be used in the script to estimate external Optic temperature as parameter required in the FLIR script.

"sensor_fixed_metadata": {
  "sensor manufacturer": "FLIR Systems",
  "sensor product name": "A645",
  "sensor serial number": "13070601",
  "sensor description": "IR Sensor 640x480",
  "sensor purpose": "measure infrared radiation from soil/crop surface",
  "buildin variable optics": "",
  "location in gantry system": "camera box, facing ground",
  "location in camera box x [m]": "0.877",
  "location in camera box y [m]": "1.361",
  "location in camera box z [m]": "0.520",
  "field of view x [m]": "1.496",
  "field of view y [m]": "1.105",
  "output data format": "16bit Grey",
  "calibration available": "calibration parameters from FLIR to convert gray value to temperature",
  "sensor computer interface": "GigE",
  "sensor id": "ir camera box",
  "calibration R": "16923.6",
  "calibration B": "1434.6",
  "calibration F": "1",
  "calibration J1": "70.0704",
  "calibration J0": "4124",
  "calibration alpha1": "0.006569",
  "calibration alpha2": "0.01262",
  "calibration X": "1.9",
  "calibration beta1": "-0.002276",
  "calibration beta2": "-0.00667"
},
"sensor_variable_metadata": {
  "current setting AutoFocus": "1",
  "current setting Manual focal length [cm]": "195",
  "camera info": "A645,Gen_A/G,GEV,1.0.0,GEV,1.2.17  (13070601)",
  "focus distance [m]": "2.021",
  "lens temperature [K]": "306.499",
  "shutter temperature [K]": "306.193",
  "front temperature [K]": "307.094"
}

[JeffWhiteAZ]

In response to my query to Andy French about the best ttemperature to use, Andy responded: Ideally the reference temperature is the detector temperature, not the lens temperature or shutter temperature , but that temperature is better than ambient air temperature. But I cannot believe FLIR doesn’t incorporate detector temperature in their calibration, which means that I could be accessible if FLIR wants it to be. Alternatively try to thermally isolate (not insulate) the FLIR so that the optic train temperature is close to or correlated to the detector temperature. Metadata recording of this temperature a very good idea. -Andy

[JeffWhiteAZ]

Do we know what the "front" temperature actually measures? It sounds like the body surrounding the lens, but it would to know for sure.

[dlebauer]

@TinoDornbusch The calibration parameters have been in the metadata since at least 08/28, but were not there on 08/03.

When were the files changed from mK to radiance? Is this information included in the metadata?

[TinoDornbusch]

Internal FLIR calibration has been changed to raw at 15.9.2016....Do you want us to enter a note in sensor_fixed_metaData, such as:

"sensor_fixed_metadata": { "comment sensor output": "Sensor output with internal FLIR calibration in 100mK until 14.9.2016, since 15.9.2016 without calibration in raw 16bit unsigned integer",

[TinoDornbusch]

@dlebauer PLease specify which notification on FLIR Metadata you would like to indicate the change in output data format.

[ZongyangLi]

Uploaded python script to convert flir binfile

[dlebauer]

@ZongyangLi could you suggest a solution to @TinoDornbusch (see two comments starting with https://github.com/terraref/computing-pipeline/issues/151#issuecomment-247756163) regarding how to describe in metadata the calibrated vs. uncalibrated data (presumably in a way that your script can handle)?

[ZongyangLi]

@dlebauer Is that means @TinoDornbusch can provide a parameter in json file indicates whether data in binfile is a 'calibrated' or 'uncalibrated'? If yes, my suggestion is

"sensor_fixed_metadata": {
"output type": "calibrated" / "raw"
}

Is that make sense?

And thanks for reminding that the calibration metadata is in the new json file, I will update the script

[dlebauer]

@ZongyangLi how about

"sensor_fixed_metadata": {
"calibrated": "true"
}

[ZongyangLi]

@dlebauer That's good for me.

[TinoDornbusch]

To confirm: You want us to put in all sensor metadata a statement, whether it is calibrated.

"sensor_fixed_metadata": { "calibrated": "true" or "false" }

[dlebauer]

I think it is only necessary for FLIR. On Thu, Oct 6, 2016 at 3:54 AM TinoDornbusch notifications@github.com wrote:

To confirm: You want us to put in all sensor metadata a statement, whether it is calibrated.

"sensor_fixed_metadata": { "calibrated": "true" or "false" }

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/terraref/computing-pipeline/issues/151#issuecomment-251904261, or mute the thread https://github.com/notifications/unsubscribe-auth/AAcX58slA_axG7RVymwf-jfcNxyNTL_Zks5qxLc4gaJpZM4Jo3xN .

[max-zilla]

@ZongyangLi the script looks good. I think it should be fairly easy to put this into an extractor for Clowder. Basically you just need to add a bit of python code using pyClowder. An example is how I took this python demosaic script: https://github.com/terraref/computing-pipeline/tree/master/scripts/stereoImager ...and added extractor files to call that script when files are uploaded into Clowder: https://github.com/terraref/extractors-stereo-rgb/tree/master/demosaic

• config.py and extractor_info.json have some parameters of the extractor, you'd probably just need to change extractorName and outputDir, and make a new variable like bin2tiffScript that points to your FLIR code
• terra.demosaic.py is the extractor code. This has 3 parts:
  • main() which i dont think you need to change
  • check_message() which gets notified when a file is added to a dataset, and gives you some parameters about the file/dataset. you can check here whether you have an FLIR file you want to process. One way could be checking dataset name.
  • process_dataset() gets called if check_message() returns true, and allows you to process the files. Paths to the files will be in paramaters. Basically here you could call get_flir(input, output) in Get_FLIR.py

I think it would be a good exercise to try writing this extractor, or at least a first version of it I can help clean up. You could use my demosaic version as a template. You can learn more about extractors here:

• https://opensource.ncsa.illinois.edu/confluence/display/CATS/Documents has a video and slides for a tutorial in writing extractors and setting up a test environment

I or possibly @robkooper can help answer questions as you go, but getting this started would help move this along while I finish up other things in the meantime. Please take a look and let me know what you think.

[ZongyangLi]

@max-zilla Thanks! I will follow your advise to put it into extractor.

@dlebauer mention that this does not need an extractor, so I didn't upload a clowder extractor script. If an extractor is needed, I will follow the examples in demosaic to write a first version of flir extractor.

[ZongyangLi]

@max-zilla Hi, I am testing flir extractor in my laptop, but the clowder develop version in my laptop do not support the input of 'mountedPaths'. Could you give me a new version that I can test in my laptop?

[max-zilla]

@ZongyangLi the mountedPaths is only necessary for local paths - you could remove it and add it back later if that solves the error. I can help with that part at the end.

But if you want to fix, I think you just need to install the branch of PyClowder where I added the mountedPath support (which lets extractors locate files on locally mounted directories instead of downloading if possible). Clone this repo (or go to where you've cloned it previously): https://opensource.ncsa.illinois.edu/bitbucket/projects/CATS/repos/pyclowder then

git checkout local-mount-path-downloading
python setup.py install

This will install my branch which knows how to handle the mountedPaths.

You probably don't need to modify Clowder itself unless you're running a very old version.

[ZongyangLi]

@max-zilla This first version of flir extractor works in my testing environment now. Could you please help me checking whether it works in other environment?
https://github.com/terraref/computing-pipeline/tree/demosaic_extractor/scripts/FLIR

[robkooper]

@max-zilla do you think you can maybe take the wordcount extractor from pyclowder2 and use that to write a a single page here on the wiki how to get started.

[dlebauer]

@max-zilla by wiki I think Rob is referring to https://github.com/terraref/documentation/blob/master/developing-clowder-extractors.md

[max-zilla]

Started working on documentation on the Clowder wiki that we can link to from TERRA documentation so we don't have to write it twice: https://opensource.ncsa.illinois.edu/confluence/display/CATS/Extractor+Development

@ZongyangLi I will test out the extractor before our meeting on Thursday this week. Thanks!

[max-zilla]

@ZongyangLi do you know when your test data is from? I downloaded a flirIr dataset from Oct 11 (2016-10-11__08-26-24-955) and got errors running extractor:

getting information from json file
Metadata file missing key: position x [m]
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/usr/local/lib/python2.7/site-packages/pyclowder/extractors.py", line 350, in thread_function_dataset
    _processFileFunction(jbody)
  File "terra_flir.py", line 116, in process_dataset
    center_position, scan_time, fov = getFlir.parse_metadata(metadata)
  File "Get_FLIR.py", line 366, in parse_metadata
    position = [float(gantry_x), float(gantry_y), float(gantry_z)]
UnboundLocalError: local variable 'gantry_x' referenced before assignment

This is due to the obviously missing data in metadata file:

"gantry_system_variable_metadata": {
      "only small set of meta data available": "measurement was done during sensor setup",
      "timestamp": "10/11/2016 08:26:24",
      "PLC control not available": "no gantry position data"
    },

Have you heard anything about PLC control not available? I'm wondering if I missed a change or news that something was temporarily down...

(for this wondering, the specific code that failed is:

try:
  gantry_meta = metadata['lemnatec_measurement_metadata']['gantry_system_variable_metadata']
  gantry_x = gantry_meta["position x [m]"]
  gantry_y = gantry_meta["position y [m]"]
  gantry_z = gantry_meta["position z [m]"]
except KeyError as err:
  fail('Metadata file missing key: ' + err.args[0])
...

)

[ZongyangLi]

@max-zilla Sorry, I didn't notice these missing data in metadata, my bad.

I have no idea about these"PLC control not available", I think I should add some code to avoid these fail.

[max-zilla]

@ZongyangLi seems like we SHOULD fail there, at least more gracefully. we can't generate the geotiff without position right?

[ZongyangLi]

@max-zilla Yes, you are right. We can't generate the geotiff without Gantry position.

Should we just return "empty" instead of letting the extractor fail?

[max-zilla]

@ZongyangLi I think we want to fail on the extractor... unless we want to generate the PNG images, doesn't look like those need position.

Not really sure what else we can do, if the PLC control was missing when it was captured I don't know how we can go back and recover that information later that will allow us to re-run the extractor once it's fixed. I think all we can do on TIFF is fail.

[max-zilla]

@jdmaloney I have created a VM for this extractor on Roger.

• IP 141.142.170.73
• reads:

/sites/ua-mac/raw_data

• writes:

/sites/ua-mac/Level_1/flirIrCamera

Can you possibly set up the bindings before you're out at the end of the week? Thanks.

[jdmaloney]

@max-zilla Yeah, I'll get this done here today or tomorrow. I'm prepping the ROGER system today for it's maintenance tomorrow, I'll work to get this in there as well.

[max-zilla]

We will deploy with just PNG functional and can generate the TIF later.

POSSIBLE SOLUTION: flirIr camera is running at the same time as stereoCamera so we can compare timestamps and get location from stereoCamera if we have a match. thanks @solmazhajmohammadi

[max-zilla]

@jdmaloney today while you create exports for the FLIR export up there, can you also create exports for the new geospatial VM I created?

IP - 141.142.170.182

reads

/sites/ua-mac/raw_data

This doesn't need to write anything, it pushes metadata to Clowder and geostreams.

Thanks, I can bug your neighbor Chad about these things next week if he's the right person too and you're out of time.

[max-zilla]

@ZongyangLi I made some modifications to address the missing data issue above: https://github.com/terraref/extractors-multispectral/pull/2/files

Basically just return None for the parse_metadata() results, and in the extractor if None is returned we skip the GeoTIFF portion. The PNG file is still generated and uploaded. I added this check to your get_flir() method as well so this error should be avoided even when run from command line.

I sent a message to @dlebauer asking to create the necessary output folder on Roger, then I'll go ahead and merge the PR + deploy it.

[ZongyangLi]

@max-zilla I have review those codes. it's great! Thanks!

[max-zilla]

Extractor has been deployed and a test extraction was verified: https://github.com/terraref/extractors-multispectral

New FLIR outputs will appear in ua-mac/sites/Level_1/flirIrCamera as they arrive via Globus. Some point soon we'll initiate processing of backlog for all these extractors.