Upload tractor data to BETYdb

[dlebauer]

TODO

• [x] make sure variables are defined in terraref BETYdb (if they do not exist
• [x] upload csv files to google docs
  • we can discuss formatting here
  • see documentation for bulk uploads, especially the traits.csv template

[Mamatemenrs]

Tried to define a variable in BETYdb, not sure if it is an access issue I could not define. Though, the csv file is shared through google docs. @dlebauer @remotesensinglab

[dlebauer]

@Mamatemenrs did you

1. check this variables list https://terraref.ncsa.illinois.edu/bety/variables
2. try to add variables to https://terraref.ncsa.illinois.edu/bety/variables/new?

If you still have access permissions we can take a look. But you should have permission to 'create' records. If you need to modify existing records I can increase your permissions. But if you still can't create a new variable, it is a bug in BETYdb

3. Where is the google doc?

[gsrohde]

@Mamatemenrs I tried temporarily downgrading my permission levels to match yours. I could access the "New Variable" edit page with no problem. I didn't actually try creating a new variable. What do you see when you try to define a variable? What was the name of the variable you tried to define?

[Mamatemenrs]

Added two variables canopy_temperature and tractor_canopy_height. I shared the CSV file to your email via google docs dlebauer@illinois.edu @dlebauer . Anyway, here is link to the file https://docs.google.com/spreadsheets/d/1NNKtmWFBdoh2iaaXZh9fsu_nNRLMpytJ8Av1bHxpgIg/edit?usp=sharing

[gsrohde]

@Mamatemenrs Looking at your data file, I thought I should make sure you understand that the variable names in the column headings must match the variable names in the database exactly. In particular, since the units are not part of the name, they shouldn't be in the heading either. This is all explained here: https://pecan.gitbooks.io/betydbdoc-dataentry/content/bulk%20upload.html.

@dlebauer Perhaps we might want to reconsider this and either allow or require the units to be in the heading as a way of helping ensure the data is interpreted in the way the user expects it to be interpreted.

[dlebauer]

@gsrohde I think for now enforcing standard units and scales is okay, it could be a quagmire to do on-the-fly conversions. But perhaps providing users with a notification of the units that the database assumes after the bulk upload is tested would be useful

@Mamatemenrs I see you created the variable "tractor_canopy_height". I am thinking that these data could be stored as "canopy_height" with the method field used to indicate that it was collected using some specific sensor mounted on a tractor. And for canopy temperature, we should also add a method. Could you please create methods records for these the sensors used for these two datasets (terraref.ncsa.illinois.edu/bety/methods/new) and then @gsrohde can update the spreadsheet accordingly?

[Mamatemenrs]

@gsrohde my bad. forgot to delete the units with parentheses from the CSV file. I'll correct that. @dlebauer I created "tractor_canopy_height" in order to differentiate the variable from the existing "canopy_height" variable, I will update. Can you tell me how to add a new field such as "method" to an existing variable in BETYdb ?.Or I can add measurement description to "Notes" field.

[dlebauer]

@Mamatemenrs you can add a new method here: terraref.ncsa.illinois.edu/bety/methods/new

We are working on the ability add methods in the bulk upload, but for now just provide a link to the new method record that you create. Here is an example: https://terraref.ncsa.illinois.edu/bety/methods/6000000003

[Mamatemenrs]

The variable names in the CVS file updated (please see the link below). I will add methods for tractor data. https://docs.google.com/spreadsheets/d/1NNKtmWFBdoh2iaaXZh9fsu_nNRLMpytJ8Av1bHxpgIg/edit?usp=sharing

[remotesensinglab]

Below is the methodology for the tractor data. We can close this issue if there are no further comments.

The phenotractor was equipped with three types of sensors for measuring plant height, temperature and canopy spectral reflectance. A RTK GPS was installed on top of the tractor, see the Figure 1.

Figure 1. Phenotractor system configuration

The distance from canopy to sensor position was measured with a sonar proximity sensor (Soutput, in mm). Canopy height (CH) was determined by combining sonar and GPS elevation data (expressed as meter above sea level). An elevation survey was conducted to determine a baseline reference elevation (Eref) for the gantry field. CH was computed according to the following equation: CH = Es – Eref - Soutput

where Es is sensor elevation, which was calculated by subtracting the vertical offset between the GPS antenna and sonar sensor from GPS antenna elevation.

Infrared radiometer (IRT) sensors were used measure canopy temperature and temperature values were recoded as degree Celsius (°C).

Canopy spectral reflectance was measured with GreenSeeker sensors and the reflectance data were used to calculate NDVI (Normalized Difference Vegetation Index). GreenSeeker sensors record reflected light energy in near infrared (780 ± 15 nm) and red (660 ± 10 nm ) portion electromagnetic spectrum from top of the canopy by using a self-illuminated light source. NDVI was calculated using following equation: NDVI=(ρ_NIR-ρ_red)/(ρ_NIR+ ρ_red )

Where ρ_NIR and ρ_red represent fraction of reflected energy in near infrared and red spectral regions, respectively.

Georeferencing

Georefencing was carried out using a specially developed Quantum GIS (GGIS, www.qgis.org ) plug-in by Andrade-Sanchez et al. (2014) during post processing. Latitude and longitude coordinates were converted to UTM coordinate system. Offset from the sensors to the GPS position on the tractor heading were computed and corrected. Next, the tractor data, which uses UTM Zone 12 (MAC coordinates), was transformed to EPSG:4326 (WGS84) USDA coordinates by performing a linear shifting as follows:
Latitude: Uy = My – 0.000015258894 Longitude: Ux = Mx + 0.000020308287 where Uy and Ux are latitude and longitude in USDA coordinate system, and My and Mx are latitude and longitude in MAC coordinate system.

Finally, georeferenced tractor data was overlaid on the gantry field polygon and mean value for each plot/genotype was calculated using the data points that fall inside the plot polygon within ArcGIS Version 10.2 (ESRI. Redlands, CA).

Reference Andrade-Sanchez, Pedro, et al. "Development and evaluation of a field-based high-throughput phenotyping platform." Functional Plant Biology 41.1 (2014): 68-79.

[dlebauer]

@remotesensinglab thanks for posting the methods ... before closing please update the documentation https://github.com/terraref/documentation/edit/master/user/protocols-tractor.md (should be a simple copy-paste of what you've written, though you can use also use Latex syntax for the equations if you prefer. )

[Mamatemenrs]

@dlebauer @remotesensinglab made 2 commits on Update protocols-tractor.md , waiting to be approved.

[dlebauer]

@Mamatemenrs I've merged your changes https://github.com/terraref/documentation/pull/108 thanks!