Processing TEROS data

[ahopple]

Hi @bpbond!

I have written a script that visualizes the TEROS 12 data for each logger (9 total; 3/plot) and was hoping you could provide me some feedback on my method.

Files can be found here: TEMPEST/Data/Soil VWC, T, and EC (TEROS12)/ and my script is called "plottest.R"

Overview of the data structure:

• TEROS 12 sensors record soil volumetric water content, temperature, and electrical conductivity (138 sensors total)
• They are spatially distributed across the plots using the grid system (map shown in TEROS 12 folder)
• Each data logger services 14, 15, or 17 sensors depending on its location
• The output format from a logger is "Teros(1,1)", "Teros(1,2)", "Teros(1,3)". "Teros(2,1)"...."Teros(22,3)"
• (n,1)=volumetric water content (requires conversion to be meaningful, m3/m3); (n,2)=temperature (C); (n,3)=electrical conductivity (uS/cm)
• Data table channels differ for each logger and can be found here: https://docs.google.com/spreadsheets/d/1IFHNaE4Tr45rjDNR7MBg7i2Z2ewIPyAhl6q2vCB_Tbw/edit?usp=sharing
• Monthly data for each logger are archived here: https://www.dropbox.com/sh/qc5vli7hgz65qtv/AADhO_BW0_LLkdfXZudCaLxFa?dl=0

What I have done: -download archived and current data for each data logger -delete rows 1, 3, and 4 (sensor ID info, cannot be loaded into R without deleting) and save as text file to logger folder on TEMPEST repo -For each logger, create a list of text files in that folder and use bind_rows to merge all files together -Create separate data frames for VWC, Temperature, and EC -Plot data and note sensor ID and data gaps for each data table channel -Clear R environment between data loggers and use same code

Does this make sense to you? Is there a better way to streamline this?

Also, you can check out the summary graphs for each data logger in "TEROS_Update_02272021,pdf", located under TEMPEST/Data/Soil VWC, T, and EC (TEROS12)/Updates/

Thank you and let me know if you have any questions!

[bpbond]

Hi @ahopple 👋

Wow, those summary PDFs looks great! Nice job. Nonetheless, looking through your script, I think we can make it much shorter, clearer, and more robust.

First question:

-delete rows 1, 3, and 4 (sensor ID info, cannot be loaded into R without deleting) and save as text file to logger folder on TEMPEST repo

We'd really prefer not to do this, because then we have to hand-modify the downloaded files every single time. Plus, we'd like commits of the raw (untouched) files. For these reasons, I'd prefer to have R handle the raw files as necessary. Do you have copies of the raw files? Could you send me an example and/or commit them?

Second, you're hand-specifying the location of the various sensors in the script, which makes for a lot of code duplication. Do you have a list of the sensor locations in table form? Ideally the script just reads that table, merges it with the Teros data, and proceeds on its merry way.

Third I'd like to go over reshaping data frames for efficient plotting :) but we can discuss that later.

Great start! Thanks for touching base about this.

[bpbond]

@ahopple It might be useful to set aside some time to go over coding specifically (as opposed to science). Lmk your thoughts.

[ahopple]

@bpbond Completely agree on setting aside time to go over coding. I would like to use tomorrow's meeting to go over some specific questions from recent transplant and tempest analyses. I'll open a new issue under the transplant repo that details my questions. Does that sound good?