COMPASS-DOE / data-workflows

Sensor data workflows and processing scripts
MIT License
4 stars 0 forks source link

BBL L1a confusion #16

Closed bpbond closed 1 year ago

bpbond commented 1 year ago

Hi, here are some examples of what I'm getting from @selinalcheng latest email. Is this correct?

PLEASE NOTE these are NOT intended to be complete tables--I understand there's other info (e.g. flags). I'm trying to only show the core data and data structuring that distinguishes L1a from L1b. Thank you! 🙏

Example 1: sensor specific to a tree

L1a

Site Plot Tree name value
x y z sapflux n.nnn
x y z sapflux_deep n.nnn

L1b

Same data as L1a, just restructured:

Site Plot Tree sapflux sapflux_deep
x y z n.nnn n.nnn

Example 2: sensor in a plot

L1a

No "Tree" entry, because one teros sensor might be mapped to multiple trees?

Site Plot name value
x y teros_sm_1 n.nnn
x y teros_temp_1 n.nnn

L1b

Straightforward:

Site Plot Tree sapflux soil_moisture soil_temp
x y z teros_sm n.nnn n.nnn

Example 3: site-level sensor

L1a

Just this?

Site name value
x PAR n.nnn

L1b

(PAR gets pulled into whatever tree or whatever wants it, basically, via plot table?)

bpbond commented 1 year ago

Just realized I'm not showing timestamps above. My understanding is that:

selinalcheng commented 1 year ago

Based on Stephanie's presentation, it looks like L1a and L1b differ only by data table format (long v. wide) and by timestamp (rounded v. original). Is that right?

In your latest email, you said L1a was long format and not joined with the plot table. Maybe we are using different terms again, which is causing confusion. When I say "plot table" I mean the experimental design information, which Steph has included in her L1a box on the slide that you linked, and which you have included in your L1a GitHub example.

In our data workflow, we do not save the long format table -- instead we reshape to wide and save that because it is more useful for our analyses. I see this difference in data table shape as pretty trivial, since the data should be able to be refolded/reshaped easily in code, but I would be happy to discuss this in a call. I just sent you an email to schedule a meeting with Roy as well

bpbond commented 1 year ago

Okay, thank you @selinalcheng -- much appreciated.