search

NEONScience / NEON-utilities

Utilities and scripts for working with NEON data. Currently: an R package with functions to join (stack) the month-by-site files in downloaded NEON data, to convert data to geoCSV format, and to download data from the API.
GNU Affero General Public License v3.0
57 stars 36 forks source link

Data cannot always be uniquely tied to the sensor that collected them in loadByProduct #86

Closed rfiorella closed 4 years ago

rfiorella commented 4 years ago

Not a bug in the code per se, but likely an instance of unintended/unexpected behavior due to a "bug" in the data.

Function

loadByProduct

Describe the bug Data.frames returned by loadByProduct are occasionally insufficient to connect observations to the sensors that made them (e.g., occurs when there is more than one sensor with the same horizontalPosition and verticalPosition values)

To Reproduce soilhf <- neonUtilities::loadByProduct("DP1.00040.001",site="ORNL",startdate="2018-01",enddate="2018-12",avg=30,check.size=F)

Expected behavior Function returns the expected nested list with 4 sub-lists: SHF_30min, readme_00040, sensor_positions_00040, and variables_00040. However, ORNL has two separate sensors that have horizontalPosition and verticalPosition values of 005 and 501, but are located at two different depths. Unless I'm missing something obvious, based on the information provided in these dataframes, I don't see a way to merge these data frames to select one depth or the other for this product, as depth information is only provided in sensor_positions_00040, but not SHF_30min.

System (please complete the following information): OSX 10.15.3, R 3.6.2, neonUtilities 1.3.3

cklunch commented 4 years ago

@rfiorella If I'm understanding correctly, I think this is about how to interpret the sensor_positions file. There are two entries in sensor_positions_00040 for ORNL HOR.VER=005.501, but they aren't two separate sensors. The first entry has an end date in Sept 2017, which is the start date of the second entry. So this is indicating that in 2017 we moved the soil heat flux plate.

We're always working on data product documentation, thanks for the alert that this is an area for improvement!

rfiorella commented 4 years ago

Ah, you're correct! Didn't see the 'end' before, but this change is clear now.

Thanks @cklunch!