fix tall trait data date format and populate NA weather data dates

[rbartelme]

Problem Scope:

After running the weather data the bnlearn ingestion script and generating the combined trait+weather tsv file here, I noticed two things:

1. Date values in tall trait datasets (YYYY Month_abbreviation dd) are a different format than the dates in the weather data (yyyy-mm-dd)

2. Date ranges are slightly off between trait and weather data (or at least that's what's happening on my end with tidyverse+lubridate

---

Examples from my data

date	location
2018-04-19	mac
2018-04-22	mac
2018-04-23	mac
2014-10-16	clemson

This looks like mac_season_6 and when I look for the minimum date in the data I have, I get the following: > min(raw_weather_data$mac_season_6_weather$date) [1] "2018-04-25"

When I find the maximum date on the Clemson dataset, I get the following: > max(raw_weather_data$clemson_weather$date) [1] "2014-10-15"

---

Task Abstraction:

Make sure we can interoperate the datasets using the same date format.

---

Trait data

• [ ] Change date format in the tall trait data to match the date format in the weather data (see comment in Problem Scope for examples)
• [ ] Update trait notebooks

Upload updated trait datasets to CyVerse

• [ ] mac_season_4_tall

• [ ] mac_season_6_tall

• [ ] ksu_tall

• [ ] clemson_tall

• [ ] Please post updated tall trait dataset CyVerse urls as a comment here

---

Weather Data

• [ ] check the input/output for mac_season_6 and clemson for the dates listed in Examples from my data
• [ ] If the dates are missing, please update the notebooks/datasets on CyVerse
• [ ] If datasets are updated, please post new dataset url's here
• [ ] Else Ignore this, strike out the If tasks above, and let me know that R is being cranky

[MagicMilly]

• I used 2018-04-25 as the first date for the MAC season 6 weather data because it is listed here as the planting date.
• I re-downloaded season 6 trait data today and found 84 observations with a date occurring before the day of planting. They were all canopy height measurements (with mean as the measurement in cm), so that seems like an error in the database.
• If this is what you wanted me to do, I'll continue investigating the other issues. If you had something else in mind, let me know.

[rbartelme]

@MagicMilly Thanks for looking into this, this indeed what I had in mind. It does seem like a database error maybe something that needs to be addressed in BetyDB?? (@dlebauer ??)

• The Clemson dates are at the end of that season.
• There was not a database error option in the weather data portion of this issue
• May be able to ignore the date format change now that we're pretty confident there may be erroneous measurements in the db

[dlebauer]

What are the actual values of canopy cover prior to planting? if they are ~0 then that is sensible

[MagicMilly]

@dlebauer The canopy height values in cm are in the column mean, so they are shown in my screenshot above as being 9, 10, 11, 12, 13 cm spread across 84 observations before the planting date.

[Chris-Schnaufer]

@MagicMilly What's the status of this? Can it be closed?

[MagicMilly]

I'm not sure about the status - @dlebauer discovered some errors in the database, so those will probably need to be resolved as a follow-up ticket. I don't believe @rbartelme will be working on this issue further.

[dlebauer]

This looks like an artifact of the algorithm used to measure canopy height. I have marked suspect data as incorrect in the database; please remove all Season 6 canopy height measurements collected before May 15.

I've created a new issue for this: https://github.com/genophenoenvo/terraref-datasets/issues/123

[Chris-Schnaufer]

@dlebauer Would you like the work per your last comment to be a separate issue, or continue on this issue? https://github.com/genophenoenvo/terraref-datasets/issues/106#issuecomment-724244443

[dlebauer]

closing; follow up in https://github.com/genophenoenvo/terraref-datasets/issues/123