Neon vertical profile for SCBI

[mcgregorian1]

To write what @teixeirak and I spoke about:

The following traits have NEON measurements at the different tower heights (10,20,30,40,50m)

• wind speed and direction (2D)
• single aspirated air temperature
• IR biological temperature
• relative humidity

I will get each of these into R, and get a zoomed-in graph of each for June 2018 to see if we can see any trends for the different heights. If we want to move forward with that, then I can also use the shortwave radiation measurement from NEON to create a threshold of what is considered a cloudy/sunny day. From there, we can give a height profile for different weather days,

[teixeirak]

We don’t necessarily need to separate by sunny/ cloudy days, although it may be interesting and informative.

[mcgregorian1]

Wind speed: the trend here is what we'd expect, with speed increasing the further up in the canopy you go.

Air temp: from 27 june on, 20m has consistently higher temps. maybe because not as much wind? could also be anomalous sensor given the temps match up for the 30m,40m, and 50m sensors.

Relative humidity: RH values were taken only at 0m and 60m, with the data overlapping only from March to mid June. There is an easily discernible trend whereby RH at ground level is often higher than RH at 60m.

Biotemp: somewhat of a trend, where the ground level infrared temp stays more constant and within a more compact range compared to further up in the canopy

[mcgregorian1]

@teixeirak

This is the distribution of the total radiation per day from May-August 2018. Do you know of a mathematically sound way to determine what would be a "cloudy" day? I'm having trouble thinking of something, plus NEON doesn't have data for 41 days during this time period

[teixeirak]

The 0 values obviously aren't trustworthy. I don't have the typical range of values for radiation in my head, but given that 0 is clearly wrong, can you please check the data quality of other days?

It seems that the data quality is substantially less than that of our ForestGEO weather station? We could use that instead.

[mcgregorian1]

Of course. Looking at the data for 2018 I know those to be fluke points.

For 2017 (may-august), we have the following distribution:

This is the data from our weather station for 2018. The x-axis is different since the radiation units are different, but ultimately the math problem of figuring out where to put that threshold of sunny vs cloudy remains.

[mcgregorian1]

I think I've figured it out.

With NEON data, I have both the direct and diffuse radiation (the graphs below are for May-August 2018). The way that I understand this is I would add the two together (as we see in the graph) to get the total radiation per day, then set a threshold and say e.g. "if difRad (diffuse) accounts for at least 50% of the total radiation that day, then that day is considered cloudy".

The other way of doing this would be to look at a graph like this (using the same data) and say, "any day that had <10000 w/m2 will be considered cloudy." This would help if using SCBI data, since SCBI met tower does not differentiate between diffuse and direct radiation. In my opinion, though, this is a little less robust. Ignore the 0 values here, those are the NAs.

So 2 things:

• NEON data is randomly missing, but is easier to more accurately assign cloudy vs sunny days.
• SCBI data shows barely any days <10000 w/m2 for 2018, despite last year we had a lot of rain. We have more days with data than NEON, but I'm not sure then if the 10,000 threshold is accurate for this data (notice the y-axis difference between this and the first graph above).

What do you think?

[mcgregorian1]

Here is an example of what a graph would look like, not separating out for cloudy/sunny days. This is windspeed in 2018 from NEON

[teixeirak]

Good start! Let's put them all on one panel.

Any idea why 20m is higher than 30m? I wonder if there's something strange with the 20 or 30 m sensor. (If you can identify one of the two as suspicious, we can just drop it.)

[teixeirak]

Regarding direct/diffuse and total rad, those data look quite suspicious. Can you please look for some reliable published source as to what would be typical?

[mcgregorian1]

Regarding direct/diffuse and total rad, those data look quite suspicious. Can you please look for some reliable published source as to what would be typical?

Do you say it's suspicious because of the numbers? Do you think both are suspicious or just the NEON (direct/diffuse)?

I've been reading through some papers, and the way that annual mean shortwave radiation is calculated is by day (Above I was doing total per day). This yields a graph like this:

Same thing but for our met tower:

Otherwise, I'm not sure how to go about this.

• a sample NOAA dataset showing over the course of a year
• Hatzianastassiou et al 1999 show sw radiation never getting above 400.
• Ruzmaikan et al 2015 give a mean annual value of 339.85 w/m2 with a max of 460.
  • these values of course are averaged across the entire year

NEON does have a measure of sun presence (either a 1 or 0) which I can use to give an estimate of how cloudy a day was, and we could go from there. However, I'd still be facing the random lack of data from the sensors (e.g. 25 June - 15 July 2018)

[teixeirak]

I was surprised at it being so variable. I don't have much experience looking at this type of data, and don't have numbers as to what is typical in my head. I'll call you about this.

[teixeirak]

Resolution: for now, we won't try to separate sunny and cloudy days. We can always return to this later if we decide it would be really interesting.

[mcgregorian1]

For the record, this is the neon data with only full data days.

[mcgregorian1]

Hi @teixeirak

Here is a consolidated graph for the vertical profile for 2018, excluding radiation (since we were only focused on that for determining sun/cloud thresholds). Mean air temperature includes night temps as well.

For the paper, is it ok that we only include 2018? I'm not sure of the reasoning for including more years since we'd be using this as representative anyways.

[teixeirak]

Looks good in general. A few things: 1- Do you trust all the data? The June T_air looks funny, and I also wonder a little about relative humidity (why does August look so different)? 2- I think it would be more informative to mean max and min daily temperatures. We might see how this looks for other variables as well. 3- let's put height on the y-axis 4- I think its fine to show 2018 only, so long as the data are good. If we have data gaps effecting this patterns, it would help to bring in other years

[mcgregorian1]

1. In terms of trusting the data, I think the question is at what point do we not trust it? I'm still seeing increased values at 20m for air temp, wind speed, and bio temp in the 2017 data. Since that's at least two years showing that trend, I would say then that it's sound?

What do you think?

2. That's fine. I've tried this with the other variables and it works well.

3. I'm not sure height on the y-axis makes for a better picture. Here's a comparison of the two, personally I'm drawn to height on the x-axis

4. I think in general 2017 would be better to use. I could potentially put all the years together but it'll take me a bit to figure out how to bring that all together.

   • in 2018:
   • air temperature 10m sensor is unusable
   • windspeed missing for 15-17 May for all heights
   • biotemp missing for 15-17 May all heights, and the 10m sensor missing 21-29 August
   • RH missing data from 17 June to 19 August (either the 0m, the 60m, or both sensors)

• in 2017:
  • wind speed is missing from 15 May to 7 June for 20m sensor
  • RH is missing at the 60m sensor from 1-21 May

[mcgregorian1]

Here are the four variables with 2017 data

[teixeirak]

3\. I'm not sure height on the y-axis makes for a better picture. Here's a comparison of the two, personally I'm drawn to height on the x-axis

The reason it looks bad with height on the y-axis is that its connecting the points in order of their x-value, whereas when you switch the axis you'd also want to switch to connecting in order of their y-value. I think that if you can make that work, it would be quite a bit more intuitive.

[teixeirak]

4\. I think in general 2017 would be better to use. I could potentially put all the years together but it'll take me a bit to figure out how to bring that all together.

I agree. However, the best would likely be to combine the two years.

[teixeirak]

2. That's fine. I've tried this with the other variables and it works well.

I like that! Can we combine day and night in a single panel with different line types?

[teixeirak]

The one substantive difference between the two years seems to be that the biological temperature increased with height in 2018 but not in 2017. Any idea why?

[teixeirak]

1. In terms of trusting the data, I think the question is at what point do we not trust it? I'm still seeing increased values at 20m for air temp, wind speed, and bio temp in the 2017 data. Since that's at least two years showing that trend, I would say then that it's sound?

It can be tricky. Part of the motivation for including multiple years in the average is that it will help to wash out values that may be slightly bad, or instances where there are misleading means because too much data is missing (e.g., probably May RH at 60 m in 2017).

[mcgregorian1]

I like that! Can we combine day and night in a single panel with different line types?

I could, but how would we differentiate between night and day? There would either have to be a threshold that includes some anomalies, or figure out the rolling sunrise and sunset times, which may take a while to figure out. Another solution for this is I can potentially draw a line on the graphs that indicates when sunrise and sunset is based on the values.

As I'm thinking about this more I'm wondering how this would make sense given that we're already splitting the max and min into different panels? For example, with air temperature, the max values will always be felt during the day and the min values will always be felt at night, so it doesn't really make sense to be adding in the night/day dimension, in my opinion

[mcgregorian1]

The one substantive difference between the two years seems to be that the biological temperature increased with height in 2018 but not in 2017. Any idea why?

The Infrared bio temp is defined by NEON as "Biological temperature (i.e., surface temperature) is measured via IR temperature sensors located in the soil array and on the tower infrastructure," thus the heat of the actual leaves on the trees, I'm guessing.

Looking at the two years' graphs, overall the mean temp (air and surface) was lower in 2018 than 2017, probably due to the increased rain in the former. Thus it seems the increase we see in biological temp with height is a function more of the temperature difference between ground and canopy than it is a common trait of a forest. In my mind, this could make sense because if you have more moisture than normal, you'll have more days where the understory wouldn't have dried out yet, whereas the canopy will be exposed directly to sun and would heat/dry out faster. When you don't have a lot of rain, like in 2017, there would be less variation in temperature from ground to canopy and not as much moisture to separate out leaf temperatures.

[mcgregorian1]

4. I agree. However, the best would likely be to combine the two years.

Ok. Is there a reason then we shouldn't just do the last 3 years? It's more data for me to go through, but in the end it's simply mashing different files together and running the same code.

3. The reason it looks bad with height on the y-axis is that its connecting the points in order of their x-value, whereas when you switch the axis you'd also want to switch to connecting in order of their y-value. I think that if you can make that work, it would be quite a bit more intuitive.

Ok! Turns out it was a formatting thing with dplyr.

Here is the graph with the combined years 2017 and 2018.

[teixeirak]

I like that! Can we combine day and night in a single panel with different line types?

I could, but how would we differentiate between night and day? There would either have to be a threshold that includes some anomalies, or figure out the rolling sunrise and sunset times, which may take a while to figure out. Another solution for this is I can potentially draw a line on the graphs that indicates when sunrise and sunset is based on the values.

As I'm thinking about this more I'm wondering how this would make sense given that we're already splitting the max and min into different panels? For example, with air temperature, the max values will always be felt during the day and the min values will always be felt at night, so it doesn't really make sense to be adding in the night/day dimension, in my opinion

I'm sorry, I wasn't careful with terminology. I meant to combine min and max on one graph with different line types. I believe max values for all variables typically occur during the day, correct (so max and min is essentially the same as day and night)? In any case, just combine max and min on the same plots.

[teixeirak]

Ok! Turns out it was a formatting thing with dplyr.

Here is the graph with the combined years 2017 and 2018.

Looks good!

[teixeirak]

Ok. Is there a reason then we shouldn't just do the last 3 years? It's more data for me to go through, but in the end it's simply mashing different files together and running the same code.

3 years would be even better!

[mcgregorian1]

In answer to all 3 comments above, here is the combined graph (max and min) with all years 2016-2018. Note that RH wasn't measured in the 2016 growing season, and I've made a note of all NEON anomalies in a README here.

[teixeirak]

Okay, that looks good. We'll want to make a few formatting adjustments later, but the content is good.