Closed teixeirak closed 3 years ago
@mcgregorian1, this is the pertinent issue. Here's what I see as the steps:
To help with step 1, I loaded some files sent to me by NEON staff in 2016. (There may be a better alternative available now.) This can at least be used to identify sites with forest. Some sites have mixed forest and non-forest, and I'd guess that the towers would be within the forest, but we'll need to confirm this.
Here's NEON's sites page: https://www.neonscience.org/field-sites/field-sites-map. It looks like this will give the same info in the spreadsheet.
@NidhiVinod , perhaps you could work on this? Once we identify sites with at least some forest, we'll need to figure out the veg type in which the tower is located. If you can't find a convenient list, it looks like this info could be found on pages like this, or we can write to NEON. I'd recommend you spend ~ 1/2 hour - 1 hour looking for the info online, and if not readily available, we can write them.
@teixeirak, here are potential sites. Each of the sites below also have some vegetation description:
Ecosystem Structure: https://data.neonscience.org/data-products/DP3.30015.001
Thanks, @NidhiVinod. Is this the full set of sites that have vegetation description online? I think there are probably more than this. Based on this file, there are 33 sites with at least some forest.
I started this file to summarize sites/ organize all this info. The sites listed there are the complete list of sites from this file that include one of the three nlcd forest classes (deciduousForest, mixedForest, evergreenForest). Here's what we'll want to do to complete this:
1- list the nlcd classes (nlcd _class_1
..._2
, ..._3
), in order of area.
2- if all classes listed are forest, we know the site is in forest (all_forest
=1; e.g., BART), and this means the site can be included (include
=1). Eventually, we'll probably want to know which forest type the tower is in (nlcd_class_tower
), but that can come later if it's hard to dig up.
3- If not all classes listed are forest (all_forest
=0; e.g., SCBI and BLAN), then we need to know which nlcd type the tower is in in order to determine if the site should be included. In the examples I filled in, I know that the SCBI tower is in the forest. I'm not positive about BLAN, so that's one where we'd need to find additional info.
Could you please work on filling that in?
@teixeirak yeah, definitely! I'll work on filling them in.
@teixeirak: I have two questions--1. is a woody wetland categorized under forest? 2. one way I am able find nlcd_class for towers is looking at tower images and vegetation around on the specific sites, and also looking at close-by nlcd class for tower base plots--> is this an okay way to find the information for nlcd_class_tower or is there a better way? (I have updated some of the information on the forested_NEON_sites.csv)
Good question about woody wetlands. The one from that list that I'm familiar with is Harvard Forest, where it's sort of a swamp with shrubs and small trees. It probably doesn't matter much, as I don't think any of the towers would be in woody wetlands (unless that's all there is). So, I'd treat woody wetlands as forest.
I don't know if there would be a better way to get this info. Perhaps its best to write NEON folks to see what they can provide? I think there's some other info we'll need as well: canopy height and probably exact instrument heights.
@teixeirak : I filled the nlcd_class_tower in the document, it was really nice of Hongyan to provide this info so quickly. Thinking about what to include and what not to--sites that don't have all forest (all_forest=0), but have their tower located in a forest--> to include (include=1; otherwise 0)?
Also, should we have a column in the same document for-- DistZaxsCnpy | The average height (m) of the canopy above the ground surface and DistZaxsDisp | The vertical distance (m) at which the wind speed would go to zero? or maybe not? What do you think?
@teixeirak : I filled the nlcd_class_tower in the document, it was really nice of Hongyan to provide this info so quickly. Thinking about what to include and what not to--sites that don't have all forest (all_forest=0), but have their tower located in a forest--> to include (include=1; otherwise 0)?
Yes, the criteria for inclusion would be that the tower is in forest.
Also, should we have a column in the same document for-- DistZaxsCnpy | The average height (m) of the canopy above the ground surface and DistZaxsDisp | The vertical distance (m) at which the wind speed would go to zero? or maybe not? What do you think?
No need to transfer this info; we can read it directly from this file by script when we get to the point of needing it.
Thanks for your work on step 1, @NidhiVinod. This is done.
@mcgregorian1 , when you have a chance, you can get started on this using the list in this file, where the include
field indicates whether the site is appropriate (1=yes; 0=no).
We have 33 sites, so while we can start with just running your existing script and generating a bunch of figures, we'll ultimately need some creative way to condense all of this into one (multi-panel) figure. Here's a starting idea:
DistZaxsCnpy
in this file). That is, normalized height = measurement height / DistZaxsCnpy, so values <1 = below-canopy, 1=at mean canopy level, >1 = above mean canopy levelclimate.variable
]-- i.e., ∆climate.variable
=climate.variable (h)
-climate.variable (h_min)
, where h_min is the lowest height at which the measurement is made. Order of operations should be calculate ∆climate.variable before taking daily max/min and averaging.climate.variable
will need to be for a single defined time period. I'd suggest January and July for each variable. So panels could include Jan Temp, July Temp, Jan RH, July RH, Jan wind, July wind, etc.Sounds good. I can get to this this weekend.
@teixeirak which climate variables did you want to use - the same as my analysis? As in, wind, RH, Tair, and Tbiological?
Also in total we have 25 sites (include==1) as opposed to 33.
The way I understand this will work is
Obviously, doing this will mean a figure that is 25 rows of 4 plots, or 100 total plots. We could technically combine all sites into one per climate variable, but we have too many to effectively differentiate via color or shape/size (plus I think that would be too messy). I can go ahead with the 100 plots, but it will need to be written to pdf in order for the whole thing to be at a readable size (i.e. it will go over multiple pages).
@teixeirak which climate variables did you want to use - the same as my analysis? As in, wind, RH, Tair, and Tbiological?
Yes, all those. It would be nice to add solar radiation / PAR and CO2, if available.
Also in total we have 25 sites (include==1) as opposed to 33.
Oops, sorry!
The way I understand this will work is
* 1st row: Wind - RH - Tair - Tbio = 4 plots for BART * 2nd row: Wind - RH - Tair - Tbio = 4 plots for SCBI * ...etc
Obviously, doing this will mean a figure that is 25 rows of 4 plots, or 100 total plots. We could technically combine all sites into one per climate variable, but we have too many to effectively differentiate via color or shape/size (plus I think that would be too messy). I can go ahead with the 100 plots, but it will need to be written to pdf in order for the whole thing to be at a readable size (i.e. it will go over multiple pages).
We can start with this to get a sense for the patterns. Ultimately, we'll want just one graph that fits on a single journal page. We'll therefore need to either select examples or drop the concern of making all sites individually identifiable. If we color code by latitude or mean annual temperature or some other appropriate variable, I think this would give us a nice plot, even if sites can't be individually identified.
We can start with this to get a sense for the patterns. Ultimately, we'll want just one graph that fits on a single journal page. We'll therefore need to either select examples or drop the concern of making all sites individually identifiable. If we color code by latitude or mean annual temperature or some other appropriate variable, I think this would give us a nice plot, even if sites can't be individually identified.
That would work! We did something similar for a paper we're hoping to get out soon using the FLUXNET sites - way too many to individually list so it was all split to evergreen vs deciduous, and it's much more readable.
But sounds good, I'll start with the full plot and we can go from there.
@teixeirak what's the date range for all of this? Are we just looking at the most recent year (2019) for January and July? Or are we looking over multiple years?
Sounds great! I like the idea of splitting into major forest types (e.g., broadleaf evergreen, broadleaf dec, conifer, and mixed).
@teixeirak what's the date range for all of this? Are we just looking at the most recent year (2019) for January and July? Or are we looking over multiple years?
Either works, as this is largely an illustrative figure. I'd favor using all the years available.
@teixeirak some guidance please
NEON's download function does not allow for me to specify "get January and July data for all instances where it is available." Instead I have three options:
What do you advise?
Definitely not the first option! That would take up too much human time.
It's not a rush, and we shouldn't have to repeat the process often (if ever), so I think that the time required for the 2nd option is fine, and see that option as slightly preferred in that it will give a longer-term representation of climate and better statistical power.
If for some reason this is inconvenient for you, the 3rd option is also fine.
It's not inconvenient! It just means I'll need to have the mac plugged in and chugging away all night. I tried doing it last night but didn't realize the power would be drained so quickly...hence why I did the SCBI time tests this morning.
On Mon, Aug 24, 2020 at 10:28 AM Kristina Anderson-Teixeira < notifications@github.com> wrote:
Definitely not the first option! That would take up too much human time.
It's not a rush, and we shouldn't have to repeat the process often (if ever), so I think that the time required for the 2nd option is fine, and see that option as slightly preferred in that it will give a longer-term representation of climate and better statistical power.
If for some reason this is inconvenient for you, the 3rd option is also fine.
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/EcoClimLab/vertical-thermal-review/issues/2#issuecomment-679160806, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJNRBEKKCA2EWTABK3F4DELSCJ2JHANCNFSM4PNNWGDA .
--
Ian McGregor
Ph.D. Student | Center for Geospatial Analytics
He/Him/His
College of Natural Resources
Jordan Hall 4120 | Campus Box 7106
North Carolina State University
2800 Faucette Dr.
Raleigh, NC 27695 USA imcgreg@ncsu.edu | 714-864-1005 | geospatial.ncsu.edu
@teixeirak which climate variables did you want to use - the same as my analysis? As in, wind, RH, Tair, and Tbiological?
Yes, all those. It would be nice to add solar radiation / PAR and CO2, if available.
For CO2, NEON has a bundle of eddy-covariance data, including basic CO2 concentration. I'm assuming this is what you're talking about? neon. If you go to this site and type in "CO2" to the search filter, you'll see the different options - there are a couple that are a vertical profile.
I found PAR as well. Are there any other variables you want before I run the 12-hour script?
I think there should be solar radiation (shortwave, longwave). That would be good to grab as well.
- y-axis = height, normalized to canopy height (
DistZaxsCnpy
in this file). That is, normalized height = measurement height / DistZaxsCnpy, so values <1 = below-canopy, 1=at mean canopy level, >1 = above mean canopy level- x- axis = ∆[
climate.variable
]-- i.e.,∆climate.variable
=climate.variable (h)
-climate.variable (h_min)
, where h_min is the lowest height at which the measurement is made. Order of operations should be calculate ∆climate.variable before taking daily max/min and averaging.
@teixeirak to make sure I understand this, let's say for SCBI I have a windspeed at 10m height of 0.45 for 12-12.30 on a particular day (in other words, this is before I do any aggregation for the day). The way I understand this, I would do the following:
climate.variable
would be the measurement at 10m (0.45) - the measurement at 10m (0.45), which would = 0?
climate.variable
should be 10m - 10m = 0?* DistZaxsCnpy is 30m for SCBI from the excel sheet, so I would get normalized height = 0.33.
correct.
delta
climate.variable
would be the measurement at 10m (0.45) - the measurement at 10m (0.45), which would = 0?* or do you mean delta `climate.variable` should be 10m - 10m = 0?
I mean the former.
* if we do the former, what should I consider to be the h_min? For example, let's say at 30m I have a windspeed of 1.3 for 14.30-15.00. Would the h_min windspeed in this case be the windspeed from 14.30-15.00 at 10m in height?
Hmmm... order of operations here could be done different ways. There are 2 steps (1- calculate the daily min or max; 2- average daily min or max for the month). The difference between the heights could be taken either before both, after 1, or after 2. I'd envisioned doing it after step 1, so we're looking at monthly mean [(daily max at h_x)-(daily max at h_0)]
* also, we know that the SCBI tower heights are completely different from the 10,20,30,40,50,60 that the downloaded NEON data gives. I think this means if we want to be fully accurate, we may need to find the actual tower specs for all of the 25 sites we're looking at.
I think we have these in the DistZaxsLvlMeasTow
field of this file.
As a side note, could you please keep all the files for this in this folder? Organization of this repo is a bit funny because its a review-- I set it up with separate folders for each analysis/ figure.
As a side note, could you please keep all the files for this in this folder? Organization of this repo is a bit funny because its a review-- I set it up with separate folders for each analysis/ figure.
Sounds good, I'll move the files
Thanks!
* also, we know that the SCBI tower heights are completely different from the 10,20,30,40,50,60 that the downloaded NEON data gives. I think this means if we want to be fully accurate, we may need to find the actual tower specs for all of the 25 sites we're looking at.
I think we have these in the
DistZaxsLvlMeasTow
field of this file.
Where did that file come from? It lists 6 different measurements for SCBI, but I know that while windspeed measures at 10,20,30,40,50 meters, RH measures at 0 and 60m, which means seven total measurements.
In other words, for the six heights I'm given for SCBI, I don't know whether those equate to the standardized 10-60m or 0-50m.
Ahhh, I see. This came from one of the NEON staff scientists (Hongyan Luo; hluo@battelleecology.org).
Why don't you use the rough heights for now, and perhaps @NidhiVinod can follow up with Hongyan to figure out how these align.
Ok sounds good. I've made the first prototype plot for SCBI using windspeed - the first plot is including h_min, and the second plot does not. I made both to show the difference since including h_min means every plot will be branching out of the bottom left corner (and potentially be messy).
Thus first questions
This is awesome progress! Thank you, @mcgregorian1 !
Thus first questions
* do you want to include h_min for the plots?
definitely, yes.
* do you want the horizontal line at y=1 to be different?
That looks good
* do you prefer different colors for the different months?
I think that having >1 month on a plot will be too much when we have multiple sites. If we want both Jan and July, we could make separate panels.
Sorry for the delay in things. The term and my final projects are ending so I'll have more time now to devote outside of the normal workday. Based on issues 6 and 15 it seems we haven't fully determined how we want this figure to go; I'm not sure there's more for me to do with the figure yet until we determine what else we're going to add to it. Is this correct, or is there another version of these plots you want to see?
Here is a plot for the variables of SCBI
Thanks, @mcgregorian1! This is looking great.
SWIR is short-wave infrared? What does it measure? It's basically a vegetation index (sensitive to water content), right?
(I'll come back and write more on this soon.)
This is independent of issue 6, but does interface (a lot) with issue #15. I'm going to add discussion of that interface under issue #15.
In the meantime, in terms of this specific figure:
As we'll put multiple sites per panel, we'll need to separate Jan and July into separate panels, and maybe (probably?) just drop Jan altogether.
It looks like the estimated height of canopy that you're using to normalize height is too low. Based on wind speed and PAR, I suspect that the canopy actually reaches higher. What are you using as the canopy height? And I assume you're using the actual instrument heights, as opposed to the rough numbers on NEON's website?
We'll probably get a more accurate canopy height from the analyses described in #15--- for example, what @eoway did in the figure below. From this, we could calculate something like height with 95% of leaf area below.
Hi @mcgregorian1, I am currently working on leaf energy balance plots. Krista mentioned that you might be working on NEON measurements for HARV forest. Wondering if you could share HARV's July mean maximum for PAR(short wave--if you have), windspeed, RH, air temperature whenever you are able to so I can use for energy balance plots please?
Sure thing, how soon do you need this? I can either get this to you tonight or by Saturday afternoon.
On Wed, Nov 18, 2020 at 10:50 AM NidhiVinod notifications@github.com wrote:
Hi @mcgregorian1 https://github.com/mcgregorian1, I am currently working on leaf energy balance plots. Krista mentioned that you might be working on NEON measurements for HARV forest. Wondering if you could share HARV's July mean maximum for PAR(short wave--if you have), windspeed, RH, air temperature whenever you are able to so I can use for energy balance plots please?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/EcoClimLab/vertical-thermal-review/issues/2#issuecomment-729769499, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJNRBEJMZSBTEDB3APBR3ELSQPUL7ANCNFSM4PNNWGDA .
--
Ian McGregor
Ph.D. Student | Center for Geospatial Analytics
He/Him/His
College of Natural Resources
Jordan Hall 4120 | Campus Box 7106
North Carolina State University
2800 Faucette Dr.
Raleigh, NC 27695 USA imcgreg@ncsu.edu | 714-864-1005 | geospatial.ncsu.edu
Sure thing, how soon do you need this? I can either get this to you tonight or by Saturday afternoon. … On Wed, Nov 18, 2020 at 10:50 AM NidhiVinod @.***> wrote: Hi @mcgregorian1 https://github.com/mcgregorian1, I am currently working on leaf energy balance plots. Krista mentioned that you might be working on NEON measurements for HARV forest. Wondering if you could share HARV's July mean maximum for PAR(short wave--if you have), windspeed, RH, air temperature whenever you are able to so I can use for energy balance plots please? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJNRBEJMZSBTEDB3APBR3ELSQPUL7ANCNFSM4PNNWGDA . -- Ian McGregor Ph.D. Student | Center for Geospatial Analytics He/Him/His College of Natural Resources Jordan Hall 4120 | Campus Box 7106 North Carolina State University 2800 Faucette Dr. Raleigh, NC 27695 USA imcgreg@ncsu.edu | 714-864-1005 | geospatial.ncsu.edu
Saturday afternoon should be fine! Thank you!
Thanks, @mcgregorian1! This is looking great.
SWIR is short-wave infrared? What does it measure? It's basically a vegetation index (sensitive to water content), right?
(I'll come back and write more on this soon.)
Hi @teixeirak
The NEON data contains the mean incoming / outgoing shortwave and longwave, so 4 total combinations. For this plot I chose to use incoming SW mean. This is part of the "shortwave and longwave radiation (net radiometer)" data product, which we chose instead of doing infrared Biological temperature.
Oh, shortwave incoming is great, but why is it lower above the canopy than below?
Also, to clarify, bioTemp is the infrared biological temperature, correct? We do want that here.
- As we'll put multiple sites per panel, we'll need to separate Jan and July into separate panels, and maybe (probably?) just drop Jan altogether.
Sounds good, that's easy enough.
- It looks like the estimated height of canopy that you're using to normalize height is too low. Based on wind speed and PAR, I suspect that the canopy actually reaches higher. What are you using as the canopy height? And I assume you're using the actual instrument heights, as opposed to the rough numbers on NEON's website? We'll probably get a more accurate canopy height from the analyses described in #15--- for example, what @eoway did in the figure below. From this, we could calculate something like height with 95% of leaf area below.
For the normalized height I'm currently using the rough NEON heights (0,10,20,30,40,50,60) divided by the canopy height from DistZaxsCnpy (from the "TIS site metadata_20190403_forNOAA.xlsx" spreadsheet).
* also, we know that the SCBI tower heights are completely different from the 10,20,30,40,50,60 that the downloaded NEON data gives. I think this means if we want to be fully accurate, we may need to find the actual tower specs for all of the 25 sites we're looking at.
I think we have these in the
DistZaxsLvlMeasTow
field of this file.Where did that file come from? It lists 6 different measurements for SCBI, but I know that while windspeed measures at 10,20,30,40,50 meters, RH measures at 0 and 60m, which means seven total measurements.
In other words, for the six heights I'm given for SCBI, I don't know whether those equate to the standardized 10-60m or 0-50m.
To which you replied:
Ahhh, I see. This came from one of the NEON staff scientists (Hongyan Luo; hluo@battelleecology.org).
Why don't you use the rough heights for now, and perhaps @NidhiVinod can follow up with Hongyan to figure out how these align.
Oh, shortwave incoming is great, but why is it lower above the canopy than below?
Ah my bad on that - the math was off. I've fixed it, but I've also realized there is no SWIR incoming for the bottom height measurement, so I can't get a delta calculation. I can assign a value of 0 to all the NAs (in order to force a delta) but I wasn't sure what you'd want for that). I suspect this might happen for other sites as well.
Also, to clarify, bioTemp is the infrared biological temperature, correct? We do want that here.
Oops sorry, yes, bioTempMean is the IR biological temperature. It is already included.
Regarding the canopy heights, thanks for the explanation, @mcgregorian1. As per my earlier comment, this is something we'll eventually need to fix, but in the meantime we can proceed with preparing the figures.
@teixeirak I've made sample plots for each site, found here.
Some notes
You'll notice none of the sites have SWIR measurements. As I was writing this I realized we hadn't decided on an answer to my earlier comment from 21 Nov. The situation is that we do have SWIR measurements for the top of the canopy but NAs for the bottom. I can assign 0 to those, but the main "danger" of doing so is that the SWIR values for the top span positive and negative, so technically 0 is a valid reading.
Thanks, @mcgregorian1 , these are great! I'm reviewing them now.
The above figure is an old version from Ian McGregor’s in-review paper, showing NEON data from SCBI. We could modify his code to analyze all forested NEON sites. We obviously wouldn’t present this much info per site.