Edits during review period (9/19 - 10/20)

[loriabendroth]

MISC

• [x] check years populated in tool for WOOSTER.COV for soil organic carbon. 2011 was greyed out but 2012 should also be because we are not including bc of quality concerns. 2012 is not in the exported file.

• [x] reword first bullet at bottom of decagon data page which refers to NAEW and VICMS. Note only applicable for NAEW now.

• [x] remove "1" people from admin list since no longer needed?

• [x] @akrherz add siteid into export of all water variables - right now plotid is exported only. Order to be siteid, plotid, timestamp, variable.

• [x] change "Conventional Tillage" to "Tillage" here; this will then match DD. https://datateam.agron.iastate.edu/cscap/dl/#tab_data

• [x] I selected all sites and all treatments, but two agro variables grey out - corn cob total N at R6 and mid-season canopy N sensing. Check on this; the data exist. Aggregate stability under Soil is also greyed out when all are selected.

• [x] change hyperlink of technical reports to open in a new tab

• [x] put two tabs under Weather Data for networked weather data and on-site weather data. I think this will be clearest for users.

• [x] bring in on-site precip data

• [x] add in VRA N applications for MO (BRADFORD.A, B1, B2) and OH (WOOSTER.COV) sites on plot-level.

• [x] review document linked to FAQ Q2 under Decagon. I dont think this document is needed for external audiences so recommended removing text and link - keep for internal records only. See yellow highlighted in doc - we dont have any "F" values in export, do we? If so, need to include explanation in FAQ of what F represents.

• [x] remove this statement at top of Google map. I dont like how it stands on its own and decided its not critical. "Experimental site locations and plot boundaries accessible via upper right bracket in map."

• [x] add "Technical Reports" tab under Publications and to the right of Thesis/Diss. Format as needed to match other tabs in look. Populate with text: technical report text.docx

• [x] move Water Quality to third item in list (instead of 4th) under Water Data, Tools subtab.

• [x] in overall dropdown, order water plotting tools in same order as under Water Data, Tools subtab.

• [x] embed analytic codes for cscap and td. see: Analytic codes to embed.docx

• [x] @akrherz please edit following in the email text:

  1. Change subject to “Sustainable Corn CAP Research Data (USDA-NIFA Award No. 2011-68002-30190)”
  2. In the last paragraph, it should read …in the database nor the suitability of its application for a particular purpose

• [x] @akrherz check admin page. Note that Peter Scharf downloaded file on 10-21 and 10-22 but he is not shown as logging in on either day. Are admin controls working correctly regarding logins?

• [x] truncate agronomic and soil variables based on recommended digits from Obj1&2

• [x] units for both C and N should be changed from % to g/kg

• [x] notice at bottom of weather page that it is "less than 9" and "more than 9". 9 is not included in either. Correct language as needed. 9 or less are included in the title name.

• [x] remove bold headers "Sustainable Corn CAP Data Download", "Water Data", and "Daily Weather Data". It is redundant to tab headings and I dont think necessary. If disagree, retain but then need to add in headers for other tabs to be consistent.

• [x] Add text to top of Sites page, below the tabs: “This download tool will auto-populate options in the following tabs based on the research sites selected below. It is recommended to “Select All” sites if you want to retain the most options moving forward.”

• [x] @akrherz change -999 in SERF Decagon data with NULL (or whatever is used to represent missing values in postgre db)

• [x] remove link to BRADFORD.A photos from Research Sites tab. Add new link to BRADFORD.C photos (https://drive.google.com/open?id=0B_Wz65q1Dw-0R3hMeXc0LUZpVEU)

• [x] change "Water Table" to "Water Table Depth" in dropdown and page header to match Jane's request.

• [x] With regards to moisture characterisitics, the readers must recognize at the very first glance whether the moisture content is gravimetric( g/100g) or volumetric( cm3/cm3).Thus,showing the units ( even though it is unit less) is being very informative and pertinent.

• [x] Change title for data on USDA NAL page to “Sustainable Corn CAP Research Data (USDA-NIFA Award No. 2011-68002-30190).” and then ripple through /dl including the data use agreement drop down, last tab on research download, and email text.

• [x] this edit from julie was made in the last tab of research export but needs to also be changed in Data Use drop down and email export: In the last paragraph of the data use agreement, it should read …in the database nor the suitability of its application for a particular purpose

• [x] change header for tile flow data to only reference CSCAP: "...are associated with the USDA-NIFA funded project: "Cropping Systems Coordinated Agricultural Project: Climate Change, Mitigation, and Adaptation in Corn-based Cropping Systems." (Award No. 2011-68002-30190)."

• [x] change header for Water Quality to only reference CSCAP also.

• [x] change header for Water Table Depth to only reference CSCAP also. Change observation interval text at bottom for DPAC to be n=5 years; 2011-2015

• [x] change "10" to "9" at bottom of weather export tab regarding file name.

• [x] change TRT2 from "Conventional Tillage" to "Tillage" in treatment master and data dictionary.

• [x] remove LAND3 as a treatment to all sites except for MO and MI sites that have landscape as a treatment. Make the plot identifier sheets show "N/A" and in tool, have it greyed out if N/A exists for landscape. (OK?)

• [x] Perhaps change to “For sites you’ve selected, available treatments will be identified by black text; unavailable in grey”. Whatever the statement, standardize across all of the tabs

• [x] Reword- use “because” instead of “as”, no “s” on helps. “For this reason, all variables are selected by default”

• [x] DPAC water table data - only goes to 12/3/2015. The rest of the year exists, can this be moved over from the TD database to finish the year?

• [x] DPAC soil moisture data - only goes to 12/6/2015. The rest of the year exists, can this be moved over fro the TD database to finish the year?

• [x] Is there a limit to how many names the weather file will export as a stacked file name? 4 worked. 8 did not.

• [x] change Groundwater Table Depth to Water Table Depth in Water Data, Tools tab

• [x] Recall how the decision was made somewhere along the line, to put the fall cover crop biomass into the spreadsheet for the next calendar year, so that fall and spring of one “cover crop season” were together in one sheet. The data dictionary descriptor needs just a little more, because the new user will not catch that. You’ve got good verbage in parameters 41-43 (…”in late fall of previous year”), but for AGR 6, 13, and 14 it just says “at fall” or “in late fall”. So we suggest those three parameters say “…….in late fall of previous year”. Then for spring (AGR 15, 16), use the terminology of AGR 44-46, “…..in spring of spreadsheet year”. (currently AGR 7, 39, 40 are probably fine because they say “at termination”, but you could add the “of spreadsheet yr” there too, if you want).

• [x] So on our Sepac data, the cover crop biomass collected in fall 2012, is listed twice (right now it’s listed in 2012 and 2013), whereas it should just be listed in 2013 (data from late fall of previous year, meaning here in 2012). This is listed twice for AGR 13 and 14 (ie the C and N), but it’s correct with just one entry in AGR 6, fall biomass.

• [x] On the drop down box, you have timeseries as one word – should be two words or hyphenated

• [x] Can the select start date box be moved more to the left? Also, it started on a random date in 2014. Probably makes most sense to set default to either 2011 or current date

• [x] Suggest rewording the Pro Tip for better clarity – “Click the legend name to de/select data series. To zoom in, click and drag your mouse within the plot frame.”

• [x] SEPAC cover crop site, needs correction related to drainage. That field has NO drainage tile (it’s a sloping field and doesn’t need it). So a) on metadata, there should be NO tile depth listed, nor a TBD on spacing, and b) on the page where one can select which data one wants from a site, there should either be “no drainage” highlighted (not shallow drainage as it is now), or the whole drainage section should be “grayed out” (ie there is no selection to be made regarding drainage).

• [x] An apparent glitch in how the soil nitrate and ammonium gets translated into the data exported. The complication is that both fall and spring data are being exported in the same column, one column for nitrate and one for ammonium. But the ORDER of which is spring and fall, appears to be random. This is very evident when looking at the 60-90 cm depth, where in fall we sampled it but in spring we did not, and therefore spring is “nan”. You can see that the export seems to put the “nan” sometimes first in that column, and sometimes second. The same occurs for other depths where there are two numbers, but sometimes spring is first and sometimes fall. This is Sepac and DPAC, so presumably some issue with export function. Is there a reason that you want to put both spring and fall in the same column? It seems like it would be simpler, and easier to understand, to have two columns each, one for fall and one for spring.

• [x] include a note in metadata about the precision of an analytical method used for estimating soil nitrate and ammonium concentrations at SEPAC

• [x] Text above map under research sites tab reads confusing - perhaps could state "Experimental site locations and plot boundaries accessible via upper right bracket in map"

• [x] When I open the map, left hand bar - suggest ordering the site areas similar to the site list; right now the NAEW site areas are at the top of the list, but further down under the "sites" section

• [x] After selecting my data for download, the blue thank you box has a spelling error. Last line - "delivered"

• [x] In the last paragraph of the data use agreement, it should read...in the database nor the suitability of its application for a particular purpose

• [x] Data dictionary tab - what is meant by global? It made me think that plots were across the globe, but I believe you're using the term to be = universal, or used across all sites

• [x] Under plot identifiers, I was looking at the soil series descript 1, 2, 3, 4 and trying to understand why some have >1. I figure it's simply bc some expt AOI's have > 1 soil type, but thought I'd go check the data dictionary. But then realized that's wrong place to go. Data dictionary is only if you have a soil tab.

• [x] Hide the Admin Tab for un-authorized viewers

• [x] we've discussed before; but anyway right at time of export to change AGR1, AGR2, SOIL1, SOIL2, etc to AGR01, AGR02, SOIL01, SOIL02?

• [x] some standardization is needed in soil series texture - there are "clay loam", "Clay Loam", "Clay loam" for example.

• [x] under Soc Econ tab > delete biophysical research heading and corresponding text and also subheading Soc Econ since the tab has that name.

• [x] above Download tab for weather data include this statement or something better: The downloaded data file will have the name of the networked Weather Station. Refer to the drop-down above for reminder as to which site it corresponds with.

• [x] @giorgichi, can you check soil series et al data for NWREC and BRADFORD.B2? Fill based on Web Soil Survey.

• [x] @giorgichi, there is a discrepancy between ROT codes in export and those in DD. ROT18, 19, 20, and 62 are in export but not in DD. ROT 58 is in DD but not in export.

• [x] ROT62 is the same as ROT7; change in data sheets for STJOHNS and treatment master. Rotation is soybean in 2011, wheat in 2012, corn in 2013, soybean in 2014, and wheat in 2015. This is the same rotation. Remove from DD also.

• [x] create new tab under Research Data, "Soc Econ Data", and put it to the right of Weather Data

• [x] change error message if an email is not entered under Export from "sorry, script has failed" (or something like this) to "There is an error with your email address. Check this."

• [x] populate the same content to Soc Econ Data as under that subheading here: http://lib.dr.iastate.edu/sustainablecorn_researchdata/

• [x] under the Stat Atlas 1 hyperlink, inset this text to describe it: This first atlas (Volume 1) provides data regarding farmer beliefs and concerns about climate change, attitudes toward adaptation and mitigation, perceived capacity to deal with the impacts and recent experiences with extreme weather events. Delineated by watershed, this collection of tables and maps provides the results of a USDA-NIFA-supported survey, conducted in 2012, of 4,778 farmers in 11 Corn-belt states.

• [x] under the Stat Atlas 2 hyperlink, inset this text to describe it: This second atlas (Volume 2) provides additional results from a 2012 survey of 4,778 farmers in 11 Corn-belt states. Here, the focus is upon farmers' specific behaviors, beliefs about climate and weather, and the tools they utilize to make farm decisions. You will find information covering timing of farming practices and farming decisions, whether and how farmers use weather information when making farming decisions; a detailed look at the influence of agricultural advisors in farmer decision making processes; farmers' personal experiences regarding weather and risks; and plans and efforts of farmers to adapt to and manage for weather and climate variability and risk.

• [x] cells are blank in export file for date under sample_canopy and sample_soilnitrate. Make these n/a.

• [x] cells in GHG and IPM with no data are empty; make so they adhere to -999, M, or whatever person selects at export to fill these.

• [x] GHG is not capitalized in column headings whereas AGR, SOIL, IPM are. Make these cap.

• [x] AGR and SOIL have columns that are empty (for variables not collected); look at AGR11-16 for example at ARL. Make so these have same filled value as other cells that are missing.

• [x] Sort exported agro data by uniqueid > year > plotid

• [x] Sort exported soil data by uniqueid > year > plotid > subsample

• [x] Sort exported GHG data by uniqueid > year > date > plotid

• [x] Sort exported IPM data by uniqueid > year > date > plotid

• [x] In Field Operations tab, biomassdate1 and biomassdate2 are populating in blanks if no date exists. The original sheet has "n/a" for blanks but arent showing up in export.

• [x] narrow data_type and growing_season column width under Notes tab; can be as wide as column header width only because wont have much for text in those cells.

• [x] in earlier request, weather file exports were renamed to site name. Can CSCAP uniqueid be added also; something like BRADFORD.A_MO1791.

Map

• [x] Add this statement to be above the Google map: “Open the map, using the upper right bracket, to see the site area boundaries for each experimental site.”
• [x] Change “Considerations” to “Plot Map Considerations”
• [x] Change bulleted text under “Considerations” to: “Site maps included below will often include…” “The maps however may contain…”

Treatments Tab

• [x] Capitalize first word for each treatment listed and then lowercase subsequent words. This will make it consistent with other tabs. Note also that CORN should be corn as fifth checkbox under Rotated Crop Systems.
• [x] Add this statement under Research Data / Treatments: “Treatments that are applicable for the sites selected will be black. Treatments in grey are not applicable for the sites selected.”

Data Tabs

• [x] Alter the statement under Research Data / Agronomic to: “Agronomic data that are available for the sites selected will be black. The usage of C and N abbreviations below…”
• [x] Add this statement under Research Data / Soil: “Soil data that are available for the sites selected will be black.”
• [x] Add this statement under Research Data / GHG: “Greenhouse gas data that are available for the sites selected will be black. The soil data listed here were collected at the same time as the greenhouse gas measurement; these are distinct fro those under the Soil tab.”
• [x] Check the GHG data – I have all sites selected yet these 4 are not options to select: Methane (CH4-C) flux Methane (CH4) correlation coefficient (R2) Soil Nitrate (NO3-N) at 0-10 cm Soil Ammonium (NH4-N) at 0-10 cm

Weather Data

• [x] For Weather Data tab, change upper text to: Supplementary weather data are available for Sustainable Corn CAP research sites and are provided from nearby weather networks. These data are most appropriate to use with these research sites because of proximity. The data are collected from a variety of sources, including the National Climatic Data Center, National Weather Service, and State Climatologists with curation and management by Iowa Environmental Mesonet. These entities revise their data and will produce updated values that are reprocessed into this archive.
• [x] Under Select Data Variables, alter this statement: “Date and Day of Year are included automatically in the export. To select multiple data, hold CTRL down while selecting options here.”
• [x] change header to “Optional: Specialized Model Formats for Export”
• [x] Under Specialized Data/Model Formats, delete the first sentence and only keep the second.

Export Items

• [x] in email text, add a line between last two paragraphs
• [x] the data dictionary in export does not match the report in SS (Sustainable Corn data Dictionary (export)) in regards to sorting. The SS report is how it should look in the export.
• [x] in the data dictionary export tab, narrow the data_type, units, accepts_null_value?, and required status columns since that is narrower data always.
• [x] Notes column should not exist in Plot Identifiers tab. Remove and also make sure not showing up in DD.
• [x] In plot identifiers, move the agro_data, soil_data, ghg_data, ipmcscap_data, and ipmusb_data columns so they come after 2015 crop column. That leaves only the soil columns to the right.
• [x] widen the date column in GHG tab by a couple characters (is this possible to do just the one column and leave others to minimize work required?). Same for IPM and Field Operations and Pesticides. In Field Operations, there are two date columns: date and biomassdate. Under Drainage Control Structure, widen the outletdate column.
• [x] Sort notes as uniqueid > overarching_data_category > data_type > growing_season

Misc

• [x] Add a line above ISU Data Contacts header under Contact Us, similar to spacing on homepages.
• [x] Change 36 to 42 number of sites for TD homepage

[akrherz]

Regarding

Check the GHG data – I have all sites selected yet these 4 are not options to select:

For GHG09, GHG10, GHG15 and GHG16, we don't have any data. Shall I just remove those options from the page?

[akrherz]

Regarding

in email text, add a line between last two paragraphs

what do you mean by 'a line' ? A vertical spacing? The email is non-HTML, so we can't do HTML elements at this time

[akrherz]

Regarding

the data dictionary in export does not match the report in SS (Sustainable Corn data Dictionary (export)) in regards to sorting. The SS report is how it should look in the export.

This is a tough one, but I hacked something in to make this happen soon.

[loriabendroth]

@akrherz thanks for getting these addressed, looks good!. Couple minor things:

• [x] capitalize “continuous” for soybean under Treatments
• [x] change “fro” to “from” in GHG header text
• [x] add statement to IPM tab: “Insects (pest and beneficial) and plant diseases that are available for the sites selected will be black.”
• [x] Those GHG data are needed in the export; @giorgichi please send these.
• [x] The email is good now; forget about adding a line. Note that the citation starts with a new line after authors and year. So that the title and source are on a new line; bring up to follow directly after year.
• I added 2 new checkboxes up above

[giorgichi]

@loriabendroth updated GHG data was sent to Daryl 3 days ago (9/18/2017). Not sure what do you ask me to send.

[akrherz]

Regarding

Note that the citation starts with a new line after authors and year. So that the title and source are on a new line; bring up to follow directly after year.

I am fairly certain what you are seeing is email client dependent, it appears fine for me.

[loriabendroth]

@akrherz few new edits up at top regarding exported file.

[akrherz]

@loriabendroth regarding

in earlier request, weather file exports were renamed to site name. Can CSCAP uniqueid be added also; something like BRADFORD.A_MO1791.

Not at this time, same issue as the request to add this ID as a column to the sheet.

[loriabendroth]

@akrherz ok, i was wondering. not a big deal.

[loriabendroth]

I brought this job back to life as misc edits will keep coming. I put a few new ones at top that I noticed when working with data.

[giorgichi]

@loriabendroth, sure.

[loriabendroth]

@akrherz is it possible to remove data for certain sites and years that have no soil data at all in the export? I have downloaded all soil data and there are many rows with NA all the way across. For example, ORR, plot 3092, 40 to 60 depth, subsample 2, year 2012. I think its still good to keep original Google entry sheets as is, but just remove from export.

[loriabendroth]

@akrherz sorry for the great pain caused by my previous request :) A few new misc items up top. I may add some more later on today also.

[giorgichi]

Plots with ROT18, ROT19, and ROT20 were not part of the original research. These plots were added later as part of IPMUSB research. That is why these rotations were not in the DD originally. Now all three rotations are added to the DD.

[giorgichi]

@loriabendroth ROT62 != ROT7

[loriabendroth]

@giorgichi ok, yes, you are correct. I am just perpetually confusing myself. @akrherz for the export, can we just change the rotation codes on a few? For example, ROT62 would become ROT7v to stand as a variant of ROT7 to signify that it is similar but not identical. Possible? If so, this would apply to just a few cases.

[giorgichi]

@akrherz soil series name/description is missing for some plots in BRADFORD.B2 when I export the data. Although all plots have it in Google Sheet. Could you update plot identifier table, hope this will solve the issue.

[akrherz]

@akrherz soil series name/description is missing for some plots in BRADFORD.B2

Yeah, the script that syncs plotids is not scheduled to run every day, so I had to manually run it and update the database schema to include the new "timing" column that appeared in the KELLOGG sheet.

The database contains one plotid now without soiltextureseries1 defined, want to fix?

select plotid, uniqueid from plotids where soiltextureseries1 is null or soiltextureseries1 = '';
 plotid |  uniqueid  
--------+------------
 118T   | NAEW.WS118
(1 row)

[akrherz]

@loriabendroth regarding

For example, ROT62 would become ROT7v to stand as a variant of ROT7 to signify that it is similar but not identical. Possible? If so, this would apply to just a few cases.

I am not following what you are asking for. You want me to make this substitution within the database or when the export spreadsheet is materialized?

[giorgichi]

@akrherz thanks for pointing out at missing soil series at NAEW.WS118. It is fixed now.

[loriabendroth]

@akrherz the timing column will eventually come out of KELLOGG (and MASON). It is just there as a placeholder now until we get clarification from site personnel on the treatment configuration.

[loriabendroth]

@akrherz for the rotation codes, I am wondering if possible for when export spreadsheet is materialized. I dont want you reworking all database structure and past programming. If possible, then @giorgichi or I can tell you which to change.

[giorgichi]

@loriabendroth - I went through the site notes again to remove (make invisible) unimportant entries. I managed to get rid of 60 of them. Now we are down to 162 entries. I recommend you to read those ones and make a final decision which notes are suitable.

[akrherz]

@loriabendroth regarding

Is there a limit to how many names the weather file will export as a stacked file name? 4 worked. 8 did not.

Large filenames can cause grief, so there are practical limits at play

[loriabendroth]

@akrherz regarding weather filenames, can you add to the statement at end of weather page whatever the limit is? i.e. If sites downloaded are between 1 and 4: The downloaded data file will have the name of the networked Weather Station.... If sites downloaded are greater than 4: The downloaded data file will be named "changeme" and the user will need to reference the drop-down above for which sites were selected.

[loriabendroth]

@giorgichi I like using "Universal" in place of "Global" to describe the variables that go across.

[giorgichi]

@loriabendroth Global is substituted with Universal in DD

[akrherz]

regarding

Is there a limit to how many names the weather file will export as a stacked file name? 4 worked. 8 did not. The limit is 10 and should be denoted on the interface now

[giorgichi]

@akrherz - I updated Landscape column for all CSCAP sites in Plot Identifiers. I entered "N/A" for sites that do not have landscape positioning as treatment. Could you sync again Plot Identifiers with the db.

[akrherz]

@giorgichi the plot ids have been synced

[loriabendroth]

@giorgichi regarding this: change title for data on USDA NAL page to “Sustainable Corn CAP Research Data (USDA-NIFA Award No. 2011-68002-30190).” I was referring to the title in the data citation. So it should be "... Villamil. 2017. Sustainable Corn CAP Research Data (USDA-NIFA Award No. 2011-68002-30190). National Agricultural..." Notice that this title in the citation is the title in the NAL portal. The NAL creates the citation we are using - you can scroll to bottom of NAL page to check that everything matches with what we have on /dl.

[loriabendroth]

@giorgichi need to correct header language for Tile Flow, Water Table, and Water Quality. Notice that it refers to two USDA-NIFA funded projects, it should be: "are associated with the USDA-NIFA funded project:"

[loriabendroth]

@akrherz notice a couple jobs up at top for you!

[giorgichi]

@loriabendroth - title you want to use does not match with the citation in NAL portal. Could you recheck the wording for the title before substituting it with another one?

[giorgichi]

Project title was changed in NAL to match with Lori's new version.

[akrherz]

@loriabendroth regarding

check admin page. Note that Peter Scharf downloaded file on 10-21 and 10-22 but he is not shown as logging in on either day. Are admin controls working correctly regarding logins?

The access controls were removed from the sustainablecorn portion of the website on 20 Oct, I verified that the download was properly logged for those two dates.

[giorgichi]

@loriabendroth - I updated variable codes in DD (AGR1 => AGR01, etc.)

[loriabendroth]

@giorgichi technical report tab looks terrific! can you just make so the hyperlinks open in a new browser tab?

[loriabendroth]

@akrherz @giorgichi just noticed error on Refereed Journals tab. Scroll to the bottom - you'll see technical report text that should be deleted.

[giorgichi]

@akrherz I know what is causing this problem

I selected all sites and all treatments, but two agro variables grey out - corn cob total N at R6 and mid-season canopy N sensing. Check on this; the data exist. Aggregate stability under Soil is also greyed out when all are selected.

But I am not sure how to fix. Can you help me now?

[akrherz]

added landscape values of 'N/A' to always be included in filter with 291691d3d2583a6550e2cab7355271410e132863

[giorgichi]

@loriabendroth - export tool currently is not capable to do this:

check years populated in tool for WOOSTER.COV for soil organic carbon. 2011 was greyed out but 2012 should also be because we are not including bc of quality concerns. 2012 is not in the exported file

You can confirm it with @akrherz

[akrherz]

@loriabendroth regarding

check years populated in tool for WOOSTER.COV for soil organic carbon. 2011 was greyed out but 2012 should also be because we are not including bc of quality concerns. 2012 is not in the exported file.

Currently, the filter logic for years includes a general check against GHG data. I have made it explicit now.