Fine-tuning/edits needed

loriabendroth commented 7 years ago

Can you address these issues/items this week? (thanks!) Several will eventually go to Daryl but I’d like you to first decide best approach, compile, and add other items you see needed before sending on.

*User Interface

[x] Under Sites tab, ISUAG.USB has full name – change this in backend where pulling from.
[x] Have C-C-C-C-SB variants under Treatments be moved to Continuous Corn grouping. It is almost that and I think more clear.
[x] I think we could change Treatment headings to Continuous Crop Systems and Rotated Crop Systems and group Cont C and Cont SB together under that first heading to tighten up some. That would allow Prairie to come under the left hand grouping with the other crops.
[x] Which site is it that has the CT (2011) and NT (12-15)? Is it an IN site? I doubt they are the only sites that never changed but could be wrong.
[x] Delete (mid-season) on backend from NIT entries to shorten; make sure the mid-season designation is in the data dictionary. Write out “rec” to be “recommendation”.
[x] In agronomic data, delete in backend “(stems and pods only; no leaves)” and make sure it is in the data dictionary instead.
[x] Remove “dry” in agronomic entries and make sure it is stated in DD instead.
[x] Abbreviate carbon and nitrogen throughout to C and N. Add a note at the top of the page with a description of abbreviations. Can do this for MRTN also on Treatments page.
[x] Change “Soil Components” header to “Soil Texture”. There is “Soil texture” data type under properties, change that to Soil Texture Name and move it with the clay, sand, silt data. Also why the extra spacing in that box and Soil Carbon?
[x] Year should have a box around the available years to match the other tabs in formatting.
[x] Change “CSCAP Data Download” header to “Sustainable Corn CAP Data Download”
[x] Develop language for the Export Options tab that explains the rights of the user, need to cite the data in any publications, that they may want to consider co-authoring with individuals, and need to make the effort to properly interpret the data and follow-up with Sustainable Corn CAP personnel to address any questions or concerns, etc. Afterwards, there will be a “button” they push to acknowledge agreement with it all. Look to some other databases for examples of text.
[x] Have an area added where people enter their email address – which is required for the export button to work. Include a statement that an email will be sent to this address with the weblinks and citation to serve as a reminder.
[x] Add SOIL19 into export. Recommend putting in as default metadata since it is often not on plot level and wont align with structure of other soil data.
[x] create an institutional group of logos to go along the base of the webpage with SUSCORN and USDA at the top. This is part of issue #28. An example I like is at the back of the Climate Change pub here (https://store.extension.iastate.edu/Product/Climate-Change-in-the-Corn-Belt); although we will want to have all institutional logos in one row if possible.

*Exported csv

[x] Have the tabs in the exported file align left to right in a way that makes sense. I recommend putting plot IDs first, then data, then management metadata and dictionary.
[x] In the Google spreadsheets, Daryl had actual crop grown that year. Need that in the exported file also so there is no confusion what is the active crop.
[x] Generate text that is added onto the top of each export tab. Download some of Daryl’s weather data to see an example. It gives web address, contact name, etc.
[x] Align the organization of the data dictionary as they are in the export. This means the data dictionary should be grouped by similar items and not alphabetical so that they match how people will go across the tabs and within the sheets. I think it is pseudo this way now.
[x] Use DL.NOTES.HV to generate Notes tab.
[x] Column named "Edit/Review Needed" is missing in "Notes".
[x] Remove column named "primary" from "Data Dictionary"
[x] Soil fertility does not work.
[x] Change "growing_season" in "Notes" to integer. Currently it shows year as YYYY.0 (for example 2011 is 2011.0). We need to get rid of the decimal part.

*Data Dictionary

[x] Update Data Dictionary with new tab names – looks like Daryl has them switched on his end so need to switch in DD.
[x] Add method column to data dictionary report that Daryl is pulling from for the export. Populate in methods descriptions for water variables. I know there is variation but for people unfamiliar with our data, something is better than nothing and it’s not included in Kladivko paper. We do have some readme texts that may provide some help? For all other variables, please read through the methods currently listed and see if they make sense for users. If not, please improve. Steph had flagged some in red/yellow - these come from the Kladivko paper so can leave that text as long as it is clear and then add a statement at the end such as “Refer to Kladivko et al., 2014…http://dx.doi.org/xxxxxx paper.”
[x] Go through each row in DD and check that it matches exactly the variable name in the csv.
[x] Add text to the “Short Description” if you think any could be misinterpreted or vague for users.
Management Data
[x] Review the DWM tab in the management spreadsheet. If you know someone had to raise/lower the gates at a particular time but never reported it, put a placeholder in as such. This is what I have done for some things like herbicide treatments for rye termination. I know they occurred but the date and products are missing. So I put an entry for it that year but the date and product are “unknown.” Other option is to have a note in the header explaining that some are missing.
[x] For the Residue, Irrigation tab, do you feel comfortable to proof these? I am questioning the value of some variables because they are not accurate enough without a lot of extra details – such as residue percent on fields with different crops or tillage. I think they either need more explanation or lack of inclusion in the /dl. Would appreciate your review and thoughts on value of each and whether we can capture them somehow else thereby eliminating that tab entirely from csv.
[x] determine how best to have N/A across the sheets so it is uniform in Daryl’s export. Note for example in the pesticides tab that I have added a lot of N/A’s on backend but none are showing up in csv export. We also have occurrences of “na” in the adjuvant column. All of the adjuvants along with NA should be capitalized but I have waited to do this until I knew how we were handling NA across all of the sheets.
[x] In the management metadata export, my preference is to not have any blanks and instead either NA or unknown (as appropriate) for each empty cell. This is in contrast to the research data where we give people a choice of having M, -999, etc. If ok with Daryl, then please populate NA or unknown for DWM and Residue, Irrigation tabs. I will do the others.
[x] Notes. Need to do something to clean and then merge comments made by field personnel as entered in Google sheets and in SS. The ones that will help with proper interpretation on a plot or field level need to be exported.
Publications
[x] Go through the output SS and all complete or accepted/in-press refereed journals. These are the ones being populated to the interface. See if any of missing source details can be populated. I went through these all recently so I don’t think there will be many at all.

giorgichi commented 7 years ago

Based on Site Treatment sheet, SEPAC is the only site that change tillage treatment during the CSCAP.

giorgichi commented 7 years ago

@loriabendroth I replaced all capital N/As with lower case n/a in the backend sheet of the management data. One of the reasons is to distinguish N/A corresponding to Nitrogen per Acre from N/A corresponding to Not Available. Another reason is to match case with unknown.

giorgichi commented 7 years ago

@loriabendroth I revised "Short Description" for all exportable variables currently in the DD. I fixed a bunch of them, except one - "nitrogen, phosphorus, potassium, sulfur, magnesium, zinc, iron, phosphate, potash, calcium". The main reason is that it's not unique and encompases two distinct groups of variables.

giorgichi commented 7 years ago

@loriabendroth this is an agreement for data use that I compiled and modified from different sources. Let me know if there is anything we need to add.

By using the datasets, you acknowledge and agree to cite the data in any publications or presentations in the following format: Sustainable Corn CAP, http://dx.doi.org/___ (2017). You are encouraged to contact the original content providers to get assistance in data interpretation, and may consider co-authoring with them. You shall have no claim of ownership to any intellectual property rights in any part of data, content, functions, features, code, data exploration tools, logos or other intellectual property comprising this database. Sustainable Corn CAP personnel is available to address any questions or concerns regarding usage and application of the data that is made available to you on this site. Sustainable Corn CAP shall NOT be responsible or liable for the accuracy, reliability and completeness of any information included in the database as well as for the suitability of its application for any particular purpose. Data users are encouraged to notify Sustainable Corn CAP personnel regarding data quality and other issues concerning data shared through this website.

giorgichi commented 7 years ago

@loriabendroth I revised methodology for all variables except AGRO data. I left it for you, since it is not covered in Kladivko's paper and you are more familiar with those procedures.

akrherz commented 7 years ago

Issues @giorgichi found today

[x] "Notes" is missing column named "Edit/Review Needed"
[x] "Data Dictionary" have redundant column names "primary", that need to be eliminated [Lori task to remove from smartsheet report]
[x] soil fertility does not work
[x] also, can you change "growing_season" in Notes to be integer. Currently it shows year as YYYY.0 (for example 2011 is 2011.0)

loriabendroth commented 7 years ago

@giorgichi there are some variables missing character length and data type. This is because I changed them to "yes" in export and hadnt been populated earlier.

giorgichi commented 7 years ago

all missing cells in"character length" and "Accepts null value?" for "CSCAP DB Export" is filled.

isudatateam / datateam

Fine-tuning/edits needed #36