nutnetadmin / nutnet-data-tracker

Track bugs, changes and updates in NutNet data
2 stars 0 forks source link

correct duplicate soil values #465

Closed nutnetadmin closed 8 months ago

nutnetadmin commented 8 months ago

duplicate plots are getting reported in comb-by-plot-clim-soil files. This appears to be due to two core reasons: 1) Post-treatment data is incorrectly flagged as pre-treatment, causing two lines of soil data to be reported for each plot. This is an easy fix of changing the flag. (PW did this by updating any soil sample code with PT - post-treatment- to have a value of 1 in the pre-or-post treatment flag column) 2) texture data is uploaded separately from chemistry data (P, K, ph, etc) causing two separate chemistry lines to be inserted into the table. On compilation each line is inserted into the comb file, causing plot duplicates with NULL values for chemistry in one line and NULL values for texture in another. This should only affect control plots where texture is uploaded.

nutnetadmin commented 8 months ago

PAW split out soil texture into a new table which solved the repeat issues stemming from point 2.

Remaining issues appear to be repeated entries into soil_cn table. Repeats can be removed but ~100 entries from three sites in 2016 have shared soil_codes but different carbon and nitrogen value. These remain to be solved.

nutnetadmin commented 8 months ago

Reuploaded cdcr.us, burrawan.au, and pinj.au 2016. pinj.au is mislabeled as ping.au in all the raw csv files. AK checked and did not find any physical record of having ping.au 2016 soils.

nutnetadmin commented 8 months ago

Last fix was that 7 sites had plot_IDs assigned that did not update to unique plot numbers. i.e. - some sites repeat plots across blocks but we use incremental plot numbers. I updated these plot_IDs and they are now indexing correct.

nutnetadmin commented 8 months ago

I am adding the following restrictions to tables to try to avoid these issues in the future

1) unique soil_code in soil_cn, soil_chemistry, and (new) soil_texture 2) unique plot_ID-year_collected in soil_sample