Identify which bands need replacing

[rudeboybert]

@jess-shue would like a single file with all stemtags that need new dendrobands so that she knows how many bands to make for 2022. For some reason there is a discrepancy in counts between

• The CI generated warnings_file.csv
• Output of fix_dendrobands.R R script

Note latter script currently require manual changes of year variables in code. Ex: 2019 -> 2020. Ensure this is done programmatically instead with statements like

current_year <-  Sys.Date() %>% year()

[rudeboybert]

For discussion on Feb 10: Here is a CSV of all 275 observations from 2021 where either a codes or notes was filled out (minus notes pertaining to anomaly detections). Note this is a much larger set of observations than those that pertain to this issue.

[rudeboybert]

(Based on discussion on Thu Feb 17) Hey @jess-shue, I think this is enough to start. I went thru all 2021 observations, dropped all those observations where

1. notes pertained to anomaly detection fixes
2. notes was only one band
3. notes was double-checked
4. other obviously non-relevant notes
5. no notes nor codes were recorded and the measure was between 3 and 150 (i.e. the safe measurements)

and here is what's left:

Definite bands to replace

In dead_marked_replace_any_tempfile.csv we have 137 measurements corresponding to 76 unique tags where either:

1. caliper_limit: The caliper measurement was no good at least once: <3 or >150
2. dead: The tree was marked dead at least once (codes equal to DC, DS, DN, I, or Q)
3. marked_replace: codes was set to RE at least once

Tag issues

Of the remaining observations, in tag_issue_tempfile.csv we have 47 measurements corresponding to 26 unique tags that have some sort of tag issue (as identified by the use of the word "tag" in notes)

Remaining

All the remainders are in remainders_tempfile.csv where we have 99 measurements corresponding to 70 tags with a real grab-bag of notes, some of them who might indicate we need to replace the band.

[jess-shue]

@rudeboybert Thanks for this breakdown! I'm going to attempt to make a pull request for the remainders_tempfile.csv as I added details to the "action_needed" column.

In the dead_marked_replace_any_tempfile.csv 1a. The < 3mm gaps I will plan to check, they may need their bands adjusted/window re-cut rather than replacing. 1b. The R code in fix_dendrobands.R checks for trees with >= 130 mm as those that should be replaced. Running that code finds the 20 listed as RE from your file and adds 16 with large windows 2a. I see 50 dead trees, with 29 of them being ash. So, @teixeirak we need to replace all 50 dead trees even those that are ash, correct? 2b. I would say the trees with an "I" and "Q" code do not need to be replaced unless they are actually dead - both may need their bands adjusted or replaced (tree with "Q" code says band gone, so will replace).

3. 20 trees marked as RE; this does match the R code in fix_dendrobands.R

From the Remaining file

1. 11 trees added to RE list with notes like "needs new band", "nail impeding", "needs repairs/adjustment", "band on ground", and "band missing" (2 were previously captured in the "definite" list)
2. Some of the odd notes are covered by the codes (but codes column not included in this file). E.g. 192570 note is "could not find" but within the dead_mark_replace file it is clear this is a dead tree with code = DS.
3. Four trees should be replaced based on these notes; all but one were in the dead_marked_replace file
4. Some notes are for coordinate or species checks or for tags to be replaced. Maybe we need an 'other_maintenance' file?

[rudeboybert]

Hey @jess-shue thanks for the update, in particular filling out the action needed column of remainders_tempfile.csv. Based on your 4 item numbered list I'll:

1. Retroactively edit the fall biannual raw data form to add codes = RE, status dead, notes = tag issues
2. Drop the stems where nothing needs to be done
3. Recompute the dead_marked_replace_any_tempfile.csv and tag_issue_tempfile.csv files

Side note for @teixeirak and @jenajordan, take a look at the remainders_tempfile.csv file: they include the difficult to classify/codify stems in the raw data forms.

[rudeboybert]

In #104 I just merged the updated CSV files. Note I moved them to the raw_data/2021/identifying_stems_with_issues folder and removed the tempfile labels b/c I figured its a good idea to hang onto those files for the future.

We now have the following counts:

1. In dead_marked_replace_any_issues.csv: ~87~ 88 bands that need replacing either b/c of codes = RE, the stem is dead, or the measure is outside [3, 150]. This corresponds exactly to the ~76~ 77 stems I originally identified plus the 11 you manually identified as RE in the remainder file.
2. In tag_issue.csv: 27 stems that have tag issues
3. In other_maintenance.csv: 11 stems that have other maintenance issues, including 5 stems ID'ed in #81 by Jen Jordan during fall survey that had bands but are not in the database
4. In remainders.csv: the ~4~ 5 additional stems that need replacing based on your manual entries.

@jess-shue regarding point 4 above: I re-added the codes column in the .csv. Could you manually enter the codes for those 4 stems in remainders.csv? (No need for PR, just do directly in master). I should similarly retroactively add the right codes in the raw data forms.

Bottom line: We are at ~87 + 4 = 91~ 88 + 5 = 93 stems identified as needing new bands.

Note added later: My mistake, I was originally counting tags, but I should've been counting tag+stemtag combos. I crossed out the original counts and added updated counts above.

[rudeboybert]

I neglected to incorporate some of the 4 points about the remainders file that Jess' identified. Based on this info, I was able to make the last retroactive codes fixes. Here is what I believe are the final set of CSV's:

1. In dead_marked_replace_any_issues.csv: 88 bands that need replacing either b/c of codes = RE, the stem is dead, or the measure is outside [3, 150].
   1. These 88 stems includes one stem with code I and one stem with code Q (but no dead code or RE code)
   2. There are no more remainder stems: i.e. all codes and notes have been accounted for.
2. In tag_issue.csv: 27 stems that have tag issues
3. In other_maintenance.csv: 11 stems that have other maintenance issues, including 5 stems ID'ed in #81 by Jen Jordan during fall survey that had bands but are not in the database

Bottom line: We need 88 new bands. I'll work on figuring out how to distribute the 52/88 bands that need to be assigned to new stems since the stem died.

FWIW here is the post-mortem on the 52 stems that died. RIP ash trees 😢

sp	n
fram	29
litu	5
quru	5
quve	3
caco	2
qupr	2
caca	1
cagl	1
fagr	1
nysy	1
qual	1
tiam	1

[jess-shue]

In #104 I just merged the updated CSV files. Note I moved them to the raw_data/2021/identifying_stems_with_issues folder and removed the tempfile labels b/c I figured its a good idea to hang onto those files for the future.

We now have the following counts:

1. In dead_marked_replace_any_issues.csv: ~87~ 88 bands that need replacing either b/c of codes = RE, the stem is dead, or the measure is outside [3, 150]. This corresponds exactly to the ~76~ 77 stems I originally identified plus the 11 you manually identified as RE in the remainder file.
2. In tag_issue.csv: 27 stems that have tag issues
3. In other_maintenance.csv: 11 stems that have other maintenance issues, including 5 stems ID'ed in review trees died/removed from census #81 by Jen Jordan during fall survey that had bands but are not in the database
4. In remainders.csv: the ~4~ 5 additional stems that need replacing based on your manual entries.

@jess-shue regarding point 4 above: I re-added the codes column in the .csv. Could you manually enter the codes for those 4 stems in remainders.csv? (No need for PR, just do directly in master). I should similarly retroactively add the right codes in the raw data forms.

Bottom line: We are at ~87 + 4 = 91~ 88 + 5 = 93 stems identified as needing new bands.

Note added later: My mistake, I was originally counting tags, but I should've been counting tag+stemtag combos. I crossed out the original counts and added updated counts above.

Sorry @rudeboybert, I'm not quite following for where to add codes for the 4 stems you mentioned. The link to the remainders.csv no longer works and remainders_originally_cataloged_by_jess.csv doesn't appear to have a codes column...

[rudeboybert]

Hey @jess-shue apologies for the lapse in communication. After asking you how to fill out the codes for the 4 stems, I figured it out on my own the next day based on your observation that most (if not all these stems) are already scheduled for removal from the database b/c of death. Sorry I didn't make this explicitly clear in my comment yesterday.

So there is no more remainders.csv file anymore, we've catalogued everything! The remainders_originally_cataloged_by_jess.csv file has all manual cataloguing you did originally, which I think is a good idea we keep.

Bottom line: The three CSV's you need are in the identifying_stems_with_issues folder: new bands needed, tag issues, and other maintenance.

[jess-shue]

Thanks @rudeboybert!

[rudeboybert]

The dead_marked_replace_any_issues.csv sheet has been updated to reflect bands to be replaced because the measure was not between [3, 130], rather than [3, 150]. We now have 102 stems marked as needing replacing (i.e. 14 additional).