leppott / MBSStools

Suite of tools for data manipulation and calculations for Maryland DNR MBSS program.
https://leppott.github.io/MBSStools/
GNU General Public License v3.0
3 stars 3 forks source link

IBI calculation - potential data QC checks #26

Closed leppott closed 4 years ago

leppott commented 5 years ago

Is your feature request related to a problem? Please describe. Potential checks:

  1. Strata spellings. Make sure all are of the correct convention (i.e., spelling).
  2. Exclude column. TRUE/FALSE vs. other formats.
  3. Add a master taxa list check. User provides and the master taxa list and reports back non-matches. a. Can do for fish and bugs. Just need the lists. If posted online much easier to update/distribute to the users but can put in a current version.
  4. Ensure samples with 0 organisms get an IBI of 1.

Describe the solution you'd like A clear and concise description of what you want to happen.

Describe alternatives you've considered A clear and concise description of any alternative solutions or features you've considered.

Additional context Add any other context or screenshots about the feature request here.

leppott commented 5 years ago

Potentially could create a report of the number of taxa that do / do not have autecological information. Concentrate on what is needed for IBI calculations.

leppott commented 5 years ago

Could run a routine on Excluded taxa as defined by user vs. automated.

BioMonTools::markExcluded.

Could import routine so don't rely on another package.

https://github.com/leppott/BioMonTools/blob/master/man/markExcluded.Rd

https://github.com/leppott/BioMonTools/blob/master/doc/ExcludedTaxaDecisionCrit.pdf

leppott commented 5 years ago

Count vs. Non-Count taxa per MBSS protocols.

For example, MBSS does not include Acari (water mites) in its samples.

leppott commented 5 years ago

Min/Max individuals.

Min = 60 Max = __

Already have the QC value in the output.

And if less than Min was the sample 100% sorted.

leppott commented 5 years ago

Should check contents of FFG and Habit columns for accepted values.

leppott commented 4 years ago

Added QC check for 0 organisms. Metric Scores and IBI are now "1".

Added one new sample to the example data to ensure catch the error.

Modified Code in metric.values(). image

Warning message to user. image

ashtonmj commented 4 years ago

Erik, I don’t think the 0 individuals = IBI of 1 holds true for bugs as it is for fish. Again, I don’t know why there are so many basic inconsistencies between them, but there are nevertheless. There are no 0 organisms benthic samples in the database; either we don’t calculate it or maybe a sample was re-taken?

Sent from my iPhone

On May 20, 2020, at 7:58 AM, Erik Leppo notifications@github.com wrote:



Added QC check for 0 organisms. Metric Scores and IBI are now "1".

Modified Code in metric.values(). [image: image] https://user-images.githubusercontent.com/24276278/82443563-b6f86180-9a6f-11ea-9b68-dbd2aedac549.png

Warning message to user. [image: image] https://user-images.githubusercontent.com/24276278/82443496-9af4c000-9a6f-11ea-9d7b-75a20db0f8c6.png

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/leppott/MBSStools/issues/26#issuecomment-631428855, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJKZ5WJZFSEWE7PCBZEV3ELRSPAYDANCNFSM4HJWFIWA .

leppott commented 4 years ago

Commented out the BUGS section.

image

For bugs have a post calculation set of code in the example to test for "large" and "small" samples.

image

leppott commented 4 years ago

Master taxa (species) is a new function qc_taxa(). Issue #13

leppott commented 4 years ago

Example console output:

image

leppott commented 4 years ago

Added "NA", "-", "DV", and "SK" as valid values for Habit.

Otherwise returns error.

image

leppott commented 4 years ago

FFG valid values converted to upper case to avoid showing errors in case:

image

leppott commented 4 years ago

FFG and Habit check all parts of multi-entry records against the valid values.

For example, ""SW, CN, SP" becomes separate entries for SW, CN, and SP and each is checked against the valid values. This allows for any combination of valid values to be checked.

leppott commented 4 years ago

Added check for fish strata as well.

image