basic qc

[auremoser]

think through some basic quality control functionality:

• some simple tools to help identify errors, outliers etc

[annakrystalli]

There are a number of areas in the package that various checks are performed but could do with being more strategic about it. Should link in with developing tests #14 .

Q: What do you consider the most important elements of ensuring quality of your data? eg:

• handling on NAs and missing values?
• checking data.types against expectations?
• handling of white.space and blank lines
• consistency of variable names throughout file.system
• consistency of species names throughout file.system
• identifying typos
• identifying outliers
• identifying duplicates

Q: What tools in r and at what stage of data processing have you found these to be most effective?

Please feel free to share and thoughts and ideas!

[auremoser]

:+1: , very good, maybe make into a forkable checklist, gist, or notebook and include examples.

[annakrystalli]

A forkable checklist? Sounds intruiging! How do I do this?

[annakrystalli]

Cool. I'd also add these resources:

• Data Smells Etherpad
• DataJ Link list
• Data Bulletproofing Checklist
• ProPublica Guides

Lots of these are for newsrooms but I thought they might be useful for everything.

[auremoser]

I would just make a checklist in one of your issues or in a gist, you can link to it in the README and people can fork and update the gist, and the checklist formating would be pure markdown:

template:

- [ ] formating

how it renders:

• [ ] like here is an item
• [ ] another thing to check

People could add their forked gists as links to the comments in your original one, or to the issue you made in your repo, and then you'd have a little library of people's checklist examples!

[annakrystalli]

One forkable gist coming up!

Here's the link: https://gist.github.com/annakrystalli/25d58384f691b6b91df143337bd5ed4c

Please feel free to fork and edit to contribute ideas!