Cleaning Up Main

[CCLynch]

Things to remove from Main branch:

• Zoungyu's Playground
• Scratch

Topics

Here are the interesting subjects that we are exploring and want to make sure we document and describe:

• Data Processing - We've transformed and combined the original features in several ways, we need to document them
• Sampling and balancing data - do get better results throwing out the data vs bootstrapping?
• Featuring selection - We've generated a lot of features, show ways to select features.
• Models - which model succeeds and why?
  • Logistic Regression
  • LDA/QDA
  • SVC
  • Trees (boosting, forests)
  • KNN

Documentation

all these files will live at the root of the repo:

• readme.md
  • This will be a project introduction and index of md files related to each model and feature selection techniques.
• sampling.md
  • describes pros/cons of upsampling vs downsampling
• preprocessing.md
  • This describes all of the of the new features and how they are derived
• plan.md
  • requirements described by bogden in guidance.md. Sophia has already taken care of this. We just need to refactor the roles and responsibilities to be more legible.
• featureSelection.md
  • Sophia is contributing from her partitioning, Jesse has backwards feature selection based on variance inflation, Zongya will contribute PCA
• models.md
  • hyperparamaters, mis-classification rate, confusion matrices and decision boundaries (where possible), learning curves of the models we've trained.

Src Organization

1. Drivers for each model (which use files in util)
2. Util
   • contains all helpers, especially anything that can be used on multiple models
   • all the stuff related to preprocessing and plotting
3. ...extra discretionary files for specific models, with the file name starting with our initials.

[jhautala]

I think this issue can be closed; we've implemented most of these items.

There are a few driver scripts in the util package that should be moved up to the src directory, but that should be addressed under issue 18.

[CCLynch]

Agreed