Challenge & Notes

[WGierke]

Challenge

• classify/label repos automatically
• analyze relevant features
• document design thoughts and training approach

Documentation Structure

1. Data Exploration and Prediction Model
   • analyze and document relevant features
   • document how to avoid overfitting
   • explain why we've decided to use the features
   • explain how we've developed the prediction model
2. Automated Classification
   • implement the app that takes the input format and creates the output format
   • either 1) prompt for the training data to use or 2) directly include the learned model
3. Validation
   • validate with Appendix B
   • create a boolean matrix with our estimated label and the predicted one
   • compute recall per category
   • compute precision per category
   • dicuss quality of results and whether higher yield or higher precision is more important
4. Extension
   • use the model for a nice app
5. Furthermore
   • document 3 repos where we think our model will yield better results
   • install and user manual
   • document decisions we made for features, algorithms, data structures, software development tools and practices

     Notes

Examples for DATA-Repositories openaddresses / openaddresses unitedstates / congress-legislators OpenExoplanetCatalogue / open_exoplanet_catalogue Chicago / food-inspections-evaluation GSA / data cernopendata / opendata.cern.ch benbalter / congressional-districts

Extension

"Improve yourself"

• Login with Github -> Stats of your own repos e.g. 30% Data, 70% Software -> Stats of repos your friends recently starred |-Data-| Software | Homework | ...| -> Stats of trending repos |-Data-| Software | Homework | ...|recently

Sources:

• https://github.com/caesar0301/awesome-public-datasets -> what's hosted at github?
• https://github.com/datasets
• https://github.com/showcases/open-data
• github.com/explore
• github.com/trending