outline tasks for `dummy-tasks`

HRDAG / training-docs

materials to study and learn about principled data processing

GNU General Public License v3.0

10 stars 4 forks source link

outline tasks for `dummy-tasks` #3

Open baileyb0t opened 2 years ago

baileyb0t commented 2 years ago

Need to plan out tasks to stage. Maybe get dataset from Kaggle or other resource to use in sample_project and pair with dummy tasks

baileyb0t commented 2 years ago

[ ] "updated dataset" issue
- snap pull to get new data
- changes to import/preproc/src/import.py to update asserts, logging of new, raw data
- changes to import/preproc/src/clean.py to handle new data
- make clean outputs in dedupe
- make all outputs in dedupe
- git add, git commit, and git push altered scripts
- snap push updated outputs

baileyb0t commented 2 years ago

Other tasks that may be useful as practice

create need for implementing standards:
- [ ] change relative path input to symlinked input
- [ ] add arg handling support to script
- [ ] add logging support to script
create need for adding a new script to an existing task
- [ ] implement, test script
- [ ] add script output as new target in makefile
create need for adding yaml file and hand/ to existing task
- [ ] create yaml file
- [ ] implement yaml arg support to script
- [ ] add yaml arg to script output target in makefile

baileyb0t commented 2 years ago

Adding sample_project link for reference. Ideally, the data in this project would be tracked with snap so practice with that tool can be included in these demo-tasks