Added drop_split_preprocess.py

dorisyycai commented 9 months ago

Added

scripts/drop_split_preprocess.py for data cleaning, data preprocessing, train/test split, model preprocessor

Added following the output files from running the script:

saved model preprocessor generated by script results/models/precipit_preprocessor.pickle
data files generated by script data/processed/X_test.csv data/processed/X_train.csv data/processed/y_test.csv data/processed/y_train.csv

Command to run drop_split_preprocess.py:

python scripts/drop_split_preprocess.py \
  --data-file=data/van_weather_1990-01-01_2023-11-06.csv \
  --data-to=data/processed  \
  --preprocessor-to=results/models \
  --seed=522

sivakornchong commented 9 months ago

Two things to be revised to improve reproducibility.

Drop column for month should not be hard coded in.
Add in the make_column_selector()

dorisyycai commented 9 months ago

function revised in https://github.com/UBC-MDS/RaincouverPrediction/pull/69

UBC-MDS / RaincouverPrediction

Added drop_split_preprocess.py #65