thinkingmachines / unicef-ai4d-poverty-mapping

UNICEF AI4D Relative Wealth Mapping Project - datasets, models, and scripts for building relative wealth estimation models across Southeast Asia (SEA)
https://thinkingmachines.github.io/unicef-ai4d-poverty-mapping
MIT License
20 stars 8 forks source link

Fix folder structure and file naming for cross-country notebooks #156

Closed tm-jace-peralta closed 1 year ago

tm-jace-peralta commented 1 year ago

What does this PR contain?

This PR implements the suggestion on revising our current folder structure and file naming conventions for the rollouts, using the following guidelines:

  1. Keep the rollout folder dated
  2. Order the rollout notebooks inside by run order
  3. Date the model file and other relevant artifacts (models, geojson files for grids/outputs) the date is like the rollout version; doesn't have to be exactly the date you produced the file. it's just a version name to tag this round of rollout
  4. Use folders to organize by country

Here is a sample implementation

2023-02-21-cross-country-rollouts/
    1_cross_country_train_model.ipynb
    2023-02-21-cross-country-model.pkl
    id/
        2_id_generate_grids.ipynb
        3_id_rollout_model.ipynb
        4_id_rollout_results_eda.ipynb
        2023-02-21-id-rollout-grids.geojson
        2023-02-21-id-rollout-output.geojson

    <other countries>/

Revisions included

Here are the steps done to start this:

Code changes

Added a snippet in 1_cross_country_model_train_model.ipynb so pkl file will have runtime date as prefix

current_date = pd.to_datetime("today").strftime("%Y-%m-%d")
model_save_path = f'./{current_date}-cross-country-model.pkl'
with open(model_save_path, "wb") as file:
    pickle.dump(model, file)
review-notebook-app[bot] commented 1 year ago

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

review-notebook-app[bot] commented 1 year ago

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

tm-jc-nacpil commented 1 year ago

Revise folder name from 2023-02-21-cross-country-rollout to 2023-02-21-cross-country-rollouts (with s) Revise training file name from 2023_02-21-crosscountry_final_model_training.ipynb to 1_cross_country_model_train_model.ipynb Save model pkl as {current_date}-cross-country-model.pkl

Just to align on underscore/dash conventions, seems like

I'm okay with this convention!