developmentseed / pearl-ml-pipeline

12 stars 3 forks source link

AML workflow to train LULC models for PEARL #8

Open srmsoumya opened 2 years ago

srmsoumya commented 2 years ago

LULC Pipeline uses AML(Azure Machine Learning) to store data, train models & evaluate scores.

To train a segmentation model for PEARL, we need access to

Our current workflow

  1. Create labeled tiffs & train/valid/test CSVs for ML model using naip-utils scripts
  2. Move the TIFFs to an Azure Blob storage
  3. Move CSVs to data/ inside src
  4. Create workspace, CPU & GPU clusters using scripts from train_azure
  5. Run run_model.py script to get the most_recent_model.pt file.
  6. Download the model & store inside src/model
  7. Execute eval, seed_data & class_distribution scripts

These scripts are tied to our AML resources.

cc/ @geohacker