Baseline Model for Segmentation Fine-Tuning of the HLS Foundation Model

This repo contains the code, performance metrics and trained model weights for a supervised CNN model as the baseline for multi-temporal crop type segmentation fine-tuning of the HLS Foundation Model (FM). The FM is released by NASA and IBM here, and the fine-tuned FM model for this task is presented here. You can also access the training dataset for this task here.

This project is funded by an award from NASA to the Center for Geospatial Analytics at Clark University.

Instructions to run the code using Docker:

Step 1- Change directory to an empty folder in your machine and clone the repo.

$ cd /to_empty/dir/on_host/

$ git clone  git@github.com:ClarkCGA/multi-temporal-crop-classification-baseline.git

$ cd path/to/cloned directory/

Step 2- Make sure the Docker daemon is running and build the Docker image as following:

$ docker build -t <image_name>:<tag> .

Example:

$ docker build -t semseg_baseline:v1 .

step 3- Run the Docker image as a container from within the cloned folder:

$ docker run --gpus all -it -p 8888:8888 -v <path/to/the/cloned-repo/on-host>:/home/workdir -v <path/to/the/dataset/on-host>:/home/data  <image_name>:<tag>

This command will start a container based on the specified Docker image and starts a JupyterLab session. Type localhost:8888 in your browser and copy the provided token from the terminal to open the JupyterLab.

step 4- Run the pipeline:

Open the jupyter notebook located at notebooks/main.ipynb.

Modify the "default_config.yaml" or create your own config file and run the cells as explained in the notebook.

Model Weights

The model weights trained on the dataset for 100 epochs with the parameters specified in the "default_config.yaml", is stored in the model_weights/multi_temporal_crop_classification.pth. Instructions to load and use the pre-trained model for zero-shot inference or warm-up training is explained in the notebook.

Evaluation metrics:

Overall Metrics:

Class-wise Metrics: