Demo Notebooks

[jejjohnson]

Add demo notebooks to demonstrate different aspects of how Helio-Tools can be used and its potential interface to machine learning.

---

Preprocessing

SDO Pipeline

• [X] Add demo preprocessing steps
• [X] Add demo config for preprocessing
• [ ] Add demo config for preprocessing + dataset + dataloader
• [ ] Add demo domain-expert validation for dataset
• [ ] Add analysis functions for plots (see ITI paper)
• [ ]

---

ML-Ready Data

Demo showcasing how we can create ML-Ready Data using scripts and parallelization. We will create 3 dataset types for each tutorial 1) single channel datasets, 2) multi-channel datasets, and 3) multi-channel time-series datasets.

• [ ] Numpy Dataset
• [ ] Image Dataset
• [ ] xarray.Dataset

---

Machine Learning DataModules

Demo Training Datasets + DataLoaders

These demos will showcase how we can create datasets and dataloaders for different datastructures. We will focus on PyTorch datasets for three data structures: 1) numpy.ndarray, 2) .png/.jpeg/… image files, and 3) xarray.Datasets. Well will discuss things like global dataset normalization and (random) patching.

• [ ] Demo DataLoader with numpy (see example notebook)
• [ ] Demo DataLoader with xarray (see example)
• [ ] Demo DataLoader with RasterVision (see example)

These demos will showcase how we can create datasets and dataloaders for inference (making predictions for new datasets). . We will focus on PyTorch datasets for three data structures: 1) numpy.ndarray, 2) .png/.jpeg/… image files, and 3) xarray.Datasets. We will discuss additional transformations that are needed for the datasets like training dataset normalizing and sliding window patching.

• [ ] Inference on a demo dataset (see example)
• [ ] Inference on a real dataset with ITI model weights (see translate.py)