Overall project deployment plan

[raehik]

There are a few parts we want this project to encompass:

• the code that trains models, but also
• PyPI package(s) or other integrations
• subsets of "raw" data (CM2.6)
• subsets of "processed" data (forcings)
• pretrained models

We should have a general understanding of how we approach each of these. We know where code will be stored (here), but what about packaging, pretrained models etc.?

[raehik]

Some notes from a meeting on 2023-10-17, focusing on the inference step of the project:

• MLflow is tightly interlinked in the inference step: model weights and various other bits are loaded through an MLflow run ID. This is handy for local runs and testing, but complicates more ad-hoc usage (e.g. loading model weights from a file and using in inference mode).
• We want to enable loading weights from files, and automatically downloading weights from HuggingFace.
• The original author wants to retain the convenient MLflow interface.
• We propose enabling both methods of running the inference step. This should involve figuring out what is loaded from previous MLflow runs and passing those in as command-line arguments.

[raehik]

I don't think this issue is well-defined. In any case, I've added notes in #97 clarifying how the project is packaged (and noting that we haven't uploaded to PyPI, but it wouldn't be hard) and what data is stored on HuggingFace.