Documentation: Architecture of GeoIPS

[biosafetylvl5]

Due on for review on: June 10, 2024

Documentation Task

• Create comprehensive documentation on the architecture of GeoIPS, including an overview of the interfaces, plugins, families, and their interactions. This will encompass conceptual explanations and high-level information without delving into code-level specifics.

Files to be edited

• docs/source/new-docs/concepts/architecture/index.rst
• docs/source/new-docs/concepts/architecture/interfaces/index.rst
• docs/source/new-docs/concepts/architecture/interfaces/yaml-based/index.rst
• docs/source/new-docs/concepts/architecture/interfaces/module-based/index.rst

Resources to be created

• UML or Mermaid rendering of the architecture layout
• Diagrams depicting the interactions between interfaces, plugins, families, module-based, and YAML-based interfaces

Scope of Documentation

• Does NOT include

  • Instructions on how to use GeoIPS
  • Embeddings from code, unless they are explicit and short examples
  • Future implementation plans

• Does include

  • Conceptual information on the architecture of GeoIPS
  • High-level overview of interfaces, plugins, families, etc. and their interactions
  • Explanations of why architectural design choices were made (even if it's just "that was the easiest answer")

Checklist for Completion

• [ ] Work done on a feature branch, eg. documentation-adding-functionality
• [ ] Write the required documentation in RST.
• [ ] Add any required resource files to a sub-folder
• [ ] #620
• [ ] #621
  • [ ] #622
  • [ ] #623
• [ ] Run ./docs/build_docs.sh html_only to ensure the documentation builds correctly, inspect the result, and edit as required to achieve a well formatted document.
• [ ] Update the relevant release notes.
• [ ] Readable and followable writing, please use a grammar checker + spell checker
• [ ] Passes doc8 checks, see the sphinx RST Primer and checkout this black-like tool @biosafetylvl5 wrote for auto-formatting RST files: pink
• [ ] A PR from your feature branch to main 😊 with @biosafetylvl5 as a reviewer

[biosafetylvl5]

Here's a potential outline:

GeoIPS (Geolocated Information Processing System)
===============================================

I. Introduction
---------------

**Core Functionality and Role of GeoIPS**
- Core GeoIPS does not perform direct computations on data.
- Provides helper functions that can be utilized by plugins.
- Ensures a modular approach, allowing external contributions and customization.

II. Plugins
-----------

**Definition and Purpose**
- A plugin is a piece of code or configuration external to the core code GeoIPS. We refer to this as core GeoIPS.
- Integrates external functionality into GeoIPS for specific computations/actions.

**Flexibility and Modularity**
- Allows users to write and integrate various types of plugins, such as algorithms and color mappers.
- Supports modularity, enabling plugins to handle specific tasks within a larger processing workflow.
- Encourages interoperability and community engagement by allowing contributions from different users.
- Example tasks handled by plugins:
  - Data processing algorithms
  - Visualization configurations (e.g., color mapping)
  - Formatting outputs
  - etc.

III. Interfaces
---------------

**Definition and Role**
- An interface in GeoIPS defines an ontology: a set of concepts and categories for plugins.
- Specifies the properties of plugins and their relationships.

**Examples of Plugin Categories**
- **Algorithms**: Define methods for processing data.
- **Color Maps**: Determine the color schemes used in visualizations.

IV. Types of Plugins
--------------------

**Module-based Plugins**
- Written in Python modules.
- Typically contain computational code that can manipulate data, but not necessarily.
- Example: An algorithm plugin written in Python that interpolates brightness temperatures.

**YAML-based Plugins**
- Written in YAML, which is a superset of JSON.
- Serve as configurations for module-based plugins.
- Define behaviors such as plot annotations or specific ways to generate products.
- Example: A YAML file specifying the color scheme and annotations for a plot generated by a module-based plugin.

V. Proc Flows
-------------

**Definition and Purpose**
- Procflow stands for processing flow.
- Defines the sequence and logic of processing steps within GeoIPS.
- Utilizes interfaces to validate and retrieve necessary plugins.
- Example: A proc flow might specify the order of applying an algorithm, then formatting the output, and finally generating a visualization.

VI. Families of Plugins
-----------------------

**Definition and Role**
- Families of plugins define the input and output of variables.
- Aids in determining the ordering of different plugin execution.
- Example: (?)

**Warning**
- The concept of families is expected to be deprecated/made unnecessary.

VII. YAML-based Schemas
-----------------------

**Purpose and Functionality**
- Define the correct format for YAML-based plugins.
- Used for validation purposes to maintain consistency and correctness.
- Example: A schema defining required fields and valid values for a color map configuration.

VIII. Conclusion
----------------

**Summary of Key Points**
- GeoIPS leverages a plugin-based architecture for flexibility and community engagement.
- Core GeoIPS does not compute data directly but provides helper functions.
- Interfaces define an ontology of plugin categories and their interactions.
- Plugins can be module-based or YAML-based, serving different purposes.
- Proc flows manage the execution sequence of plugins.
- Families of plugins define input/output relationships, soon to be replaced by order-based proc flows.
- YAML-based schemas ensure the validity and structure of YAML-based plugins.

[biosafetylvl5]

@jsolbrig ^ here's an outline