First version of the SD-Core bundle with tooling to generate bundles from code

gruyaume commented 1 year ago

Overview

Tooling to generate SD-Core bundles from code. Right now, the only SD-Core bundle variant that is supported is the sdcore-user-plane. In the future, more variants will be added. This will make it easier to support multiple charm bundles containing variants of the same operators.

Usage

To render the SDCORE_USER_PLANE bundle and output it in the bundle directory:

pip3 install -r render_bundle/requirements.txt
python3 render_bundle/render_bundle.py --bundle_variant SDCORE_USER_PLANE --output_file bundle/bundle.yaml

This will generate the following bundle:

bundle: kubernetes
name: sdcore-user-plane
description: The SD-Core User Plane bundle contains the 5G User Plane Function (UPF).
applications:
  upf:
    charm: sdcore-upf
    channel: edge
    scale: 1
    trust: true
  grafana-agent-k8s:
    charm: grafana-agent-k8s
    channel: latest/stable
    scale: 1
    trust: false
relations:
  - - upf:metrics-endpoint
    - grafana-agent-k8s:metrics-endpoint

The Charm Bundle Generator Library

The project leverages a new library called charm_bundle_generator. This library makes it easy to render any charm bundle from pydantic charm objects (Application, Relation, Resource, etc.) Currently the lib is part of the project, but in the future, it could be extracted to a pypi package.

Usage

Running the following python script:

from lib.charm_bundle_generator import Application, CharmBundle, Relation

bundle = CharmBundle(
    description="My wonderful bundle description",
    name="My wonderful bundle name",
    applications=[
        Application(name="app-1", charm="app-1-charm"),
        Application(name="app-2", charm="app-2-charm"),
    ],
    relations=[
        Relation(
            app_1_name="app-1",
            app_1_relation_name="banana",
            app_2_name="app-2",
            app_2_relation_name="fruit",
        )
    ],
)
bundle.render(output_file="bundle.yaml")

yields the following bundle.yaml content:

bundle: kubernetes
name: My wonderful bundle name
description: My wonderful bundle description
applications:
  app-1:
    charm: app-1-charm
    scale: 1
    trust: false
  app-2:
    charm: app-2-charm
    scale: 1
    trust: false
relations:
  - - app-1:banana
    - app-2:fruit

saltiyazan commented 1 year ago

Great work. It's a very clean and easy to read and follow way build bundle files. I'm curious how will this look like when there are more bundle files and some of them containing more charms. I'm also a bit skeptical regarding the number of steps one needs to take to build a bundle file VS. the added value it brings compared to writing and maintaining the bundle files manually in bundle.yaml or in jinja templates.

gruyaume commented 1 year ago

Great work. It's a very clean and easy to read and follow way build bundle files. I'm curious how will this look like when there are more bundle files and some of them containing more charms. I'm also a bit skeptical regarding the number of steps one needs to take to build a bundle file VS. the added value it brings compared to writing and maintaining the bundle files manually in bundle.yaml or in jinja templates.

Those are good questions. The rationale behind this is:

We will need to support multiple bundle variants for the SDCORE project (4), each of which with the option to deploy from a different channel or from a local charm. This makes for a total of 16 different bundles. Maintaining 16 different bundles will make it hard not to make mistakes when we make changes. Therefore, a way to automatically render such bundles makes sense.

One option is to use 1 Jinja template that contains all the ifs and fors needed for the SD-CORE bundles. While this is possible, it is hard to read a large jinja template with such logic in it, especially with all the variations we'll need.

A second option is to have 4 jinja templates for the 4 bundle variants. But again, some of those would have a lot in common and we will have to remember that changes need to be applied at multiple places.

A third option is the proposal in this PR. It comes with the following advantages:

All the bundle content logic is handled in Python code. We are all used to writing, reviewing, testing Python Code.
Changes can be done in only 1 place. Ex. if one needs to make change to the UPF resource, they only need to make the change in the UPF class.

Adding a new bundle to the list of variants isn't especially complex either. You need to create a new class in the bundles.py (ex. SDCoreCP(CharmBundle)) and have it contain all the application/relations that make such bundle (amf, nrf,, etc.).

End users won't use this project. The goal for this project is to make our life simpler when making changes to our bundles.

As for us, to build a bundle.yaml file, it takes exactly 1 step.

pip3 install -r render_bundle/requirements.txt
python3 render_bundle/render_bundle.py --bundle_variant=SDCORE_UP

saltiyazan commented 1 year ago

Again, really nice idea and work. Looks good to me so it's approved, but I remain a bit hesitant to decide which is the better approach, this one or the one where we have separate jinja templates for all bundles. I assume it doesn't need to be decided now. I will give it a bit more thought to compare.

canonical / sdcore-bundles