Can modules use/mimic bundles?

[m13t]

Hi Stefan,

Based on the documentation regarding modules and bundles, these appear to be very distinct separate pieces of functionality and the only way they interoperate is by bundles including modules. However bundles aren’t packaged up as OCI images (that I can see) so they’re not stored and version in a repository, so I suppose source control would be the only place to store these for reusability.

Essentially what I’m wondering is, can a module not have a dependency on another module along with its own resource, similar to how one might author a helm chart that includes a dependent chart plus other resources? This is of particular interest where you might have a reusable chart with some common pieces in which is included by a lot of other charts.

Understand this is the purpose of bundles but as these distinct from published and versioned modules, there’s no really good way to distribute these.

Is this a design decision or just something that’s not been considered so far?

[stefanprodan]

Understand this is the purpose of bundles but as these distinct from published and versioned modules, there’s no really good way to distribute these.

Currently Bundles must be kept in Git and applied in CI. The Timoni Bundles/Modules relationship is similar to Docker compose files/container images, Terraform scripts/providers, Helmfile/charts, etc.

Distributing Bundles as OCI artifacts was problematic due to the embedded values, that in many cases would contain secrets. But now with the introduction of the Runtime API, where secret values could come from the cluster or from env vars (CI secrets) makes me comfortable with the idea of packaging Bundles with their Runtime and distribute them somehow.

Can a module not have a dependency on another module along with its own resource, similar to how one might author a helm chart that includes a dependent chart plus other resources?

Once CUE implements package management, you'll be able to publish and reuse CUE code in your modules, in the same way you can reuse Go packages in Go projects.

[m13t]

Thanks for the info, this all makes sense, I just wasn't sure of what the reasoning was.

I was specifically curious about how one might distribute a larger solution of multiple components as a single package that end-users can simply consume and provide values. I suppose an example could be something like a couple of service applications, and an optional set of data store components. Specifically so me as the author of the service can include the option of something like mysql or postgres and simply providing the option in the values to select which one but having authored the module, can ensure that the version and configuration for the data store would be compatible with the applications in the module. I suppose this could be considered an anti-pattern in some regards and should instead provide the requirements for the data store to the end-users rather than bundling, but that is commonplace with a lot of Helm charts and would certainly be useful for in-house things.

[stefanprodan]

IMO Timoni Bundles are better suited to express complex applications than Helm umbrella charts as Timoni Bundles offer a clean separation between individual components (modules) and how they interact with each other. Unlike Helm values, a Timoni Bundle is a program, you take inputs from the runtime and you can configure the various interactions between the app components.

For example:

bundle: {
    _profile:  "dev"  @timoni(runtime:string:ENV)
    _password: string @timoni(runtime:string:REDISPASS)

    apiVersion: "v1alpha1"
    name:       "podinfo"
    instances: {
        //deploy redis only on dev, on prod use Amazon ElastiCache
        if _profile == "dev" {
            redis: {
                module: {
                    url:     "oci://ghcr.io/stefanprodan/modules/redis"
                    version: "7.0.11"
                    digest:  "sha256:0e0f40e7824ff8c59e309d9b8bf19e235c6e779c93aafeeeaa86cbca49989950"
                }
                namespace: "podinfo"
                values: {
                    maxmemory: 256
                    readonly: replicas: 1
                    password: _password
                }
            }
        }
        podinfo: {
            module: {
                url:     "oci://ghcr.io/stefanprodan/modules/podinfo"
                version: "6.3.6"
                digest:  "sha256:b317bc7e4736287af10cbf52f9c0f1e8cd1f8c180bba29bc563fe697bfa2dd3d"
            }
            namespace: "podinfo"
            values: {
                caching: {
                    enabled: true
                    // In-cluster Redis in dev
                    if _profile == "dev" {
                        redisURL: "tcp://:\(_password)@redis:6379"
                    }

                    // Amazon ElastiCache in production 
                    if _profile == "prod" {
                        redisURL: "tcp://redis-01.7abc2d.0001.usw2.cache.amazonaws.com:6379"
                    }
                }
                // HPA in production 
                if _profile == "prod" {
                    autoscaling: {
                        enabled:     true
                        minReplicas: 2
                        maxReplicas: 10
                        cpu:         90
                    }
                }
            }
        }
    }
}

[m13t]

I think this makes a lot of sense, and having it more opinionated will likely lead to less abuse of capabilities like you see with Helm charts. I do like the clear separation, even if a bundle isn't packaged up.

As part of a bundle, can you also include resources as you would in a module? For example if you had a bundle to include some component but you then also needed to create something like a TargetGroupBinding resource for the AWS Load Balancer Controller, or perhaps an Istio VirtualService?

[stefanprodan]

As part of a bundle, can you also include resources as you would in a module?

No, but as with the HPA example above, you could express that as a feature inside the Module that you enable in the Bundle and you set things like AWS Account ID, IAM Role, etc from the Runtime.

[stefanprodan]

Anyway I'm not dismissing the possibility in the future to allow an ad hoc resource set to be define inside a Bundle.

[m13t]

Sure, appreciate its very early days. I'm very fond of things so far and curious how things develop over time so I've just been thinking about things I do currently with Helm or Kustomise and how those kinds of things will work with Timoni.

Currently I have a lot of clusters that use a lot of common off the shelf helm charts for things you'd expect like Prometheus, Grafana, cert manager and all that kind of stuff. So what I was wondering is should these be ported to Timoni modules, you wouldn't expect the module authors to cater for every kind of ingress solution that exists. I could use a bundle to install all my cluster dependencies, but I'd then have to create a distinct module to tie in the ingress. Current solution would be a wrapper chart that pulls in the component, and then includes say an Istio manifest to expose the service, or whatever kind of ingress controller you're using.

Perhaps having a dedicated module to tie all these things together isn't bad, it would be more clear in some ways, just a little more verbose.

[stefanprodan]

Currently I have a lot of clusters that use a lot of common off the shelf helm charts for things you'd expect like Prometheus, Grafana, cert manager and all that kind of stuff.

How I would deploy these off the shelf Helm charts is by creating a Timoni module containing Flux HelmRelease definitions.

In the future I plan to create a Timoni controller for Flux, the Bundles would be fetched from Git or OCI by Flux source-controller and reconciled by timoni-controller.

[m13t]

How I would deploy these off the shelf Helm charts is by creating a Timoni module containing Flux HelmRelease definitions.

Sorry, I was meaning in a Timoni world where software creators offer native Timoni modules :)

[stefanprodan]

Ah Ok I understand now, you would want to supply resources directly in the Bundle, instead of having some Module for the ingress or other glue material. Not sure if this doesn't blur the line too much, as Bundles then have to import Kubernetes schemas and CRDs... it gets fast really messy.

[m13t]

Yes that's what I was thinking exactly, and again I'm essentially transposing existing patterns here and just figuring out what the Timoni way would be.

I hadn't thought about the issues with the schemas for bundles. That would certainly over complicate things and make the whole experience worse. As you said before, if a Bundle is to Timoni what a compose file is to Docker, then you would want to keep that as clean and declarative as possible.

[m13t]

Anyway, thanks for taking the time to discuss, much appreciated. Happy if you want to close this issue off.

[stefanprodan]

Hey @m13t today I've released Timoni v0.14.0 which comes with support for distributing bundles & runtimes. Please see https://timoni.sh/bundle-distribution/ and let me know if that helps.

[m13t]

Brilliant, I'll definitely take a look - thank you.