Discussion: naming of "app of apps"

kuosandys commented 2 years ago

This is the issue to discuss the naming of:

"app of apps"

This is the generic term used to describe the concept of an app representing a collection of other apps. When such an app's app CR is created, it results in the creation of app CRs for each individual app.

Examples of an "app of apps" would be the security-pack app, the default-apps-openstack app, and the default-apps-aws app.

An alternative to "app of apps" would be "app pack" but we are already using "cloud native packs" and for default apps these are not app packs.

Metadata

We also need a way to identify "app of apps". The application.giantswarm.io/in-cluster-app annotation in the "parent" app's Chart.yaml aims to:

indicate the hierarchical relationship of this app with its "child" apps (this is in combination with the giantswarm.io/managed-by:<parent-app> label on each of the "child" apps)
ensure that the "app of apps" app CR is created correctly.

MarcelMue commented 2 years ago

Adding @Oshratn here to also get her input.

puja108 commented 2 years ago

I like "collection" in this context, it's also a word we've already been using internally in our releng stuff.

Another interesting naming would be "composition", which I got inspired to by crossplane's composite resources. It's a bit more elegant than collection, but that might also be a downside as it saises expectations above it just being a collection.

Oshratn commented 2 years ago

The image I am getting from the description is something unfolding or spawning more stuff. Now I need to find an elegant word or turn of phrase. I'll have a think.

Oshratn commented 2 years ago

First I'd like to say that collection is a fine word. That being said I consulted my thesaurus and did some associative thinking to ensure that we didn't overlook an amazing option. So I am sharing my list as a brain dump and marked my favorites in bold.

spread fan compound consolidation amalgam alloy coalition coalescence medley aggregate/aggregation conglomerate composite chain combination array constellation sequence assemblage assortment

marians commented 2 years ago

I'll give my +1 for app collection.

BTW, I think depending on context it might also possible to use "collection" only, without the "app". But the official term should be "app collection" I think.

We will also need terms for the parent/child app. There is the app that defines the app collection. Then there are the apps that are the "members" of the collection. How to name these?

MarcelMue commented 2 years ago

The only issue I see currently is that we use collection already in a different context: https://github.com/giantswarm/aws-app-collection

The technical implementation is completely different there and also the purpose. It's GS only right now but it will 100% lead to confusion IMO

gianfranco-l commented 2 years ago

I cannot attend the brainstorming session (because of an overlapping customer call) so I'll leave my inputs below.

+1 on Marcel's input: I think app collection is guaranteed that will create confusion internally with the term app-collection being used already for a different concept with a different purpose in a different context.

Another term that I find confusing is stack: it implies a relationship between the "parent" and the "children" that is different from the real relationship that is in place in the "app-of-apps" approach.

I agree with Marian that the term should be in some way reflective of the relationship between the parent and the child apps, so that it is natural then to identify the terms to be used for both the approach, the parent app and the child apps.

Another key aspect I think is worth to keep in mind and make explicit with the naming is that the "parent" and the "children" have the same logical structure and functioning, i.e. they are both 'apps'.

The only term that comes to my mind that would satisfy all of the above is the term wrap: the wrapper app(parent) contains(type of relationship) the wrapped apps (children). Both the wrapper and the wrapped apps do have the same logical structure and functioning (they are apps), the only difference being the wrapper contains apps. I also like that wrap (or app-wrap) is extremely short. I do recognize though is a term that does not communicate fanciness. It also sounds a bit like a wordplay (it can be a plus, actually :D )

Furthermore, I like the term bundle (and bundled apps), it is missing though a clear name for the parent app that make clear the kind of relationship in place (the bundle-app and the bundled apps does not sound nice to me)

I like also composition , it sounds elegant too. However, to me it does not communicate that the composition-app and the app in the composition have the same structure and functioning. I find this term also very hard to pronounce and just to raise a bit too much the expectations about what the concept it really is all about.

To make an analogy to foster some ideas' association, I see the "app-of-apps" approach similar in concept to the nesting dolls, so app nesting, nest-app and nested apps is another suggestion I am throwing in. Again, like the wrap idea, it is short, explain the whole idea and the relationships between the entities in play, though it is not communicating fanciness.

rossf7 commented 2 years ago

My 2 preferred names are bundle and collection. Bundle has the benefit of not being in use and I like how our usage aligns with this dictionary definition.

A group of things fastened together for convenient handling https://www.merriam-webster.com/dictionary/bundle

it is missing though a clear name for the parent app that make clear the kind of relationship in place (the bundle-app and the bundled apps does not sound nice to me)

The child apps also exist as standalone apps so I think the bundle name still works. e.g Falco is part of the security bundle. You can install Falco standalone or as part of the security bundle.

puja108 commented 2 years ago

Considering the internal confusion that collection would bring I'd +1 bundle here. I'd like to keep this wording simple, as it is really a very simple concept, it is not anything fancy nor does it have much added value beyond the grouping usability itself and effects of a grouped versioning (which could even be a downside in some eyes)

MarcelMue commented 2 years ago

It seems like we have soft agreement on bundle at least here then. How should we move forward with the decision making?

Another session to agree or async voting on e.g. slack?

MarcelMue commented 2 years ago

Or simply writing an PDR/ADR/whatever?

kuosandys commented 2 years ago

It seems like we have soft agreement on bundle at least here then. How should we move forward with the decision making?

Another session to agree or async voting on e.g. slack?

I think with a soft agreement here already, we can proceed with announcing the proposal on Slack. If there are any vetoes or concerns, we can schedule a session for further discussion.

MarcelMue commented 2 years ago

Sounds like a plan to me :rocket:

marians commented 2 years ago

Please also add the term(s) coined here to https://intranet.giantswarm.io/docs/glossary/

teemow commented 2 years ago

So to have the security pack you need to install the security bundle? Doesn't this sound inconsistent? I get that cloud-native packs and app packs aren't the same. But also cloud native packs isn't something that is written in stone forever.

On the management cluster would you rather install the aws bundle to be able to create clusters on AWS or the aws pack? Speaking of the language in our UIs.

Given that we are a little bit further with our cloud native packs and a default cluster would have all observability, connectivity, security and dx functionalities installed. Would we use the word "pack" to install/uninstall a bundle?

If a new cluster release is installed via the app platform. Would we still call this an app? The repos are now prefixed with cluster-* (eg https://github.com/giantswarm/cluster-openstack). If we don't call it an app anymore how do we call app platform?

For me these discussions are quite related and I am not sure if bundle or app bundle makes it even more complicated to understand for our users.

ghost commented 2 years ago

We chat this in @giantswarm/team-rainbow and we can only help with providing advice on choosing a proper name then ownership of driving it would be in @giantswarm/team-honeybadger.

gianfranco-l commented 2 years ago

Shall we bring this to SIG Product given the scope of the naming involved? When it came to choose the name "cloud native packs" this was discussed in SIG PRoduct WH (and finalized in Team Horizon/SIG Company maybe?), being the scope here even larger (as hinted by Timo's comment), I'd say that maybe SIG Product is the right forum for this?

@PrashantR31 @marians what are your inputs and advices in choosing a proper name?

cc @giantswarm/team-honeybadger

ghost commented 2 years ago

My preference goes to:

1st choice: app collection (meets the purpose and sounds posh as well) 2nd choice: app gallery

But yeah, let us bring that to SIG Product WH for a quick discussion

gianfranco-l commented 2 years ago

interesting perspective (credits to @MarcelMue for finding it)

App of apps pattern used in ArgoCD, where they also use the terms bundle and app bundle internally

https://medium.com/dzerolabs/turbocharge-argocd-with-app-of-apps-pattern-and-kustomized-helm-ea4993190e7c

update: at ArgoCD, app bundle name has been replaced by the term ApplicationSet

https://argo-cd.readthedocs.io/en/stable/user-guide/application-set/

marians commented 2 years ago

Responding to @teemow

So to have the security pack you need to install the security bundle?

That depends on how we represent and name these bundles. I could imagine that we have a bundle called "security pack". In terms of our product terminology it's a pack. Technically it's an app bundle to be installed.

Doesn't this sound inconsistent? I get that cloud-native packs and app packs aren't the same. But also cloud native packs isn't something that is written in stone forever.

For me it works. For users to work, we probably have to be consistent and careful with our use of the term "pack" vs. "bundle" especially in documentation. The term "app bundle" will probably not appear on the marketing level communication, but it will definitely in technical docs.

On the management cluster would you rather install the aws bundle to be able to create clusters on AWS or the aws pack? Speaking of the language in our UIs.

So far I haven't heard of a plan to offer an "AWS pack" or "Azure pack".

For creating a workload cluster in CAPI, I think the technical description would be something like "configuring and installing a cluster app bundle". But I'm not 100% savvy with the concepts here. Maybe Marcel can help to propose wording.

Given that we are a little bit further with our cloud native packs and a default cluster would have all observability, connectivity, security and dx functionalities installed. Would we use the word "pack" to install/uninstall a bundle?

As above, in my opinion it could be about (un)installing an app bundle named "observability pack", "security pack" etc.

If a new cluster release is installed via the app platform. Would we still call this an app? The repos are now prefixed with cluster-* (eg https://github.com/giantswarm/cluster-openstack). If we don't call it an app anymore how do we call app platform?

To me this depends on the question whether these are a single app, or an app of apps. If the latter, then they should be called something with "app bundle".

teemow commented 2 years ago

Thanks @marians. Your differentiation makes sense. A good understanding of both concepts internally is important so we don't confuse customers with our terminology.

So far I haven't heard of a plan to offer an "AWS pack" or "Azure pack".

To clarify this a bit. My is that in the future you can install the CAPI controllers for different providers on a management cluster. But they will be optional. Whether we call them "bundle" or "pack" is still hypothetical. I just wanted to bring this up to see how the terms can evolve in the future.

To me this depends on the question whether these are a single app, or an app of apps. If the latter, then they should be called something with "app bundle".

Regarding my last point. It was more about the word "app" not about "bundle". My question was if it will stay semantically correct/useful to keep calling the charts => apps. Do we consider a helm chart that only installs CRs for our API to be an "app"? To be clear: It is not a CRD and not an operator or any kind of other service or executable. It is just a set of manifests that define a cluster. In the future we could have helm charts that configure crossplane (eg to connect a cluster with the datacenter of the customer), And maybe more things. In a way "app-of-apps" is something similar. It is an app that installs CRs for more apps. If we find a new name for this concept, do we also want to find a new name for the cluster definition or is this still an app? Or do we want to define the general concept of an abstraction within our API?

Ok. Let's not get too philosophical. I just wanted to make sure all things have been considered. All good from my side. I know naming is hard and I don't want to make it more complicated then currently necessary.

marians commented 2 years ago

Do we consider a helm chart that only installs CRs for our API to be an "app"?

If something is really only a Helm chart and does not use an App CR, or app-operator, then I think there is no need to call it anything else than "Helm chart" and we should not call it "app".

My understanding is that the app of apps we talk about here is an App CR that when installed will result in more App CRs being installed etc.

teemow commented 2 years ago

I think you still haven't understood how the capi releases work. If there is an AppCR with a cluster release the app-operator installs a helm chart on the Management Cluster that only contains CAPI CRs. There is no real app. But we call it app and AppCR.

gianfranco-l commented 2 years ago

as agreed during SIG product, we'll discuss this on SIG Product WH on April 28th . cc @MarcelMue @piontec @rossf7

puja108 commented 2 years ago

To be honest, an App was always just a pointer to a Helm chart and never needed that to be an actual "application" (as in "install a software application"), it was always just a bunch of YAML (we had the case already in the RBAC bootstrap app for example that it contained only some RBAC config), that's also why Batman was reluctant to call it Application, but we also never came up with anything better than App. In the end we just relied on "App" = "A vehicle to deploy set of K8s Resources".

Now the context of re-using it for cluster releases changes the context a bit, so we could rethink it, but we should also know that it's not a simple thing to just rename a CRD, it means migration in a way that is not directly supported in K8s. Open to suggestions though.

I was just thinking to keep it simple, we'd just introduce the new concept of a bundle for now.

We still have quite a bit of work to do on the actual UX of the concepts as they are evolving no matter the naming, especially in the context that an App can now contain the definition of a cluster and it's upgrades and the fact that the cluster deployment is actually 2 connected apps, one of which is a bundle. So, yeah, lots of things to consider, of which the actual naming is a smaller one IMHO.

teemow commented 2 years ago

Both bundles and clusters are the same kind of abstractions. They are a bunch of kubernetes resources. But kubernetes resources that we call "Management API". This is why I wanted to think about it as a whole. I am fine with calling it a bundle and somehow making it a first class citizen (however this materializes). But do we treat clusters the same way? Is the cluster app concept a first class citizen too?

puja108 commented 2 years ago

I think that might be a very good way to think about it, with naming app-of-apps to e.g. bundle we make it more or less a first class citizen. Doing the same with the very special cluster-app, so I could imagine us naming that, too, to make it more first class.

teemow commented 2 years ago

Taking it even further. In case we extend the management api with crossplane there could be apps with specific crossplane manifests too. Example: attach a Transit Gateway to the management cluster. This will contain multiple AWS resources that are pre-configured and can be tweaked via a values.yaml. These apps could be called infrastructure or so. You could have bundles of multiple clusters and infrastructure too.

teemow commented 2 years ago

Ok I stop now. The name bundle is fine with me.

gianfranco-l commented 2 years ago

Since everyone settled on the term bundle (at least for now 😅 ) , shall we remove this task from SIG product SWH and consolidate the decision in a PDR/ADR? cc @puja108

kuosandys commented 2 years ago

Hi @gianfranco-l 🙂 We (@MarcelMue and some of @giantswarm/team-rainbow) talked about the next steps following on from this issue at SIG UX sync on Tuesday, 19th April. As this naming process has been more involved than expected, we decided I'd bring the decision to the SIG Product sync on Monday to get a final thumbs-up from everyone before proceeding with documentation. Does this timeline align well with your team's current plans?

gianfranco-l commented 2 years ago

@kuosandys fine for me, thank you!

gianfranco-l commented 2 years ago

in SIG product it was decided to settle on app bundle for now so we will proceed with documenting this in GS intranet glossary and relevant docs. @kuosandys are you ok with honeybadger documenting this now or do you see this in rainbow?

kuosandys commented 2 years ago

in SIG product it was decided to settle on app bundle for now so we will proceed with documenting this in GS intranet glossary and relevant docs. @kuosandys are you ok with honeybadger documenting this now or do you see this in rainbow?

I think honeybadger would be the best suited to take this forward 👍 Please feel free to close this issue as you see fit, as well.

piontec commented 2 years ago

OK, let me summarize this a bit and with some examples and my favorite bullet points, so we can check we really think the same. This is wider scoped, but I tried to address a few related concerns at the same time. Please give me your +1/-1.

Definitions

Note: Please think about ? in parentheses.

Any Helm Chart (doesn't matter what it contains, might have or have not an App CR inside) delivered by App Platform is still called an "app"
A Helm Chart that contains (only?) a set (more than 1?) of App CRs is called an "app bundle" or "bundle" for short.

Ideas

Note: Let's reflect our ideas in repo naming as well.

"Bundles" are a software implementation used to deliver a "Cloud Native Pack (CNP)". I.e. "Security CNP" (or "Security Pack" for short) is deployed by installing a bundle that lives in the security-bundle repo and delivers an app bundle called "Security Pack". If a bundle is part of a CNP, it has to have "Pack" in its name. Otherwise, the bundle should just use "bundle" naming.
A "bundle" is a subtype of an "app" (an "app" specialization). I propose to add a Chart.yaml annotation like: application.giantswarm.io/app-type: bundle to capture that explicitly (super useful for UX later on).
In the future (or already) different specializations of an app can be created. For clusters delivered as apps, we should create "cluster" app specialization and annotate it with application.giantswarm.io/app-type: cluster. In the example of Crossplane, we can easily extend it to something like application.giantswarm.io/app-type: infra.

Examples and naming consequences

Any repo that provides a single app should have -app suffix. Example: kyverno-app.
Any repo that uses app specialization, should have a suffix reflecting this specialization:
1. Any repo that provides a bundle should have -bundle suffix. Example security-bundle. This is irrelevant to a CNP existing for that bundle. We can have debugging-bundle that offers a hypothetical set of cluster debugging tools without any CNP offering for it.
2. Any repo that provides clusters as apps should have a -cluster suffix, like openstack-cluster.

teemow commented 2 years ago

As I understand the idea now is that a bundle is a helm chart. So within a helm chart the manifests would contain AppCRs and there would be a values yaml to configure the config maps for those apps, correct? In the GitOps respository we would then add an AppCR with a config map that would then contain all configurations for all applications in the app bundle.

Like this: Git repo contains AppCR and Configmap=> installs bundle chart with configuration => which installs AppCRs with Configmaps

Will configuration be passed through without the need to change the bundle? Do you already have an idea how this would work?

As an alternative. How much sense would it make to have a bundle as a main citizen (CRD) in the API? So the git repo would contain a bundle manifest and configmaps for all apps.

Looking at other package managers this is typically solved via dependency management:

So in the case of these package managers their meta packages (what we call bundle) would be a plain AppCR with a list of dependencies on other Apps. I guess it could also have a helm chart but it doesn't need to have one. This would be less semantic than having bundles as first class citizen (CRD). But we definitely need to implement dependencies too at some point.

What's your opinion here? I'd like to figure out how we can make this with as few indirections for the users as possible. If you need to jump three places to understand how and what to configure apps in app bundles this will be a frustrating experience.

And speaking of indirection and asking the big question again: Are we really sure that we need the AppCR concept in the long run? Wouldn't it be nicer to just use helm releases in flux and plain helm along the way? Which features do we believe will never be part of flux or helm that we need to implement ourselves? Why can't we add those features to helm instead of creating an additional concept?

Helm itself has dependencies: https://helm.sh/docs/helm/helm_dependency/ Helm releases have too: DependsOn https://fluxcd.io/docs/components/helm/helmreleases/

"Bundles" are a software implementation used to deliver a "Cloud Native Pack (CNP)". I.e. "Security CNP" (or "Security Pack" for short) is deployed by installing a bundle that lives in the security-bundle repo and delivers an app bundle called "Security Pack". If a bundle is part of a CNP, it has to have "Pack" in its name. Otherwise, the bundle should just use "bundle" naming.

I agree that packs can be implemented by bundles. It could even be bundles of bundles. Example: Monitoring bundle (installs prometheus, alertmanager, grafana), Logging bundle (installs loki, promtail, grafana), Tracing bundle (installs ....) => observability pack (installs monitoring, logging and tracing bundle).

piontec commented 2 years ago

A ton of valid points, Timo :) Let me try to answer in-line. Still, the most important assumption first: we need some concept and implementation of a "bundle" right now, without waiting for something we might want or need to implement ourselves. So, it's not the final solution, but a step on the way; a one we can have right away, as all the features are already there.

As I understand the idea now is that a bundle is a helm chart. So within a helm chart the manifests would contain AppCRs and there would be a values yaml to configure the config maps for those apps, correct? In the GitOps respository we would then add an AppCR with a config map that would then contain all configurations for all applications in the app bundle.

Like this: Git repo contains AppCR and Configmap=> installs bundle chart with configuration => which installs AppCRs with Configmaps

Exactly.

Will configuration be passed through without the need to change the bundle? Do you already have an idea how this would work?

Depends on bundle implementation: what you make available (and how you make it available) in values.yaml.

As an alternative. How much sense would it make to have a bundle as a main citizen (CRD) in the API? So the git repo would contain a bundle manifest and configmaps for all apps.

Quite a lot, with some "buts". I was discussing this mainly with Ross for a long time. Our solution for now was "to check what we can get with flux, with as little effort on our side as possible". That's the result of it. ("buts" are addressed below).

So in the case of these package managers their meta packages (what we call bundle) would be a plain AppCR with a list of dependencies on other Apps. I guess it could also have a helm chart but it doesn't need to have one. This would be less semantic than having bundles as first class citizen (CRD). But we definitely need to implement dependencies too at some point.

What's your opinion here? I'd like to figure out how we can make this with as few indirections for the users as possible. If you need to jump three places to understand how and what to configure apps in app bundles this will be a frustrating experience.

And speaking of indirection and asking the big question again: Are we really sure that we need the AppCR concept in the long run? Wouldn't it be nicer to just use helm releases in flux and plain helm along the way? Which features do we believe will never be part of flux or helm that we need to implement ourselves? Why can't we add those features to helm instead of creating an additional concept?

OK, here we come to the core (and far away from the original topic of "naming bundles").

I think that App CRs as they are now (a way to configure a Helm Chart for a remote cluster) are making less and less sense, because we have other projects that can basically do the same. Still, having App CRs at our disposal, opens some really interesting possibilities, some of which we're already investing in/investigating. They include:

extending Helm's meta-data with our custom meta-data: info about deployment restrictions (number of simultaneous deployments, supported infra providers, ...), info about tests-confirmed upgrade paths (in progress, but paused), info about real platform release compatibility (on our roadmap in the "soon" section);
abstracting actual app package format and delivery mechanism (if in the future we want to switch chart-operator to something else or even Helm Chart to something else, we have an abstraction layer that can hide such migration);
real package management: checking dependencies, including what is installed on a remote cluster in terms of dependencies and detecting missing stuff, version restrictions, abstract (bundle) packages.

Obviously, features above need our code and work.

Helm itself has dependencies: https://helm.sh/docs/helm/helm_dependency/ Helm releases have too: DependsOn https://fluxcd.io/docs/components/helm/helmreleases/

That's not what we need. These are "sub-chart" dependencies - what is a dependency is packed inside your chart and delivered together. I.e. it means that if you have 3 Charts which need Grafana, every one of them will include its own Grafana. We found Helm's dependencies very limited.

DependsOn in Flux is much better, as it's dynamic (checked runtime, not build-time), and it supports some version restrictions and such. Still, all the restrictions must be specified in Flux manifests, not in App packages (charts), which makes the design bad - dependency and version control happens in a different place than chart build itself.

So, as for App CRs, I'm happy to discuss it with you (and anybody else), as this is my focus as well. Currently, I see these choices, but I'm really not sure which one is better:

ditch App CR and app-operator completely, try to get the same features in Helm+Flux (possibly extending them)
invest more in App CR, including features that are not available and harder to get in Helm+Flux

Long term, leaving stuff as it is, is the worst option to me. WDYT?

puja108 commented 2 years ago

I'd really like it if we could evaluate the two options you mention in more detail. Especially, looking into how much work it would be to bring the 3-4 added value functionalities of current/future app platform to upstream Helm+Flux. Here, it is not just about just adding features to upstream projects (as those sometimes want to keep some focus), but maybe being able to build it in a plugin manner that directly extends upstream and that could be used as a Flux extension by more than just us. Not sure how well Flux is built for plugins/extensions, but that might be a good thing to talk to upstream about at KubeCon for example.

teemow commented 2 years ago

Thank you @piontec! Great explanation and context. I definitely agree with short term vs long term and what @puja108 said.

gianfranco-l commented 2 years ago

the concept of App Bundle made clear that app platform architecture should be rediscussed. The conversation will happen in this ticket

on the naming, we settled down for now on the name App Bundle. To remain within the original scope of this ticket, @piontec what would be the best next step to consolidate this decision? ADR?Docs?

piontec commented 2 years ago

PDR - product decision record, I guess? But I'm not sure if we go into such effort for just agreeing on a name of something. let's rather add it to one of our doc pages. Does it sound OK to you?

MarcelMue commented 2 years ago

I think a PDR is justified considering that we had 40+ comments here and multiple discussions on the topic.

gianfranco-l commented 2 years ago

we agreed in @giantswarm/team-honeybadger that both PDR (to formally and officially consider this decision done and agreed) and docs (to bring knowledge internally and externally-where makes sense-) would be needed ideally. Realistically (given current team bandwidth), @piontec will figure out what is the best possible effort to do both PDR and docs in the most efficient way, will inform this thread and work on that to finalize this work

piontec commented 2 years ago

Relevant PDR has been created: https://github.com/giantswarm/giantswarm/pull/22268. Please check there for last minute comments and adjustments. Otherwise, we're proceeding as described in the PDR.

giantswarm / roadmap