Promotion Directives could populate metadata into status

[jessesuen]

Proposed Feature

We should provide a way for directives to produce metadata in a status field. For example, the argocd-update step could populate application names and/or URLs:

status:
  metadata:
    argocd-update:
      applications:            # the structure under the directive name would be free to update
      - name: guestbook-a
         url: https://argocd.example.com/applications/argocd/guestbook

Motivation

1. We wish to provide 3rd party directives equal footing to native directives (such as presenting information into the UI)
2. We recently lost the ability to use wildcards for authorized-stages. The reason was partly technical since application name references were no longer easily discoverable. But if we allowed directives to populate information into a well known status location, it would restore our ability to index on them.

Suggested Implementation

The directive interface should have the ability to return stage metadata that we could store in the stage's status.

[krancour]

Steps can already produce output that is already part of the PromotionStatus:

https://github.com/akuity/kargo/blob/b21da75271229ed35f5df3343db6d474a0a5354a/api/v1alpha1/promotion_types.go#L149C2-L149C7

I'm not sure if you are asking for this to be part of StageStatus instead? If so, it also has this already via status.lastPromotion.status.state.

We wish to provide 3rd party directives equal footing to native directives (such as presenting information into the UI)

Despite this already existing, I am iffy on how it allows for built-in directives not to be treated as special. Ultimately, to make use of data that's saved in this map, don't other parts of Kargo need to have the intelligence to go looking for information they know built-in directives may have left behind?

[jessesuen]

I'm not sure if you are asking for this to be part of StageStatus instead?

I am. In fact, the motivation of this issue was based on https://github.com/akuity/kargo/issues/2746#issuecomment-2412656296.

If so, it also has this already via status.lastPromotion.status.state.

lastPromotion would be too ephemeral for my liking. The motivation is to drive UI features, like a deep link. We would want the ability to store it some place more formal/assured.

don't other parts of Kargo need to have the intelligence to go looking for information they know built-in directives may have left behind?

Absolutely. So this feature would probably only be useful in conjunction with a generic UI deep link or UI extension feature.

[krancour]

Ok... I might be starting to understand...

lastPromotion would be too ephemeral for my liking.

don't other parts of Kargo need to have the intelligence to go looking for information they know built-in directives may have left behind?

Absolutely. So this feature would probably only be useful in conjunction with a generic UI deep link or UI extension feature.

populate information into a well known status location

☝️ I should have zeroed in on this more. ☝️

I think some confusion may have been that these requirements might not align with the example:

status:
  metadata:
    argocd-update:
      applications:            # the structure under the directive name would be free to update
      - name: guestbook-a
         url: https://argocd.example.com/applications/argocd/guestbook

In the example above, something would need to know that it was the argocd-update step that left this information behind. Suppose that, in fact, this information were left behind by an alternative, third-party Argo CD update step. The location of the information would be different.

If the path to the information were not contingent on what left it behind, e.g. argocd-update, then the location is truly well-known. Are we converging?

[hiddeco]

What is the advantage of reflecting (more and more) data from Promotions into the Stage, over making the frontend learn how to collect the data directly from the (latest) Promotion(s)?

[jessesuen]

To be frank, I'm not 100% convinced about my own idea.

The big picture is that we want Stage specific information surfaced in the UI in an extensible manner. As of now, our proposed extension point is 3rd party promotion directives, but it could be something else that we haven't yet thought of.

In fact, ideally, the way we allow third parties to surface stage-specific metadata should not even require a promotion to have happened, which is a flaw that I see in my proposal. The only reason I suggested allowing promotion steps populate metadata into status, is because it's the only extension mechanism that we are thinking about at this time. If we think of another way, I'd be more than happy to amend this proposal/close it.

[Brightside56]

In the example above, something would need to know that it was the argocd-update step that left this information behind. Suppose that, in fact, this information were left behind by an alternative, third-party Argo CD update step. The location of the information would be different.

Directive could write information to promotion status field and there could be another directive which allows to propagate/transform arbitrary promotion status field into stage status field taken by UI