knative / eventing

Event-driven application platform for Kubernetes
https://knative.dev/docs/eventing
Apache License 2.0
1.42k stars 599 forks source link

Trigger: add customized fields into cloudevent extension after the event passes a trigger filter #7704

Open xiaoajlszt opened 9 months ago

xiaoajlszt commented 9 months ago

Problem broker --> trigger1 --> app --> ... --> trigger2 --> app --> ... --> trigger3 --> app --> ...

Similar to the event delivery process above, since we reuse apps (three apps are actually one app), we need to know the location of the event (that is, which app in the workflow) based on some fields in the event. Then we obtain the corresponding configuration according to the location to process the event, and continue to deliver the event.

So I hope to add customized fields into cloudevent extension after the event passes a trigger filter. Maybe You can add a kv map in the trigger spec. thx!

Persona: Which persona is this feature for?

Exit Criteria A measurable (binary) test that would indicate that the problem has been resolved.

Time Estimate (optional): How many developer-days do you think this may take to resolve?

Additional context (optional) Add any other context about the feature request here.

pierDipi commented 9 months ago

For sources, we have something similar to what you're describing https://knative.dev/docs/eventing/sources/apiserversource/reference/#cloudevent-overrides but it's not based on some fields in the event.

Can you expand a bit more on the based on some fields in the event with a few examples?

xiaoajlszt commented 9 months ago

For example, when we handle an abnormal event, we usually need to go through multiple processing steps, which form a workflow.

We may reuse some steps (these steps are run by the same pod, while each step has its own configuration).

During the delivery of events, we save the current step's position(in the workflow) in the cloudevent extension, and the pod can obtain the configuration of current step by the position in the cloudevent.

Cali0707 commented 6 months ago

Hey @xiaoajlszt - in your use case, do you send the new event back to the broker by replying to the original request sent from the trigger, or is it a fully new request to the broker?

pierDipi commented 5 months ago

This feature might be similar to or combined with https://github.com/knative/eventing/issues/8001

if we allow you defining lightweight transformation on a JSON represented event using JSON path, I think, it would solve both:

ceOverrides:
  jsonTransform:
    - from: .data.myfield
      to: myextensionfield
    - from: .data.status.conditions[...].type
      to: completed
Cali0707 commented 5 months ago

@pierDipi another idea could be to use CESQL to calculate the new values rather than JSON path

pierDipi commented 5 months ago

@Cali0707 is CESQL able to inspect the data too? in #8001, they need to inspect/use the data field

Cali0707 commented 4 months ago

@pierDipi checking the data field in CESQL is a desired addition that has been discussed for a v1.1 in the future (only supporting e.g. JSON data). But, this hasn't started design yet, and no one is driving that as of now

github-actions[bot] commented 1 month ago

This issue is stale because it has been open for 90 days with no activity. It will automatically close after 30 more days of inactivity. Reopen the issue with /reopen. Mark the issue as fresh by adding the comment /remove-lifecycle stale.

pierDipi commented 1 week ago

/remove-lifecycle stale