Display Trigger Logs in the UI

[anna-geller]

This is mostly a UI issue, with only small enhancements needed on the backend side (to expose + filter the logs).

Triggers can sometimes be difficult to troubleshoot. Imagine a scenario: you configured a new trigger but it doesn't generate any executions. What happened? There are many potential culprits and questions you may ask yourself:

• Did you enter incorrect credentials?
• Did you enter the wrong S3 bucket name or path?
• Did the trigger fail to detect the new file or event?
• Is the trigger even running?
• What's the last time the trigger polled by bucket for new files? How many files did it detect?

It's difficult to answer these questions without additional observability to the trigger's behavior. That's where Trigger Logs come in handy.

Proposal

For each trigger, we should add an expand section in the UI that displays the trigger logs.

Implementation

We already log a lot for each triggerId, but we'll need to:

• reevaluate what's logged and on which level
• expose this information in the UI (by adding a filter on the backend side)

1. Add a new log field to the trigger schema:
   • This field will be a boolean that enables or disables logging for the trigger.
   • By default, logging will be disabled for Realtime Triggers to avoid privacy concerns (as Realtime Triggers process/store data directly rather than through internal storage).
   • For other triggers, logging will be enabled by default.
2. Add a new logLevel field to the trigger schema:
   • This field will specify the log level for the trigger.
   • By default, the log level will be set to TRACE for allTriggers.

Next steps

• [x] @Nico-Kestra, can you add a Figma design for the trigger logs in the UI? Not sure whether "expand" or a button/link pointing to a new Trigger Logs page/tab would be better. Ideally, we should add it to both, the Triggers tab on the Flow page and the Administration Triggers page.
• [ ] add the log and logLevel fields to the trigger schema
• [ ] add the log and logLevel fields to the low-code editor for triggers
• [ ] for Realtime Triggers — only log on INFO level when the trigger starts polling + every hour a heartbeat-style log showing the trigger is still active; given Realtime Triggers process data directly, we should set a higher log level by default e.g. INFO rather than TRACE
• [ ] for Schedule Triggers — here, we only log when the trigger starts and on errors

[tchiotludo]

• the expandable state should the one already available on element-plus
• we will display the standard log that is already available on Logs page (just with a auto filter)
• We should have pagination also

[anna-geller]

as discussed @loicmathieu, ignore extra fields like the log field for Realtime Trigger if not needed

[anna-geller]

@loicmathieu for the UI: https://www.figma.com/design/ew0uXk0NRXJ2NBBJTNe2n1/01_UI?node-id=1446-8466&t=0n2sGBtfHm0IDPFq-4

[loicmathieu]

@MilosPaunovic, what needs to be done is to add an expand button (already available on element-plus) to the triggers page that will display the logs of the trigger as seen in the above screenshot.

There are 3 places where we list triggers:

• Flow -> Triggers
• Administration -> Triggers
• Administration -> Cluster monitor -> click on a service (TBD, see https://github.com/kestra-io/kestra-ee/issues/1299)