OpenTelemetry would provide the ability to grab metrics and tracing from mechanic and surface it, if desired, for better insight into what's going on inside. It'd also offer the ability to use correlation IDs to associate logs back to specific triggering node updates.
Update 2024-Oct-2
Adding some more detail here on how this implementation probably looks:
Tracing is enabled/disabled based on a configmap toggle. The configmap only toggles whether the exporter is no-op or stdout. This also potentially enables support for multiple exporters in the future if.
There's no tracing for the main function - earliest tracing start is on the first node update received and the spans start and stop based on the update func execution.
We can put a few milestones together based on the above:
[ ] Update mechanic config to toggle tracing output
OpenTelemetry would provide the ability to grab metrics and tracing from mechanic and surface it, if desired, for better insight into what's going on inside. It'd also offer the ability to use correlation IDs to associate logs back to specific triggering node updates.
Update 2024-Oct-2 Adding some more detail here on how this implementation probably looks:
We can put a few milestones together based on the above: