Adding metrics to Swift Jobs

[thoven87]

Here's my first attempt at adding Observability to the Swift jobs project.

I would love some feedback on how to make metrics better. cc @adam-fowler

Since everyone might use different tools to display their dashboards, I will attempt to provide a grafana dashboard as it is what I am most familiar with.

Lastly, most of the tests are a compilation of the current job tests and Hummingbird metrics tests.

[thoven87]

Here are some initial comments. In general this look good, I was wondering if it is worthwhile adding a metric for the retry count.

For another PR if we added a dateCreated to the QueuedJob, we could add a time job queued metric. In a way dateCreated is probably a useful piece data regardless.

Thanks, for the feedback. I added a retry counter. I will open another PR to add both createdAt and maybe Job name? Right now, Job.id.name does not always exist in every context. In some context, it's just Job.id which is a UUID.

[Joannis]

Looks great! Just one final bit of consistency

[thoven87]

I think this Pull request is ready. Below is a dashboard I created to test this implementation.

Next step would be to add tracing and queued at to compute avg wait time.

[thoven87]

also previous implementation of the dashboard to show the failed metrics

[thoven87]

This is great

Please merge at your convenience! What's the best place to share RED metrics dashboards? I am in the process of creating a few Grafana dashboards for Hummingbird.

There would be three dashboards in total:

1 - Display the overall health of an application including jobs and http metrics, spans and error logs. 2 - Display Http metrics, spans and error logs 3 - Display Jobs metrics, spans and error logs

[adam-fowler]

Just waiting on a final approval from @Joannis

[Joannis]

@thoven87 I'd love to post them either in a docs section, on docs.hummingbird.codes as a DocC guide. I think we can also highlight it on swiftonserver.com

[thoven87]

@thoven87 I'd love to post them either in a docs section, on docs.hummingbird.codes as a DocC guide. I think we can also highlight it on swiftonserver.com

@Joannis I suppose for DooC guide it would need to be added in a section in this repository Indeed, it would be great to highlight it on swiftonserver.com as well.

[adam-fowler]

You could possibly include something in the observability docs but https://docs.hummingbird.codes/2.0/documentation/hummingbird/loggingmetricsandtracing.

I think an end to end metrics and tracing article in swiftonserver.com which uses these would be a great addition.

[thoven87]

You could possibly include something in the observability docs but https://docs.hummingbird.codes/2.0/documentation/hummingbird/loggingmetricsandtracing.

I think an end to end metrics and tracing article in swiftonserver.com which uses these would be a great addition.

Thanks Adam, I will take a look and see what I can do.