search

bensheldon / good_job

Multithreaded, Postgres-based, Active Job backend for Ruby on Rails.
https://goodjob-demo.herokuapp.com/
MIT License
2.71k stars 203 forks source link

[Feature request] OpenTelemetry - Link job execution to the request which queued it #1342

Open DanielHeath opened 7 months ago

DanielHeath commented 7 months ago

OpenTelemetry can link events which are causally related, and those links are damn useful for diagnosing weird bugs. Sidekiq integrates this nicely.

To support that, I would like to store metadata in a before_enqueue hook, so that I can link the execution of the job to the request that enqueued it (and then e.g. how long it took to run). This is different from serialized_params in that the base job class should handle it.

Storing this on good_jobs as a nullable column should add no storage overhead for users who don't use it, because the null columns bitmap doesn't need to grow to store a 23rd column. I would suggest either metadata as a jsonb column, or links as an array of text.

bensheldon commented 7 months ago

This sounds like it could be stored in serialized_params (arbitrary data that's stored at enqueue and only used during perform). Here's an [example](https://github.com/bensheldon/good_job/discussions/901#discussioncomment-5394194) of serializing and deserializing attributes into that hash (they don't have to be passed viaset` but can be grabbed via a global or wherever).

I really like OTel, but I don't see a future of adding new db columns that are solution-specific.

tjefferson08 commented 7 months ago

@DanielHeath my application has the behavior you've described, we're using opentelemetry-ruby-contrib's provided instrumentation for ActiveJob configured with the default :link propagation