shinyscorpion / task_bunny

TaskBunny is a background processing application written in Elixir and uses RabbitMQ as a messaging backend
MIT License
202 stars 30 forks source link

Add error log to message body #17

Closed ono closed 7 years ago

ono commented 7 years ago

We would like to add failed logs to the message so it helps identifying issue instantly.

Here is the example message.

{
  "payload": {
    "name": "Joe"
  },
  "job": "TaskBunny.MessageTest.NameJob",
  "errors": [
    {
      "result": "{:error, \"HTTP Request error\"}",
      "pid": "#PID<0.244.0>",
      "failed_at": "2017-02-23T10:08:51.366951Z"
    }
  ],
  "created_at": "2017-02-23T10:08:51.356652Z"
}

The errors are sorted in chronological order (order to newer).

To add error message, we stopped using dead letter exchange on worker queue. You need to reset existing queues after this change because of that.

We still use dead letter exchange on retry queue. However it no longer has expiration time on queue layer and it's set to each messages. It will allow us to write more complicated retry logic like this in future.

IanLuites commented 7 years ago

👍

Only thing is: the inclusion of the pid is dubious. The pid is not shared over instances and seems language specific. (On the other hand it also doesn't hurt.)