search

zalando / nakadi

A distributed event bus that implements a RESTful API abstraction on top of Kafka-like queues
https://nakadi.io
MIT License
952 stars 293 forks source link

Expose partition in BatchItemResponse (useful for 207s) #1584

Closed fghibellini closed 5 months ago

fghibellini commented 5 months ago

One-line summary

Expose partition in partial success reponses.

Zalando ticket : ARUHA-1619

Description

Currently if an event fails to be published it returns a response of the following format:

HTTP/1.1 207 Multi-Status
...
[
  {
    "eid": "669B7E68-7654-4733-B3A9-BF36B41E4E59",
    "publishing_status": "submitted",
    "step": "publishing",
    "detail": ""
  },
  {
    "eid": "26f29c33-a9e2-421f-80fb-1696ee2d60d9",
    "publishing_status": "failed", <-- NOTICE: failed event
    "step": "publishing",
    "detail": "internal error"
  }
]

with this change each event's response record will contain a partition field with the partition number i.e. the response would look like:

HTTP/1.1 207 Multi-Status
...
[
  {
    "eid": "669B7E68-7654-4733-B3A9-BF36B41E4E59",
    "publishing_status": "submitted",
    "partition": "3", <-- newly exposed field
    "step": "publishing",
    "detail": ""
  },
  {
    "eid": "26f29c33-a9e2-421f-80fb-1696ee2d60d9",
    "publishing_status": "failed",
    "partition": "7", <-- newly exposed field
    "step": "publishing",
    "detail": "internal error"
  }
]

this will inform the users about the unavailability of some partitions, possibly allowing them to forward the events to other partitions (in some scenarios).

Review

Deployment Notes

These should highlight any db migrations, feature toggles, etc.