jsquire
commented
4 years ago Revising the estimate based on found work discovered through iteration, review, and discussion with various stakeholders.
Revised Estimate: 7 Original Estimate: 3
Closed jsquire closed 4 years ago
jsquire
commented
4 years ago Revising the estimate based on found work discovered through iteration, review, and discussion with various stakeholders.
Revised Estimate: 7 Original Estimate: 3
Summary
When publishing events to Event Hubs, timeouts or other transient failures may introduce ambiguity into the understanding of whether a batch of events was received by the service. Because Event Hubs has an at-least-once guarantee for delivery and consumers are strongly encouraged to be idempotent in their processing, the common approach is to resend any batch where the status of receipt was unknown.
In some specialized scenarios, producers have a need to make efforts to avoid publishing duplicate events. To support these scenarios, the Event Hubs service is adding support for annotating messages with metadata to indicate the sequence in which events were intended to be published and to use that as an indicator of which events were already received by a the target partition. This functionality comes at a performance cost, however, and requires producers to follow a strict set of semantics to allow the service to perform server-side deduplication based on the intent of the producer.
Scope of Work
Draft a design proposal that captures the requirements for Event Hubs clients to opt into the "Idempotent Producer" feature offered by the Event Hubs service and meet its requirements for publishing.
Once the design has been discussed and approved with stakeholders and interested parties, create the issues needed for its implementation and provide the level of effort estimation for scheduling.
Success Criteria
An design has been created which illustrates the proposed approach and details any public-facing API changes.
The design has been shared within the Event Hubs feature team, the Event Hubs service team, and with the .NET architect for feedback; any tweaks or iterations to the design have been captured and the design has been updated.
If moving forward, the changes for any public API surface have been approved by the .NET language architect.
Issues to track any work that results form the design/review have been filed and triaged.