Azure Event Hubs Commitments for Python (July 2021-March 2022)

[AlexGhiondea]

Azure Event Hubs is a highly scalable publish-subscribe service that can ingest millions of events per second and stream them to multiple consumers. This lets you process and analyze the massive amounts of data produced by your connected devices and applications.

The azure-eventhub package provides the SDK for Python developers to write applications that use Azure Event Hubs. Version 5 of this package follows the latest guidelines for writing Python SDKs for Azure services with the goal of being a client library that is developer-friendly, idiomatic to the Python ecosystem, and as consistent across different languages and platforms as possible.

This issue tracks the work we have planned for this package for the duration of July - December 2021

New features

• [ ] Design & implement a buffered producer that allows users to queue events to be sent with implicit batching that happens behind the scenes
• [ ] Add the idempotent producer feature as described in https://gist.github.com/jsquire/1cc4db6b3ca4ef13d26dc5315483555b to the new buffered producer
• [ ] Release the package that has the checkpoint store that uses Azure Tables
• [ ] If Service Bus plans on the backend redirect feature, design a common model if possible between that and the receiver redirect feature in Event Hubs

Testing improvements:

• [ ] Update stress tests to use the new infrastructure where we now containerize the tests and have automation to run the tests on a scheduled basis
• [ ] Update perf tests to use the perf test framework that will then allow them to be automated

General improvements:

• Fix incoming customer reported issues
• Build Trouble Shooting Guides
• Review and improve logging story

[ramya-rao-a]

Re-opening as we are planning to re-use this issue to track the extended semester work