apache / pulsar-client-python

Apache Pulsar Python client library
https://pulsar.apache.org/
Apache License 2.0
51 stars 40 forks source link

[Enhancement/Feature Issue #47] Added support for KeySharedPolicy for the consumer when in KeyShared mode. #109

Closed hyperevo closed 1 year ago

hyperevo commented 1 year ago

Motivation

The pulsar python client lacks support for defining KeyShared behaviour like out of order message delivery and sticky-hash, auto-hash for consumers in KeyShared mode. This PR adds full support. The user can now provide a KeySharedPolicy when starting a consumer with client.subscribe() #47

The ConsumerConfiguration::KeySharedPolicy and related setter/getter are now exposed to the Python client in this PR.

Modifications

In order to add this functionality I also had to modify the C++ client slightly. The setter function for setting the sticky ranges in the C++ client takes a parameter of the type std::initializer_list as seen here https://github.com/apache/pulsar-client-cpp/blob/86d66bdb09bdb596a2a5c25b0a09f3d0182d683e/include/pulsar/KeySharedPolicy.h#L99. But std::initializer_list types are not supported and probably won't ever be supported by pybind11. See https://github.com/pybind/pybind11/issues/1302#issuecomment-369090893. So I added an overloaded setter function that takes a parameter of the type std::vector. This modification enables us to have python bindings for KeySharedPolicy. I have submitted a PR to the C++ client here https://github.com/apache/pulsar-client-cpp/pull/242.

This PR won't work until that referenced C++ client PR is merged.

Verifying this change

Added 4 new unit tests. Ran them to ensure they pass.

Documentation

Added documentation in the form of comments and docstrings to the new classes and parameters that were added.

hyperevo commented 1 year ago

@BewareMyPower Would you be able to help review this PR along with the dependent PR on the C++ client? I see that you are a contributor to both repos.

BewareMyPower commented 1 year ago

Sure.

hyperevo commented 1 year ago

Thanks!

hyperevo commented 1 year ago

Now that the pulsar cpp client included the code I added in the newest 3.2.0 release, the fixes in this PR will be compatable with it.

BewareMyPower commented 1 year ago

The Oauth2Test.test_base64 seems very flaky, I will rerun it for now.

BewareMyPower commented 1 year ago

@hyperevo I've left my comments here. Since we're going to release 3.2.0 soon (https://lists.apache.org/thread/4ryrobppjl93gcml63hcnpnjkrn6lrtn), please address the comments ASAP if you want this PR to be included in 3.2.0.

hyperevo commented 1 year ago

@BewareMyPower All the comments should be resolved now.