apache / pulsar-client-python

Apache Pulsar Python client library
https://pulsar.apache.org/
Apache License 2.0
53 stars 43 forks source link

Use Protobuf as schema registry for python client #49

Open zhavir opened 3 years ago

zhavir commented 3 years ago

Is your enhancement request related to a problem? Please describe. I was wondering if there was a possibility to use protobuf in order to auto-generate code for my pulsar client services (both producer and consumer). By reading documentation and searching on the internet I was able to find some references related to other languages (like Java or C).

Describe the solution you'd like I've tried to do something like that, obviously without success.

import _pulsar
from pulsar import Client
from pulsar.schema import Schema

from generated_schema.test_pb2 import TodoList

class ProtobufSchema(Schema):
    def __init__(self, record_cls):
        super(ProtobufSchema,
              self).__init__(record_cls, _pulsar.SchemaType.PROTOBUF, <IdontKnwoHowToGetTheSchema>,
                             'PROTOBUF')

    def encode(self, obj):
        self._validate_object_type(obj)
        return obj.SerializeToString()

    def decode(self, data):
        return self._record_cls.FromString(data)

def test():
    client = Client('pulsar://localhost:6650')
    producer = client.create_producer(topic='persistent://cloudacademy/accounts/sso-configuration-updated.v1',
                                      schema=ProtobufSchema(TodoList))

    producer.send(TodoList(owner_id=1, owner_name='test'))

Additional context For a reference, I'm expecting something like the integration with Avro, but instead with Protobuf (with autogenerated classes)

github-actions[bot] commented 2 years ago

The issue had no activity for 30 days, mark with Stale label.

factoidforrest commented 2 months ago

Was this ever implimented?

bpereto commented 2 weeks ago

it seems to be present in cpp, which is the base for the python client: https://github.com/apache/pulsar-client-cpp/blob/3efa80a2667aae599118f27b8741c8236b2386c8/lib/ProtobufNativeSchema.cc#L36

so, extending should be trivial.