[Parent issue] Support for Apache Pulsar

[hpvd]

Since there is a continious demand for it, just an overview for current state of Support for Apache Pulsar:

• when looking into beams i/o connector matrix pulsar is not listed yet: https://beam.apache.org/documentation/io/connectors/

• but there is a sdk package Package org.apache.beam.sdk.io.pulsar https://beam.apache.org/releases/javadoc/current/org/apache/beam/sdk/io/pulsar/package-summary.html

• in v2.39 of beam a relatively basic pulsar io was introduced: Implemented Apache PulsarIO https://beam.apache.org/blog/beam-2.39.0/ for details see: https://issues.apache.org/jira/browse/BEAM-8218

• there are some open issues in beam for pulsar: https://github.com/apache/beam/issues?q=is%3Aissue+pulsar+is%3Aopen

  • 30688

  • 30152

  • 22054

  • 22027

  • 22025

  • 21714

  • 21560

  • 31079

• there were also some non merged pull request marked as stale and automatically closed, which may be worth to have a look on https://github.com/apache/beam/pulls?q=is%3Apr+pulsar+is%3Aclosed+label%3Astale

  • 26410

  • 22375

  • 22028

  • 22026

  • 17473

  • 16749

• in general it would be a good longterm target, to reach the same quality of integration in Beam as there is for Kafka

[hpvd]

regarding the beginning of pulsar integration from https://www.youtube.com/watch?v=L_Pm2yKgA6g

[damccorm]

Hey @hpvd - have you been able to make any progress on this issue? I agree that the current state of the IO isn't great right now, so there's some good contribution opportunity :)