Apache Pulsar support

[devlap2]

Expected Behavior Many data pipelines technologies (like Spark\ Flink\ Camel) offer Apache Pulsar interactions. I think this should be supported within Spring Integration either.

Current Behavior No support.

Context A user of Apache Pulsar. I'd like there to be a ReactivePulsarInboundChannelAdapter, and a ReactivePulsarOutboundChannelAdapter. Synchronous channel adapters can be a nice addition, but personally I won't use them.

[artembilan]

Right. The reactive channel adapter can be turned to imperative very easy - block(), subscribe() etc. So, I suggest the name as plain as possible: PulsarMessageHandler and PulsarMessageProducer. We cannot call them "channel adapter" since this particular impls are not endpoints to claim the name. We call "channel adapter" a combination of components: an endpoint and specific MessageHandler impl. So, technically there is no a ChannelAdapter component. This is a logical entity to align with specific EI pattern. Therefore don't call target protocol impl classes as ChannelAdapter: they are going to extend specific class - MessageProducerSupport for inbound side and AbstractReactiveMessageHandler for outbound.

See for example: ZeroMqMessageHandler and ZeroMqMessageProducer.

I know there are many classes in the framework with a ChannelAdapter in their name, but the can is out of the bag for decade already. So, it is hard to rename them at the moment.

The contribution is welcome: https://github.com/spring-projects/spring-integration/blob/main/CONTRIBUTING.adoc !

[markusherbert]

Support.

I also agree with @artembilan about the fact that both the reactive and non-reactive channel adapters can be the same channel adapter, but I do understand why you suggest the separation. I think the will of Reactive Spring Integration users to use ReactiveXMessageHandler is because it's pretty hard to understand if a channel adapter is reactor friendly or not. The solution can be pretty simple. I'll open an issue on the subject.

[artembilan]

I think we can clear state that in the Javadocs of the class and in the docs. Not sure if you are going to use a new feature without reading about it to determine what does it expect as an input, what it does internally, and what is the result (if any).

[markusherbert]

@artembilan I don't exactly agree about the java doc since this is a thing that people should infer from the reference, because this is the place where people start to read about the library and this is the place where the information about stack choosing should sit. The javadoc should contain details about implementation. Reactive support is more important than this since people determine based on this whether to use this class or not.

[artembilan]

FYI, we have started some effort with Spring for Apache Pulsar: https://github.com/spring-projects-experimental/spring-pulsar.

Feel free to give some feedback and contribute!

//CC @sobychacko

[rolkhas2]

@artembilan will the Pulsar channel adapters get released in Spring Integration 6?

[artembilan]

Hi @rolkhas2 ,

I don't think so. The Spring for Apache Pulsar is in early stages, so it is unlikely the channel adapters on the matter will make it into the currently expected 6.0 this Fall. On the other hand we can make an experimental extension project in the https://github.com/spring-projects/spring-integration-extensions with its own release cycle. But again: we are short-handed here, so the contribution is welcome. Otherwise I don't make any promises in the foreseeable future.

[artembilan]

Some source for inspiration: https://github.com/lhotari/pulsar-spring-cloud-stream-binder

[artembilan]

Cross-link with Spring for Apache Pulsar: https://github.com/spring-projects/spring-pulsar/issues/137.

Not clear yet where those channel adapters are going to be implemented.

/CC @onobc

[onobc]

We ended up implementing the SCSt binder leveraging the SB auto-configuration. As such, we are not sure when we will get to implementing the channel adapters. Most likely not until after we GA. I would consider this on-hold for now.

[artembilan]

Having Spring Pulsar promoted to official Spring IO repository we are good to have a contribution for channel adapters in Spring Integration. We are more than welcome to accept such a contribution: https://github.com/spring-projects/spring-integration/blob/main/CONTRIBUTING.adoc.