allow multiple connections for client

[jangalinski]

The way we use the ZeebeClientBuilder bean today, a spring boot application will only be able to connect to one broker. It should be possible to connect to multiple brokers (especially since we do not have multiple topics anymore).

Todo: do not use builder as a bean to avoid singleton, support st. like BeanFactory.

Motivation: I could imagine situation where a service (lets say responsible for invoicing) acts as a zeebe client for different context (it might subscribe to invoicing jobs from department a and department b that run arbitrary processes on different zeebe clusters).
In such a case, I could of course run multiple instances of my invoice service, some connected to a) some connected to b), but being able to have multiple workers in a service that are connected to different clusters would allow more flexibility ...

[salaboy]

I do believe that this might be an use case that we want to cover.. but I wonder how the implementation will look like.. From the top of my head I see worker names connecting to different brokers.. and for that it sounds that we will need to have a way to map those things together and then as you mention instantiate a bean for each connection

[cameronbraid]

I need this feature for the following use case.

I have an in production zeebe 0.26 broker which I currently connect to from a java app.

I am about to install a 1.x broker along side this, and will keep the 0.26 broker running until all its workflows have terminated.

So I need my app to connect to both brokers at the same time.

Can it be achieved by the following changes

• Make @ZeebeWorker repeatable (so I can apply it twice to a service method)
• Use a different @ZeebeWorker.name for the new broker
• Introduce a new per worker name zeebe configuration

So my service would look like this

  @ZeebeWorker(type = CHECK_AVAILABILITY_TASK_TYPE, timeout = 1000 * 60 * 10)
  @ZeebeWorker(name="new", type = CHECK_AVAILABILITY_TASK_TYPE, timeout = 1000 * 60 * 10)
  public void checkAvailability(final JobClient jobClient, final ActivatedJob job) {

I would then configure like :

zeebe:
  client:
    worker:
        defaultName: old
    namedWorkerConfig:
        old:
          broker:
            contactPoint: zeebe-zeebe-gateway-old:26500
          security:
            plaintext: 'true'
        new:
          broker:
            contactPoint: zeebe-zeebe-gateway-new:26500
          security:
            plaintext: 'true'

[berndruecker]

Hey @cameronbraid - thanks for sharing that use case! In general, we plan to allow upgrading your broker (OK - not 0.x - but upcoming versions), sO I think this is less required. Given that it increases complexity quite a bit I would hold our horses and wait for more people to request this. I close it for now - everybody is invited to comment here if they also miss the feature - so we might reconsider in the future

[NiteshB2993]

Hi @berndruecker, I am stuck at a similar situation that @cameronbraid has mentioned above. Currently we are using Camunda zeebe 8.2.0 cluster with elasticsearch, and our team has decided to move to 8.5.0 version but with opensearch.

Now, the elasticsearch data migration to opensearch is not supported as per the camunda documentation. So, we need to configure both the clients in our application to handle the existing running processes until they completes.

Is there any way we can achieve this now? Can we configure two brokers for single application?

[berndruecker]

Hi @NiteshB2993: I don't think this is easily possible with spring-zeebe at the moment, however you can of course instantiate two client connections using the underlying Zeebe Java Client yourself. I guess you could also easily use spring-zeebe for one of those connections, but not the second. But of course you can build your own bean managing the second client connection.